File Watcher Monitoring (draft)

IT-Conductor File Watcher facilitates application integrations by tracking file exchanges between business partners, clients, providers, etc. This monitoring mechanism enables seamless communication through file sharing, where entities post files into designated directories for application consumption.

The file watcher functionality covers SMB (Samba) File Server environments, enabling users to monitor shares, directories, files, their existence, size, and modification time. By setting up alerts, users can ensure that specific conditions regarding the files, such as timely arrival, are met. Users can trigger automated processing and enable event-driven integrations by setting up recovery actions. This proactive monitoring and automation approach enhances operational efficiency and helps maintain compliance and data integrity across integrated systems.

Pre-Requisites

Set up a file server for file management and watcher in IT-Conductor. See File Server for more details.

Configure File Watcher Monitoring in IT-Conductor

To configure the file watcher monitoring in IT-Conductor, follow the instructions below.

Add New File Retriever

Once the file server has been added, perform the following steps to set up folders and files periodically to pull from the file server into IT-Conductor.

1. Visit service.itconductor.com and enter your login credentials.

2. Navigate to the file server node and click Retrievers.

Open

Figure 1: File Server Node in Service Grid

2. Click to add a new retriever.

Open

Figure 2: Existing Retrievers in the File Server

3. Fill out all the necessary information in the Create File Retriever wizard. Once completed, click to add the system.

• Name - refers to the name given to the retriever.

• Description - refers to any relevant information about the retriever being added.

• Repeat interval - refers to the frequency with which each retriever pulls files into IT-Conductor. By default, this is set to every 15 minutes. However, depending on the sensitivity of the information, this can be adjusted as needed.

• Share - refers to the share that will be pulled into IT-Conductor.

• Folder - refers to the folder's name inside the share that will be pulled into IT-Conductor.

• File Name/Mask - refers to the file names that will be pulled into IT-Conductor. You can use regex in this field. (Optional)

After saving, it’ll take a few minutes for IT-Conductor to pull the files in that share and folder.

Add New File Watcher

By setting up the file watcher, you specify how folders and files periodically pulled from the file server are monitored.

1. On the file server node, click File Watchers.

2. Click on Create New Object to create a new file watcher.

3. On the following screen, fill out the following details

   1. Name - refers to the name assigned to this file watcher.

   2. Description - refers to any relevant information about the file watcher being added.

   3. Application - refers to the file server.

   4. File Mask -This is an optional file selector - a regular expression (Regex) instruction where you specify what files will match the criteria. In the simplest form, specify the full path of the file, but make sure you escape special characters such as []().:\^$ with '\' as: \[, \], \(, \) etc. For more advanced options, please refer to online regex tutorials and testers such as https://www.freeformatter.com/regex-tester.html

4. Click to save.

5. Click the recently created file watcher. Then click the Threshold Overrides icon to create an override to specify what the file watcher will monitor.

6. On this page, you may see the previously created overrides for monitoring. Click to create a new override, or click to create an override from an existing template. In this example, we’re creating a new override.

7. Fill out all the necessary information in the Create New Override wizard. Once completed, click to add the new override.

• Name - refers to the assigned name for the override being added.

• Description - any relevant information about the override being added.

• Object Criteria - refers to the specific attributes that will be monitored. If you add the folder path in step 3, it will appear here. Under Object Criteria, you may specify what you want to monitor in this path. To add more criteria, click on the Add New Row icon, then fill in the following fields:

  • Name - refers to the exact criteria that you can monitor. You can choose one of the available criteria from the drop-down menu.

  • Oper - refers to the operator validation (=, !=, >, <, in, regex, NULL, etc).

  • Value - refers to the exact value that will be monitored. This is an open field where you can specify file names or formats for monitoring.

In the example above, we add the object criteria Path, and the operator '=', which means that the override will search for files within the folder path that we add.

• Scheduling - refers to the section where you can specify when the override will perform validation. You may choose to run the override on a specific day and time. If you don't specify a day, it will run daily at the indicated time. Alternatively, you can assign a pre-existing schedule from the dropdown menu.

  • Start - refers to the time when the override will start monitoring.

  • End - refers to the time when the override will stop monitoring.

• Aggregation - refers to the metrics used to define the aggregation values.

  • Aggregation interval - defines the period within which the files are collected and added to the file server.

  • Consecutive interval - refers to the regularity or frequency of occurrences within a defined amount of minutes.

  • Aggregation - refers to the function that will be applied, such as sum, average, count, minimum or maximum.

The Scheduling and Aggregation sections tell the override when to look for files. The example above specifies a time window between 6:30 am and 7:00 am. During that time, the override will start searching for files added to the server in the previous 360 min, as specified in the Consecutive Interval field below.

• Thresholds - refers to the metrics used to define the threshold values.

  • Warning Value - refers to the value that will determine if the threshold was reached. If this field is blank or zero, it means that there’s no data

  • Warning Operator - refers to the operator validation (=, !=, >, <, in, regex, NULL, etc).

  • Warning Severity - refers to the severity status that will be triggered if the validation reaches the Warning Value.

• Alerting - refers to the section where you can indicate when the users will receive the alerts.

  • Alert On - refers to the status that will trigger the alert and notify the users. This is usually set to “Warning”.

  • Alert Message - refers to the message that the users will see. This message is customizable by the user.

The Thereshold and Alerting sections tell the override when to send a message to the user. This example specifies that if there have been 0 posted files in the server, the status of the override will turn to Warning, and if the status is Warning, an alert will be sent to the user.

one last request is to note that a 'blank' value in the warning field means no data (ie. no files matching the pattern)?

8. Click to save.

Monitor Files in IT-Conductor

To monitor the files pulled from the file server, locate the previously configured file server under the File Servers node in the service grid. Click Files to view all the files that have been added to that share.

By clicking on any of the files, you can see the following information from each file:

• Description - refers to any relevant information about the file.

• File name - refers to the name of the file and its extension.

• Path - refers to the path’s full size, including the share and folders before it.

• Size - refers to the file size.

• Last modified - refers to the time and date when the file was last modified or added to the server.

File Watchers

To monitor the existing file watchers, locate the File Watchers node in the service grid.

The number displayed next to the file watcher indicates the number of alerts generated by the override. Click to inspect the activities.

By enlarging the chart you can see the activity of the override. By clicking on the green bars, you can see which files have arrived recently and at what time. The image below is an example of 3 files arriving into the folder path at 8:30pm.

Alternatively, the following image represents an alert being sent to the user’s email because the override didn’t detect any files during the consecutive interval period.