Docs > Configuration > Alert Creation
The following guide explains the whole workflow and you can configure the steps between Simple, Compound, and Programmable Alerts.
You can create as many alert rules as needed in vuSmartMaps.
The system will periodically evaluate these alerts and generate alert notifications when the conditions are evaluated to True. These alert notifications are sent through the configured channels for each alert and send notifications when the conditions are met.
vuSmartMaps Alerts can be accessed by navigating from the left navigation menu (Configure Observability > Alerts).
You can set up a Simple Alert by entering the Basic Details and configuring a single Data Model.
To create a new alert rule, click the ‘+’ button and follow the steps below.
Use this section to configure the descriptive contents of the notifications generated based on this alert.
💡Note: Portions of Summary and Description fields can be dynamically formed using contents from the actual alert document. Use the format specifiers for this purpose. For example, configuring Summary as “Server CPU usage is now %m for %G” will result in an alert notification to contain summary as “Server CPU usage is now 76% for host:1.1.1.1”
Supported format specifiers are:
vuSmartMaps allows users to model their data using a Data Modelling workspace which can then be used for alerting. You can configure one or multiple data models and convert any business logic to alerts.
Select a Data Model from the list based on the requirement and the time for which the Data Model is to be evaluated.
Include the previous steps from Simple Alert Creation, that is, Basic Information and Data Models and continue to configure Logic Conditions explained further.
When using multiple rules, here’s what to keep in mind:
Example: This alert fires when too many HTTP requests (>3%) with a response status between 400 and 599.
Alert notification behavior and notification contents can be modified using the conditions specified here. For example, based on the value of a metric (Metric Condition), you can decide on the email recipients for a notification.
Based on the metric condition, duration, and/or severity of the alert, three types of controls are possible:
The evaluation conditions configured here are executed from top to bottom. The system stops the execution as soon as a condition matches. There are 3 kinds of conditions available in the logical blocks.
💡Note: All three conditions can have an AND or OR relationship among them based on the Match all the following conditions or Match any of the following conditions flags.
Alert Evaluation Conditions:
In some cases, you might need more control over how your alert rule behaves beyond the basic settings like thresholds and alert channels. This is where alert evaluation conditions come in.
For example:
You can configure these additional conditions, as shown in the snapshot below.
As can be seen in the screenshot, the user can configure
💡Note: Evaluation conditions are like a set of rules that the system checks one by one. It starts at the top and goes through each rule. When it finds a rule that matches the current situation, it stops checking further rules and takes the actions specified in that rule.
In other words, the first matching rule in the list is the one that counts, and the system won’t bother with the rest once it finds a match.
Common Use Cases: These options give you a lot of flexibility in how you set up your alerts. We’ll explore these features in more detail here.
Include the previous steps from Simple Alert Creation and Compound Alert Creation, that is, Basic Information, Data Models and Logic Conditions, and continue configuring the Evaluation Script and Alert Controls as explained further.
vuSmartMaps lets you use Python scripts to create programmable alerts. Using an evaluation script, you can generate alerts for breaching any business logic. Below is a typical alert engine execution workflow and where the evaluation script is used.
The evolution script runs after metrics are checked and thresholds are applied, allowing you to customize alert behavior. Apart from implementing business logic to generate the alert, you can also tweak alert notification content and channel settings like who gets notified.
In simple terms, the Python script lets you create detailed and specific alert conditions that fit your needs. You can write this script in the “Evaluation Script” section.
In the evaluation script, you can do the following:
Example: Adding New Fields using Evaluation Script
You can add new fields to the notifications generated by the system using the evaluation script.
For instance, if you need to include a new field or category with values based on the transaction success rate metric, you can achieve this with the following script snippet.
|
success_rate = get_DM_value(D, 1, ‘Success Rate’) if success_rate and success_rate > 90: DYNAMIC_FIELDS[‘category’] = ‘Normal’ else: DYNAMIC_FIELDS[‘category’] = ‘Need Investigation’ RESULT = True |
As can be seen, any field to be added to the notification generated can be specified in the DYNAMIC_FIELDS dictionary with the corresponding key and value.
Common Use Cases: These options give you a lot of flexibility in how you set up your alerts. We’ll explore these features in more detail here.
💡Note: The programming interface is supported exclusively using Python Language.
In this step, you can set up how alert notifications work for this rule. You can configure notification channels, and recipients, enable or disable alarm mode, and control the intervals for active alert rule notifications.
| notification_level: 0 |
Use this section to control the alert notification channels.
Sample SMS – vuSmartMaps Alert APP – UPI KPI-Technical Decline TH:0.5% ACT – 0.52 % Details – FailedAt: XYZ RespCode: U48 AlertTime: 22.06.2020 08:35:33 IST ActiveSince: 12 Hour(s) and 30 min(s) Past Incidents Today – 7 incident(s) spanning 45 Min(s)💡Note: The alert document contents can be seen on the search page by selecting the notification index. The alert document lists the full set of fields that can be included in the SMS message using the placeholder string format defined above.💡Note: To use SMS as an alert notification channel, you need to activate SMS APIs and SMS Gateways in your vuSmartMaps installation. Please reach out to our support team for detailed information on this process.
The system uses a predefined email format for notifications. A sample email notification is shown below.
The default notification email format used by the system can be replaced with a user-configured static text.
💡Note: The use of Slack as an alert notification requires a special activation in the vuSmartMaps installation to enable Business APIs for Slack. Please contact the support team for more details on this.
💡Note: The use of Microsoft Teams as an alert notification requires a special activation in the vuSmartMaps installation to enable Business APIs for Microsoft Teams. Please contact the support team for more details on this.
This section is useful for configuring Settings for vuSmartMaps to integrate with the ITSM system. This will be used to open tickets based on alert conditions and collect details of tickets present in the ITSM system.
Add content that needs to be sent as a message. Use “{{field_name_from_alert_doc}}” as a variable inside content which can be substituted with a corresponding value. Eg: Alert_Name: {{Alert-Rule-Name}} and Severity: {{severity}}. The variable inside the brackets will be substituted with values in real-time.
Runbook Automation: You can attach your alerts to the Runbook automation scripts. The system invokes the configured scripts when the alert condition turns active.
For instance, in a rule monitoring router interfaces, you can configure a script to bounce (restart) an interface when it goes Down.
The script can use information from the alert notification and configuration to make decisions. Below is a sample script template.
|
if __name__ == “__main__”: try: opts, args = getopt.getopt(sys.argv[1:], “”, []) except getopt.GetoptError: sys.exit(1) # Collecting arguments alert_document = json.loads(args[0]) alert_configuration = json.loads(args[1]) mobile_number = alert_configuration.get(‘mobile_sms’, None) sms_content = alert_configuration.get(‘sms_content’, None) print(mobile_number) sms_list = _prepare_sms_body( alert_document, alert_configuration, mobile_number, sms_content) |
A typical alert document passed to the script is shown below:
|
{ “History”: { “Today”: { “Event Count”: 19, “Active For”: 26745.087917 }, “Last 7 Days”: { “Event Count”: 78, “Active For”: 211002.844929 }, “Last 1 Month”: { “Event Count”: 238, “Active For”: 1702128.547532 } }, “Alert-Rule-Evalution-Duration”: “12 Minutes”, “duration”: 300, “alert_id”: “3275183”, “timestamp”: “2020-06-14 06:21:51”, “R1”: { “name”: “BMV For Server CPU Utilization R1”, “M1”: { “value_for_eval_duration”: “0.8537999987602234”, “matched_threshold”: “> 0.01”, “field”: “system.cpu.total.norm.pct”, “color”: “#05a608”, “insights”: “Looks all fine”, “label”: “SYSTEM.CPU.TOTAL.NORM.PCT”, “formatted_value_for_eval_duration”: “85.38%”, “type”: “number” }, “status”: “Available” }, “severity”: “warning”, “summary”: “CPU Utilization for target:10.121.9.56”, “alarm_state”: “Alarm New”, “Alert-Rule-Name”: “Server CPU Utilization”, “group_values”: “10.121.9.56”, “target”: “10.121.9.56”, “tags”: [ “Server Health”, “10.121.9.56”, “DAQ” ], “start_time”: “2020-06-14T06:16:51.000Z”, “Type”: “CPU Utilization”, “@timestamp”: “2020-06-14T06:21:51.000Z”, “group_label”: “Server CPU Utilization 10.121.9.56”, } |
A typical alert configuration is passed to the script is shown below:
|
{ “enableAlert”: true, “enableThrottle”: true, “alertByReport”: false, “activeStartTime”: “00:00:00”, “alertEmailBody”: “”, “alertByTicket”: false, “severity”: “warning”, “advancedConfiguration”: “\”tags\”:\n – \”Server Health\””, “throttleDuration”: 2, “description”: “Process Memory Utilization for %g is %m”, “alertReportList”: “[]”, “enable_ansible_playbook”: false, “ansible_playbook_name”: “”, “summary”: “Process Memory Utilization”, “throttleDurationType”: “hour”, “alertEmailId”: “”, “enable_runbook_automation”: false, “activeAlertCheck”: “”, “alertEmailGroup”: “”, “runbook_script”: “”, “alertByEmail”: false, “ansible_playbook_options”: “”, “ruleLevelThreshold”: “”, “title”: “Server Process Memory Utilization” } |
Once you’ve written your runbook script, place it in the specified location on the shipper, and then restart the alert container.
Example 2:
|
import smtplib import getopt import sys import json if __name__ == “__main__”: try: opts, args = getopt.getopt(sys.argv[1:], “”, []) except getopt.GetoptError: sys.exit(1) # Collecting arguments alert_document = json.loads(args[0]) alert_configuration = json.loads(args[1]) alarm_state = alert_document.get(“alarm_state”) alert_id = alert_document.get(“alert_id”) alert_name = alert_document.get(“Alert-Rule-Name”) if alarm_state != “Alarm New”: sys.exit(1) # create a server instance server = smtplib.SMTP(“smtp.example.com”, 587) # start server connection server.starttls() # login using username and password server.login(“[email protected]”, “password”) # create message msg = ( “Hello, this is a test email for an alarm new “ + f“notification for alert {alert_name} with alert id {alert_id}“ ) # send message server.sendmail(“[email protected]”, “[email protected]”, msg) # end server connection server.quit() sys.exit(1) |
Generate a report when the alert condition turns active. The report is sent out over email to the email recipients configured in the report.
Users can create any number of alert rules in vuSmartMaps. The system will evaluate all configured alert rules at regular intervals and track the alarm state corresponding to each.
The saved alert rules in the system can be viewed or modified by users who have appropriate access permissions for individual saved rules.
To view existing alert rules, simply click the Alerts button under the Explore section situated on the right side of the Home page.
Edit: Click on the Edit Button under the Actions column to view or edit the respective Alert Rule.
Delete: Click on the Delete Button under the Actions column to delete the respective Alert Rule.
Multi Delete: Select one or more Alert Rules and click the Delete button at the top right.
Now, You can click on Permissions to manage Object Level Permissions in the Alert Rule.
The screen will look like this.
For every role, you can attribute 3 types of permission.
Edit: Click on this to edit the Alert Rule.
Clone: Click on this to replicate the current Alert Rule. This is the easiest and quickest way to duplicate an existing Alert Rule and make the relevant changes to match your requirements.
Enter the Clone name to create a unique name.
An alert rule can be disabled using the Alert Enable Switch button on the top right corner of the alert rule. When an alert is disabled, vuSmartMaps stops evaluating the alert conditions for the rule and will not generate any notifications.
Enable: Check one or more Alert Rules and click Enable.
Disable: Check one or more Alert Rules and click Disable.
By default, alert rules in vuSmartMaps work as alarms. In alarm mode, the system continuously monitors the alert conditions. It triggers an active alert notification when the conditions become true and a clear notification when they become false.
For example, let’s say you have an alert rule to monitor host CPU usage. When the CPU usage exceeds the set threshold, vuSmartMaps sends a notification. As long as the CPU usage remains above the threshold, no additional notifications are sent. Once the CPU usage falls below the threshold, vuSmartMaps sends a clear notification.
The clear notification for alarms will include details about how long the alarm was active and provide metrics and information for the entire duration of the alarm.
You can configure an alert rule to work in non-alarm mode if you need general information notifications. When alarm mode is disabled, vuSmartMaps will generate notifications whenever the alert rule conditions are met. It will also continue to send alert notifications at regular intervals as long as the conditions are met.
However, in this mode, vuSmartMaps won’t track the alert state, and it won’t send clear notifications when the conditions are no longer met.
Non-alarm mode-based alerts are particularly useful for one-time event cases.
For instance, when monitoring Syslog messages and you only want to be notified when a specific event like ‘System rebooted’ is detected, you can configure a non-alarm mode to check for this condition. In such cases, vuSmartMaps will send notifications each time the event occurs without tracking it as an ongoing alarm.
Switching between alarm mode and non-alarm mode is a flexible process for users. You can change an alert rule from alarm mode to non-alarm mode or vice versa by adjusting the configuration and saving it.
It’s important to remember that when you switch from alarm mode to non-alarm mode, the system stops tracking the alarm state for the rule. This means that no clear notification will be sent for alarm conditions that were active at the time of the configuration change.
The same principle applies when an alert rule in alarm mode is disabled in the configuration. In this case, as well, the system won’t send an explicit clear notification.
Browse through our resources to learn how you can accelerate digital transformation within your organisation.
VuNet’s Business-Centric Observability platform, vuSmartMaps™ seamlessly links IT performance to business metrics and business journey performance. It empowers SRE and IT Ops teams to improve service success rates and transaction response times, while simultaneously providing business teams with critical, real-time insights. This enables faster incident detection and response.