When a problem occurs at a Datacenter, Application, or SLA, the Monitoring Station can send alerts to users. Alerts are notifications that inform users who are configured to receive alerts of the problem. The notification message contains the following information:
Whenever the status of an Element changes - for example from Critical to Warning - up.time sends an alert.
You can also configure alert escalations that occur if a warning is sent and is not acted upon. For example, if an alert is sent to a system administrator and the administrator does not attend to the problem within a specified amount of time, then the alert will be sent to the administrator’s manager.
up.time can send alerts via:
The following is a sample email alert:
Notification type: Problem
1/12/2008 10:52
Host: filter
Host State: N/A
Service: FS Capacity - Filter
Service State: WARN/
Output: /var is 92% full
The following is a sample pager alert:
subject:
CRIT Alert
content:
5/7/2005 13:22
Type: Problem
Service: FTP (CRIT)
Host: filter (CRIT)
For more information on alerts, see Monitor Alert Settings.
Alerts in up.time follow a specific flow. When up.time detects a problem with a host, it issues an alert. up.time then continues to check the host at specific intervals and reports on the status of the host.
Considering the following example:
up.time encounters a critical error on a host. up.time performs three rechecks at one minute intervals - all of which return a critical error - and then sends an alert after the third recheck.
up.time then checks the host every two hours. While up.time encounters two critical errors, it does not send an alert. Then, the status of the host changes from critical to warning. When this change is detected, up.time sends an alert informing recipients of the change in status. When the status of the host changes to OK, up.time issues an alert informing recipients that the host has recovered.
This alert flow is illustrated in the following diagram:
Alert Profiles are templates that tell up.time how to react to various alerts that are generated by service checks. Alert Profiles enable up.time to execute a series of actions in response to the failure of a service check or when a threshold is exceeded. The following diagram illustrates how an Alert Profile works:
An Alert Profile can send an alert via email, or to a pager or a cell phone, or a Windows popup alert. You can configure any or all of these actions to occur simultaneously. For example, if a Web server process stops responding, the system administrator can be notified.
In order to receive popup alerts from up.time , the Windows messaging service must be enabled on the recipient’s computer.
Email Alert
Sends the alert to the email addresses of the members of a Notification Group.
Pager Alert
Sends the alert to the pagers of the members of a Notification Group.
Script Alert
Uses a script to send the alert via SMS to the mobile phones of the members of a Notification Group.
Since this alert option relies on a script or batch file, you must enter its name and path in the Script Path field (for example, on Linux, /usr/local/uptime/scripts/scriptAlert.sh
). When the alert is triggered, up.time runs the script and passes the script or batch file a set of parameters. The script is run for each up.time user who will receive the SMS message.
For details on how to create the script, see the Client Care Web site Knowledge Base article Creating Custom Alert Scripts in up.time Alert Profiles.
Windows Popup Alert
Sends the alert via the Windows messaging service to the desktops of the members of a Notification Group.
Notification type: Problem 27/4/2006 09:19
Host: Test Host (OK)
Service: Test Monitor
Service State: OK
Output: This is a test notification; please ignore.
You can associate an Alert Profile to any Service Monitor, Application, or SLA if their state changes from OK to Warning or Critical. Alert Profiles are normally associated with any of these monitored items at the time of their configuration; Alert Profile assocations can also be modified with existing service monitor definitions.
See Using Service Monitors, Working with Applications, and Adding and Editing SLA Definitions for more information about configuring Service Monitors, Applications, and SLAs, respectively.
up.time ’s standard alert format is well suited for most alerting needs. However, you can modify the content of the alert. up.time comes with three custom alert templates. You can change the content of the alert by adding or removing variables from the template.
The variables are the building blocks of a custom alert format. You can add or remove variables to suit your needs.
These alert variables are also available as input parameter values when configuring an Action Profile to initiate a VMware vCenter Orchestrator workflow.
The table below explains the variables available in custom alerts, as well as Orchestrator input parameters :
Variable | Definition |
---|---|
$DISPLAYNAME$ | The name of the Element as it appears in the up.time Web interface. A system can have a different display name than the hostname. For example, you can assign the display name Toronto Mail Server to a system with the host name 10.1.1.6 . |
$DATETIME$ | The date and time at which the alert was generated. This appears in the subject line of the message. |
$SERVICENAME$ | The name of the service, along with the name of the host for which the alert was generated. For example, if the alert was generated by the ping check for the server MailHub, then PING-MailHub appears in the alert. This appears in the subject line of the message. |
$SERVICESTATE$ | One of the following:
This appears in the subject line of the message. |
$DATE$ | The date on which the alert was generated. |
$TIME$ | The time at which the alert was generated. |
$HOSTNAME$ | The name of the host (as saved in up.time ) for which this alert was generated. |
$HOSTSTATE$ | The status of the host, which can be one of the following:
|
$TYPE$ | The type of notification, which can be one of the following:
|
$OUTPUT$ | The output of the monitor that generated the alert. For example, Ping completed: 1 sent, 100.0% loss, 0.0ms average round trip time |
Action Profiles are templates that direct up.time when it encounters a problem on a monitored system. You can associate an Action Profile to any Service Monitor, Application, or SLA if their state changes from OK to Warning or Critical. Action Profiles are normally associated with any of these monitored Elements at the time of their configuration; Action Profile assocations can also be changed when you are modifying existing service monitor definitions.
See Using Service Monitors, Working with Applications, and Adding and Editing SLA Definitions for more information about configuring Service Monitors, Applications, and SLAs, respectively.
Actions include one of the following tasks:
As templates, Action Profiles can be reused for any number of Service Monitor configurations. This means you can create a series of them as standard actions used to respond to typical types of problems you may encounter, depending on what role a Service Monitor is playing (e.g., availability or performance).
If an administrator has integrated up.time with VMware vCenter Orchestrator (see VMware vCenter Orchestrator Integration), you can configure Action Profiles to initiate Orchestrator workflows.
Orchestrator is a VMware vCenter Server add-on that allows its administrators to create workflows that automate vCenter management tasks. These Orchestrator workflows are open ended: all vCenter actions are available for automation through the processing of parameters and runtime arguments. up.time Action Profiles can be configured to provide input parameters to specific workflows, thus integrating vCenter management with up.time ’s monitoring and alerting capabilities.
For example, if up.time is monitoring memory, CPU, and hard disk use for a virtualized server, the passing of performance thresholds can trigger an Action Profile that, in turn, triggers an Orchestrator workflow that creates a new virtual machine to alleviate resource strain. In a converse example, if up.time is monitoring a virtualized server for long periods of inactivity, a triggered Action Profile can initiate an Orchestrator workflow that shuts down the instance to free up resources.
By tightly integrating up.time ’s monitoring and alerting with VMware vCenter Orchestrator’s automated virtual environment administration, you can accelerate your organization’s reaction time with virtual systems management, and map established policies to automated actions.
When configuring Action Profiles, up.time communicates with Orchestrator and dynamically produces a list of all available workflows. (This includes any third-party workflow packages that have been installed on the Orchestrator server, including the up.time Orchestrator package.)
When a workflow is selected, and the Get Parameters button is clicked, the corresponding input parameter fields are dynamically displayed, allowing you to specify parameter values required to completely configure the workflow for execution should an up.time alert initiate it.
When configuring a VMware vCenter Orchestrator workflow, you have at your disposal a set of up.time -specific variables that can be entered as parameter variables, and whose ensuing runtime values will be passed to the Orchestrator workflow during execution. The variables available to you are those that are used when creating a custom alert format. See Custom Alert Format Variables for information.
You can also configure an Action Profile to send an SNMP trap to a particular host. An SNMP trap is notification that is issued by a system that is running SNMP when a problem occurs. The host to which the SNMP trap is sent must be running an SNMP trap listener.
If you use SNMP traps, the trap message will be sent in the format specified by the up.time MIB. This MIB is found in the scripts directory. The uptime software enterprise OID is .1.3.6.1.4.1.24216 .
"/usr/local/uptime/recover.sh" "24/12/2007 5:01:05" "Problem" "printserver" "null" "WinSrv-Print Spooler" "CRIT/threshold error" "servicestatus: Not Running does not match Running (Service 'Print Spooler' found, status: Not Running, took 12ms)"
You can also use the recovery script to file trouble tickets with a system like Remedy, or to interact with third party software packages. |
Windows Host
The name of the host on which the service is running.
Enter You can use this dynamic hostname in conjunction with service groups, where an issue can originate from one of many hosts. |
Windows Service
The display name of the specific Windows service to which the Action Profile will apply. The display name of a service appears in the Name column of the Services Control Panel, or in the Description column of the Windows Task Manager Services tab.
The service display name must be entered verbatim, including spaces, otherwise it will not be correctly processed. Double-clicking a service name in the Services Control Panel opens a properties window where you can highlight and copy the service Display name. |
Windows Service
The display name of the specific Windows service to which the Action Profile will apply. The display name of a service appears in the Name column of the Services Control Panel, or in the Description column of the Windows Task Manager Services tab.
The service display name must be entered verbatim, including spaces, otherwise it will not be correctly processed. Double-clicking a service name in the Services Control Panel opens a properties window where you can highlight and copy the service Display name. |
.1.3.6.1.2.1.34.4.1.7
.Monitoring Periods are the times over which a service monitor will be actively monitoring a host. The Monitoring Periods also apply to the times when up.time sends alerts
up.time comes with the following Monitoring Periods:
Monitoring is performed 24 hours a day, seven days a week.
Monitoring is performed from 9 a.m. to 5 p.m., Monday to Friday.
No monitoring is carried out.
You can add Monitoring Periods that suit your needs. For example, you can create a Monitoring Period called Weekends that only monitors a host from 12:00 a.m. on Saturday to 11:59 p.m. on Sunday.
Email Alert
Sends the alert to the email addresses of the members of a Notification Group.
Pager Alert
Sends the alert to the pagers of the members of a Notification Group.
Script Alert
Uses a script to send the alert via SMS to the mobile phones of the members of a Notification Group.
Since this alert option relies on a script or batch file, you must enter its name and path in the Script Path field (for example, /usr/local/uptime/scripts/scriptAlert.sh ).
When the alert is triggered, up.time runs the script and passes the script or batch file a set of parameters. The script is run for each up.time user who will receive the SMS message.
For details on how to create the script, see the Client Care Web site Knowledge Base article “Creating Custom Alert Scripts in up.time Alert Profiles”.
Windows Popup Alert
Sends the alert via the Windows messaging service to the desktops of the members of a Notification Group.