Monitoring systems are great, but they also need to have a mechanism for alerting, we love our dashboards here at Timebeat but even we understand people can't be staring at the screen all day.
Timebeat can integrate alerts into any alerting system and if the list below doesn't include what you need let us know and we can get it configured for you.
Current alerting modules/notification channels:
- Google Hangout Chats
- Kafka REST Proxy
- Microsoft Teams
- Prometheus AlertManager
- Sensu Go
- Threema Gateway
System Alerts are some of the most important within Timebeat. These primarily link to annotations and to host status.
So lets configure a Notification channel:
Notification channel Configuration
For this example, we will be using the simplest method to configure a notification channel which is email (but you will find it's pretty simple for all of them).
On the left hand menu find the "Bell" icon and select notification channel.
From their select "Add channel"
For name, this can be what you want, this will be an Identifier for later when we configure the alert itself.
The Type, in this case, is Email but for an alternative channel just select the one you are after In the dropdown.
With the email channel below we find the "Address" field, just insert the email address you wish to receive the alert to.
Below is an example filled in.
And that is that, nothing more complicated than a couple of fields. Once completed just hit save and then lets get to configuring the alert itself.
Creating an alert based on data:
So first things first, we are going to need to build a line graph for the data we want to be alerted on (sometimes you might need to get creative but if you struggle just to get in touch with one of our alerting experts here at Timebeat who would be more than happy to help).
For this example, we will be using offset as our metric as it's simple to understand.
So we have our starting point with a line graph showing some simple data and in this case for ease we are using an absolute value for offset (always positive), but you can configure +/- alerts if desired (I.e. a range trigger).
At the bottom of the graph you can see the alert tab, select this and you will be taken through to the alert config section.
Below we can see once we select the Alert tab we see another button to create an alert. Just select that and then we can start the configuration.
Below I have filled in the areas required. This has configured the system to alert when above a threshold of 8ns, and as you can see the graph updates with a red line and a red shading. Feel free to fiddle with your settings here to suit your alerting requirements.
A key item to note in the above screenshot is the notification section at the bottom of the Image.
Where we have the "Send to" section you will select the box (which I have configured to have "test channel" but you will find empty), to have the notification channel name we created early.
This will then trigger the notification channel instructions at the point the alert is triggered.
Potential alert triggers you might be interested in:
Step: When Timebeat steps a host clock an annotation will be logged, this will trigger the Step alert and will notify you via the configured channels as well as the home page alerts list.
UTC Source Failover: When Primary sources are unavailable and secondary UTC sources have been configured an annotation is created and then an alert will be triggered and will notify you via the configured channels as well as the home page alerts list.
Interference: When an alternative application other than Timebeat alters a device clock an annotation will be created for Interference mode and then an alert will be triggered and will notify you via the configured channels as well as the home page alerts list.
Error within Log: When a log entry is listed as Warn level and an alert will be triggered and will notify you via the configured channels as well as the home page alerts list.
Offset: When the offset of a clock goes above or below a certain threshold.
This is one of the most useful alerts for allowing you to understand which hosts are reporting. When a host is reporting this graph will appear as 1. When the host stops reporting the graph will show 0, which will trigger the alert and will notify you via the configured channels as well as the home page alerts list.