pulling text from db to add to error message

5 hours ago, Mosh said:

What I would love to be able to do is run a Groovy script on alerts between the Alert Rules and Escalation stages of alert processing so that the alert message can be enriched or additional custom logic can be used to determine if the alert should proceed to Escalation stage (the script could return a true or false to control whether the alert processing continues normally or is aborted). 

Example use case, alert is triggered > alert rule is matched > script checks CMDB and change management calendar, returns false if CI is in change or true to continue to Escalation.

In the previous monitoring tool we deployed for our clients (Nagios primarily), we developed a notification script that used a Perl Template::Toolkit handler to generate the output (this could be any of a number of template systems like jinja2, etc., we are just old-school Perl geeks).  One of the features in our system is a callback function that allows running a limited set of scripts to gather more information, just like an admin would if they were involved at the time the alert triggered.  Since the output from the callback was available to the template code, we could also create conditional output based on the callback output.  For example, if a callback produced a list of the top 5 processes from a Windows box, we would generally tack that onto the notification as additional information.  But the template could also check if the output contained a particular process, at which point it could add "Also, since XXX is high, please refer to YYY for more information on how to troubleshoot".  We have had that in place for like 10 years and it has been extremely frustrating to be restricted to unconditional token substitution.