API Method to CLEAR an Alert (toggle off=on Alert Enable)
Hello,
We have a need to via API, CLEAR an alert. I don’t see any API Methods to clear an alert. I only see methods to GET or POST a Note/ACK.
Can you assist me with what API method we can use to clear (toggle on/off) an alert?
Thanks,
Darren Dudgeon
13 comments
LM User
·2 years agoThat’s an interesting one. I had something like that setup for my Device by Collector Auto Grouping DS which would create a group structure with a group for each collector with its assigned devices in its group. There was an option called cascadeSDT. At the time, group SDT wasn’t working right, so i built it so that whenever the collector went into SDT, the group would get alerting disabled.
The problem was that long standing alerts, or alerts that had already been acked would get closed and reopened. If you’re already disabling/enabling to clear alerts (it’s already part of your workflow) then go for it. I’d rather have the ability to selectively clear an alert to see if it comes back. Sometimes someone will close a ticket even though the alert is still open. Being able to just clear the alert manually and let it come back would be good because it would open a new ticket stating that the problem is still happening.
Barb Ramirez
·2 years agoThis is maybe similar to me wanting all staff to be able to toggle alerting when SDT is set. This will clear the alert in the ticketing system.
LM User
·2 years agoBasically: if you want the alert cleared, you have to either fix the problem or remove the threshold so LM doesn’t think it’s a problem anymore. I do believe alerts clearing on object deletion is a resolved issue.
AustinC
·2 years agoI remember the Clearing being an issue when we first onboarded, and I believe it was fixed in a series of myriad portal release a few years back. I have not seen the issues reoccur yet, but my scope has also changed such that it’s always possible the issue is not 100% handled. At the very least, it has not been such a significant problem as to warrant my attention.
David Bond
·2 years agoYes, we should lobby for instance/resource/website/... deletions to result in a set of instanceDeleted/resourceDeleted etc. Alert status notifications.
Mike Moniz
·2 years agoHeads up that sometimes LM will not send a clear message. It’s MUCH better than it use to be, but there might be situations where it might still happen. I think it might still happen if you delete a device/instance with active alerts, but I haven’t checked in a while. Suggest having a way to bypass any Close blocking if you do run into that, perhaps by privileged users.
David Bond
·2 years agoConfucius says: "Only when Service Operations and LogicMonitor both agree, is an Incident truly over."
The simplest approach is to assign the current latest alerting state into a separate ticket field and not permit the ITSM workflow to close the ticket if that field is not set to "clear".
AustinC
·2 years agoOur NOC solely uses ServiceNow Event Management, and we have no need for ‘toggling’ alerts due to how we’ve implemented our workflows.
If you are using outgoing webhooks to notify your ticketing system, you can configure your outbound integrations in a way that will ensure alerts re-clock -- this will ensure that currently active alarms always renotify. LogicMonitor Escalation Chains can be set to ‘escalate after XX minutes,’ even if there is only one stage -- it will simply renotify the exact same integration. This is how we use it, and we have never had to ‘temporarily turn off alerting’ to handle problematic/long-running alerts.
I would strongly urge one against ‘toggling’ alerts as they remain active in order to clear them, because you may run into the very real problem whereby you leave those instances disabled, which is going to eventually bite you in the rear almost guaranteed. There are a lot of procedural things that I think you would be better suited towards working on (e.g. improving alerting hygiene/education and training/setting healthy boundaries) than facilitating bad practices. I say that as someone that has been on a similar journey, so I appreciate it’s commonly easier said than done :)
Mike Moniz
·2 years agoI suggest instead setup the ticketing system to prevent the ticket from being able to be closed, or to reopen if alert was found to be still active. You can use the API to query for active alerts when attempting to close an alert ticket. It can get a bit complicated if you merge alert tickets, but doable.
Otherwise you would need to disable then shortly after enable alerting, which adds delays and can cause alert duration in LM to be wrong and reports perhaps thrown off.
Darren_Dudgeon
OP2 years agoWow - thanks for all the great ideas. I will review to see if they work.
Some clarification:
The issue that we are trying to solve is that our operational team works from ticketing system. They close the ticket sometimes without solving the alert. The alert remains active, and since they don’t work from logicmonitor, the alert will never re-trigger. This applies to all datasources, all devices.
So we need an API call from the ticketing system to clear the alert when the incident is closed, so it can re-trigger on next poll and open a new incident/ticket.
Mike Moniz
·2 years agoYou would clear an alert by solving the conditions that cause the alert in the first place, or modify the conditions (or disable alerting) so it is no longer considered an alert.
If you are looking to fully disable an alert, I believe you can look at the various “Threshold” APIs at https://www.logicmonitor.com/swagger-ui-master/api-v3/dist/#/Thresholds and modify the “disableAlerting” property.
AustinC
·2 years agohttps://www.logicmonitor.com/swagger-ui-master/api-v3/dist/#/Datasource%20Instances/updateDeviceDatasourceInstanceById - Here is the API call I was referencing to validate the `disableAlerting` property.
AustinC
·2 years agoIt sounds like you’re looking at this from the ‘Alert’ angle -- Alerting is controlled by the datasources/datasource instances, and I see that there are properties to ‘`disableAlerting`’ in the API -- have you looked at toggling these ‘sliders’ via the DataSources Instances resources?
Note that one does not ‘clear’ Alerts from the Alerts page -- one can only ‘acknowledge’ Alerts. To ‘clear’ an alert means:
Ergo, you must control ‘alert toggling’ from the DataSource/device/instance perspective, and not from the ‘Alert’ perspective. You CAN, however, ‘SDT’ an alert from the Alert perspective (but this simply disables alert routing mechanisms and allows you to filter out SDTd alerts -- it DOES NOT prevent the alert from generating/appearing).