External Alerting - Script - Medium for self-heal/actions? | Archive

Joe_Tran · 2019-03-19T23:53:26Z

I will admit that I had completely forgotten that External Alerting was a thing. When we first started with LogicMonitor (like 3+ years ago), someone at LogicMonitor had mentioned External Alerting as a potential solution for a random use-case and I had immediately disregarded out of hand--favoring a Custom HTTP Delivery integration instead. Fast forward to now and this a href="https://communities.logicmonitor.com/topic/2245-feed-lm-alerts-to-splunk-tool-excluding-custom-email-delivery-method/?do=findComment&comment=5924" rel="">post, and all this recent talk about self-heal and actions and an idea was sparked. Some internal partners are building automation tools to resolve issues and are pretty comfortable with some DIY. Originally, I had figured I would have to get them setup with an AWS Gateway+Lambda function that can receive alerts triggered which would then start a cascade of custom code, in the correct AWS VPC, to self-heal--but why bother when we have external alerting right? The client environments that I monitor and that these internal partners manage have dedicated collectors in each client environment. Just assign that client's collector to that client's resource groups, throw in a broker-like script that takes in necessary resource metadata, datasource, and execute the necessary remediation scripts. Disregard any alerts for datasources not supported by our self-healing project. This assumes that I'm interpreting External Alerting correctly. The key thing for this to work for my use case would be the ability to have External Alerting AND our normal Alert Rules apply to the same resources/alerts. The Alert Rules would still be responsible for delivering the alert to our ticketing system. Timing of when Alert Rules would trigger and when External Alerting would trigger would be nice. The support center page for this makes it seem the collector polls the resource group at regular, but unknown, intervals. The Alert Rules would populate alert with the ##externalticketid## and it would be neat to have the External Alerting also take that in as a parameter to update said ticket. I would also need to know if the script executed from this is subject to timeouts, concurrency limits, etc or if there is a limit to the number of External Alerting configs. Am I way off base?

MoreDiscussions

External Alerting - Script - Medium for self-heal/actions? | Archive | LogicMonitor Community