Forum Discussion

Cole_McDonald's avatar
Cole_McDonald
Icon for Professor rankProfessor
6 years ago

Multi-Tenant per-customer alerting thoughts/ideas

@Mosh and @Paul Armenakis1456476360/ @Paul Armenakis Let's get this discussion started.  How are you guys implementing this distinction/segregation of alerting for your customers?

We've got our per client servers groups in folders within a Customers folder.  The alerting flows through the rules top to bottom, so I'm handling customer specific alerts first with the highest load companies at the top of the list to lower the load on the LM servers as it tries to match the alert to a rule.  As the rules stop running on a match, I have to build in notifications to our team whenever we hit one that is supposed to notify the client as well, as it won't move on to our team's default alert rules.  I would love to see a pass-through check box to allow multiple alert rules to match against an alert for exactly these reasons.  As far as these go... I have a teams integration, a teams + group email, and a ticket generation escalation chain to address different urgencies.

The escalation chains all have a blank at the end so that they fire the most urgent one, then repeat on the blank... that way, we don't end up with a million of the same alert over and over... or worse, a million of the same ticket over and over.

  • Mosh's avatar
    Mosh
    Icon for Professor rankProfessor
    6 years ago

    First I'd like to say that I'm happy with the way Recipients and Escalations work.  We too have the problem of lots of recipients and escalations because generally our infrastructure operations want to be notified only for infrastructure alerts, but the business service teams want to be notified for all alerts for their business service (even if they can't do anything about it, they just want to be informed).

    What I've always though would be useful would be to be able to set on the resource group an "Always notify this group" option.

  • Cole_McDonald's avatar
    Cole_McDonald
    Icon for Professor rankProfessor
    6 years ago
    On 9/24/2019 at 1:38 PM, Mosh said:

    First I'd like to say that I'm happy with the way Recipients and Escalations work.  We too have the problem of lots of recipients and escalations because generally our infrastructure operations want to be notified only for infrastructure alerts, but the business service teams want to be notified for all alerts for their business service (even if they can't do anything about it, they just want to be informed).

    What I've always though would be useful would be to be able to set on the resource group an "Always notify this group" option.

    From a current methods type of perspective.  Having a catch all alerts of priority X with a pass through check box would handle that as well.  That pass through (or conversely a "Stop Processing") option in a rule would be fairly easy to implement from a backend code perspective and wouldn't take any grand restructuring on the part of the dev team at LM.  Just a UI item that skips/includes the if/then statement to stop processing the alert.  Depending on how they've implemented it at the back end, it can be 3-4 lines of code and included in the next release :)/emoticons/smile@2x.png 2x" title=":)" width="20">