Forum Discussion

Kelemvor's avatar
Kelemvor
Icon for Expert rankExpert
3 months ago

Orphaned tickets when a alert changes severity...

Hi,

I came in today and had a bunch of leftover tickets from the weekend.  They were all for Low Disk Space since we had patching on Saturday.  I know all the servers are working fine so I wasn't sure why I had these alerts.  After some research, I think LM has a slight issue in how it interacts with Integrations.  I'm specifically using Zendesk but I'm not sure if that matters.

Here's what happened.

  1. Error alert came in for a server with low disk space.
  2. LM used our ZD integration and fired the Active line which creates a ticket in ZD for us.
  3. 6 minutes later, the disk space dropped below the next threshold and became a Critical.
  4. LM used our ZD integration and fired the Active line which creates a ticket in ZD for us.  Now we had two tickets for the same alert.  One for the Error and one for the Critical.
  5. Later on, the alert cleared.
  6. LM used our ZD integration and fired the Clear line which closes the ticket in ZD for us.  However, it only fired this line once, for the Critical alert, which closed the second ticket that had been created.  This left the Error ticket still open and now orphaned.

The subject line of the tickets has the Severity in them, so it's right that it created a second ticket when the alert went Critical.  However, it should have either then, or when it closed the alert, fired two Clear integrations to close out both the tickets that had been created.

Has anyone else ever noticed anything like this?  I'm not sure if we have something broken or if this has always been like this or what.

Thanks.

integrations
  • Mike_Moniz's avatar
    Mike_Moniz
    Icon for Professor rankProfessor
    3 months ago

    Rereading your steps. If #4 really did kick off an Active call instead of Escalated, that sounds like a bug or it cleared between alerts. I would confirm on the Integration Log tab.

    (I meant Escalated instead of UPDATED, in previous message)

  • Mike_Moniz's avatar
    Mike_Moniz
    Icon for Professor rankProfessor
    3 months ago

    I'm not familiar with Zendesk integrations. Did you set it up per https://www.logicmonitor.com/support/alerts/integrations/create-update-close-tickets-zendesk-response-alerts? If so, make sure you have ##externalticketid## within the URL for the UPDATE type as documented there so LM will update the existing ticket and not create new one.

    You want to make sure that LM is tracking the ticket number via the ##externalticketid## token. You can't just use the LMA# for example as that will change when it jumps between alert levels.

    • Kelemvor's avatar
      Kelemvor
      Icon for Expert rankExpert
      3 months ago

      Yep.  We have it setup exactly like that.  The problem seems to be on LMs side in that it's sending a second Create but never sending the second Close.