Scripted Alert Thresholds
It should be possible to groovyscript Alert Thresholds, based on (for example) ILPs and hostProperties.
I need to modify the SNMP_Network_Interfaces to vary the InDiscardPercent threshold depending on whether this is a radioMAC interface type and whether it is a given customer.
Something along the lines of:
def isRadio = instanceProps.get('auto.interface.type') == 'radioMAC';
def customerCode = hostProps.get('customer.code');
if(isRadio && customerCode == 'ACME')
{
// No threshold
return '';
}
// The default
return '> 10';
David Bond
Posted 3 years ago·Last reply 3 years ago
26 comments
LM User
·3 years agoOh yeah, i’ve got the FEED numbers for each one so they should make it into PB.
Well, 🤞🤞🤞, hoping we can import from PB, assuming they are already submitted. I honestly have no idea the level of effort. Assuming you have submitted these before?
LM User
·3 years agoMe too. Up to 57 official requests.
David Bond
OP3 years agoSomewhere, I have a spreadsheet of Feature Requests that went into the black hole that is the “Feedback” button.
I look forward to the data entry exercise of posting them into a public “Ideas Board” 😁
tswisdom
·3 years ago@Stuart Weenig I’ll check to see what I can find out!
Also, in the near future we are planning to implement an idea board that would show the status of feature requests ;)
Now that’s a feature I can get behind! Hoping it’ll have some form of voting and discussion builtin, that should allow customers to play a larger role in product development.
@Stuart Weenig I’ll check to see what I can find out!
Also, in the near future we are planning to implement an idea board that would show the status of feature requests ;)
LM User
·3 years agoHow long did this feature take to implement? Assume the same number of years? However, they do seem to be accelerating product development under the new product head/leader. Would be great to have that new product lead person host a webinar for the community about his style, roadmap, feature request philosophy, etc. @A11ey do you think you could make that happen?
mnagel
·3 years agoThe way LM would tell you to do this is to tune those thresholds at the group level for each datapoint. But you cannot create instance groups dynamically or do much with them even then (e.g., instance groups are not really groups, are not matchable in alert rules and you cannot tune thresholds for instance groups). Perhaps if they fixed those issues to work as you show, then the “normal” LM way would take care of it. Great idea!
Exciting news from v187. Not multiple thresholds, but it’s progress.
That is very good news! I am shocked to see that under “Fixed Issues” as I am so often told things like this are feature requests (along with many issues I consider bug fixes). Regardless, will be nice to see this finally happen. Wonder when instance groups will be selectable in Alert Rules….
Michael Raymond
·3 years agoThis is all great feedback! I’ll address this in a feedback request internally.
@David Bond -- Like Stuart mentioned, you can import the swagger.json file into Postman, but the auth isn’t in line with LM’s auth.
I’m not sure about how much of this stuff is possible on the Postman/Swagger side, but I’ll note it all.
LM User
·3 years agoAlso, if it’s possible, it would be nice to have the pre-request script built into the swagger.json file that can be imported into postman. That way, all you have to do is setup your environment and you’re ready to go. Currently, the environment is setup with the wrong auth type and the variable names differ from the setup instructions that have been used for a long time to setup postman.
https://community.logicmonitor.com/product-discussions-22/accessing-the-logicmonitor-rest-api-with-postman-and-lmv1-api-token-authentication-1703Now that LM is actually putting a little effort into the community, making this bit easier to onboard will go a long way towards building more trust between LM and the community.
David Bond
OP3 years ago@Michael Raymond - please do! Also, there’s a typo on that page: numOfKuberntesDevices.
@Stuart Weenig - the built-in testing capability will not work due to LogicMonitor’s bizarre security mechanism. Your best bet is to develop your own LogicMonitor Postman collection using the pre-request script mechanism documented here:
https://community.logicmonitor.com/product-discussions-22/accessing-the-logicmonitor-rest-api-with-postman-and-lmv1-api-token-authentication-1703Again, @Michael Raymond - a published Postman collection with full endpoint coverage would be nice!
LM User
·3 years agoWhile you’re asking around, do you think it would be possible to enable swagger’s built in testing capability? In other swagger documentation, i’m able to either sign in or provide api credentials and run the test right within the docs, giving me a way into the data without having to go to all the trouble of setting up postman (not that it’s an issue once it’s setup, but you get my point).
Michael Raymond
·3 years agoI completely get what you mean. I would love to see a more concise/clean/updated API doc section as well.
For me, I’ve been leveraging this page as a sort of v3 home base: https://www.logicmonitor.com/support/rest-api-change-log
I’ll ask around and see what can be done about the SEO around the v3 Swagger page.
David Bond
OP3 years ago@Michael Raymond - thanks for the URL. LM should either:
I understand that V1/V2 are about to end of life, so it’s a problem that the first link on Google for the search “LogicMonitor swagger” links to the V2 docs.
And you’re right - my main point is that the docs (even this V3 one) are:
Michael Raymond
·3 years agoNot a hope. The current Swagger is at version 2 still(!)
https://www.logicmonitor.com/swagger-ui-master/dist/
LM API v3 has a separate URL: https://www.logicmonitor.com/swagger-ui-master/api-v3/dist/
It’s actually embedded within the support doc here: https://www.logicmonitor.com/support/v3-swagger-documentation
I know this isn’t the main point, but wanted to point it out.
David Bond
OP3 years agoNot a hope. The current Swagger is at version 2 still(!)
https://www.logicmonitor.com/swagger-ui-master/dist/
LM User
·3 years agoOh, I know they will have built it. I was more referring to supporting/documenting it.
David Bond
OP3 years ago@Stuart Weenig - yes, they will have done.
All LogicMonitor UI actions are mediated via the API. You can press F12 in your browser, go to the Network tab in Developer Tools and view the transactions occurring. Everything goes via santaba/rest
LM User
·3 years ago@Stuart Weenig - that IS good news. So many questions though. As part of the upgrade, does the system look at all instances and if all instances in that group have the same threshold applied, automatically removes the INSTANCE thresholds and re-applies it at the group level? I sincerely hope so, but don’t expect so.
I doubt they would make changes so far reaching as that without user intervention. My guess is that they’re just now going to be available to set. One can hope they added API endpoints to do it, so we can create our own migration options. I would also hope they add the option in the global definition of the datasource to define thresholds at the group level. That would simplify our network interface monitoring.
David Bond
OP3 years ago@Stuart Weenig - that IS good news. So many questions though. As part of the upgrade, does the system look at all instances and if all instances in that group have the same threshold applied, automatically removes the INSTANCE thresholds and re-applies it at the group level? I sincerely hope so, but don’t expect so.
LM User
·3 years agoThe way LM would tell you to do this is to tune those thresholds at the group level for each datapoint. But you cannot create instance groups dynamically or do much with them even then (e.g., instance groups are not really groups, are not matchable in alert rules and you cannot tune thresholds for instance groups). Perhaps if they fixed those issues to work as you show, then the “normal” LM way would take care of it. Great idea!
Exciting news from v187. Not multiple thresholds, but it’s progress.
Michael Raymond
·3 years agoYeah I’m seeing that now. I do get that it can get a bit cumbersome.
LM User
·3 years agoThis is exactly what @mnagel was talking about not wanting to have to do:
Michael Raymond
·3 years ago@David Bond I do agree about your overall feedback. Having the ability to reference the output of scripts for complex datapoints would solve this easily. RE:
https://community.logicmonitor.com/feature-requests-5/complex-groovy-datapoint-access-to-script-batchscript-output-3509?postid=11716#post11716As for the HTTP status codes, perhaps you can try adding in the status codes you want to alert for in as complex datapoints. Check out the complex datapoint 503_error for the ‘HTTPS-’ DataSource:
David Bond
OP3 years agoJudging by changes I can see going on inside the v3 API, I’m hopeful that Instance Groups are about to become first class citizens in the Resource hierarchy and not just a tag. Yes, it would be great if you could set SDTs, Alert thresholds etc. at this level. I’ve been asking for years though, so I won’t hold my breath.
Frankly, the Alerting logic has always been too simplistic. I would love to be able to use switch statements on HTTP status codes, for example.
The whole “> 1 2 3” thing is woefully inadequate.
mnagel
·3 years agoThe way LM would tell you to do this is to tune those thresholds at the group level for each datapoint. But you cannot create instance groups dynamically or do much with them even then (e.g., instance groups are not really groups, are not matchable in alert rules and you cannot tune thresholds for instance groups). Perhaps if they fixed those issues to work as you show, then the “normal” LM way would take care of it. Great idea!
It also would be nice to be allowed to set multiple thresholds for datapoints if needed rather than hacking modules to create shadow datapoints in order to do so (my go-to example for why this is desirable is interface bandwidth -- it is perfectly reasonable to care about values that are too low OR too high). It would also be great to have actual alert templates (not unconditional token substitution for a subset of alerts), but I dream...