Forum Discussion

Mike_Moniz's avatar
Mike_Moniz
Icon for Professor rankProfessor
6 years ago

Fast search for disabled Host Status check

LogicMonitor's Host Status DataSource is very important DataSource that provides notification of Host Down and we check that it's never disabled. It's a common thing I see were someone "just wants to check ping" and disabled all other DataSources including Host Status without realizing it's importance.

I've had a script that checks every device in the system in a loop and verifies none of them are disabled. It basically runs a REST call /device/devices/{ID}/devicedatasources?filter=dataSourceName:HostStatus for each device and checks alertDisableStatus and stopMonitoring. With thousands of devices this takes a long time and I'm wondering if anyone has suggestions on how to get this information any quicker? Some way to query multiple/all devices at once perhaps?

I tried looking at from the DataSource side (/setting/datasources/) but it doesn't provide that information.

Thanks!

  • Cole_McDonald's avatar
    Cole_McDonald
    Icon for Professor rankProfessor
    6 years ago

    What language are you using?

    I'm not sure what groovyscript has available in the way of filtering, but if you can find a way to pre-filter your results, it'll have fewer iterations to do... that will reduce the time it takes to do so.  What are the conditions/properties you're sampling/matching for this task?  In powershell, you can grab the whole chunk of them, then pipe them through a where to reduce the found set to just the ones with specific hoststatii.. or at the least, exclude known good ones to allow the rest of the set to be tested.  If you're matching it agianst another set of objects, you can use the -includes comparison operator to do a faster comparison once you've gathered the test set and the comparison set.

  • Mike_Moniz's avatar
    Mike_Moniz
    Icon for Professor rankProfessor
    6 years ago

    Powershell and this is not a DataSource but a independent script to check for issues. I've already limited the request best as I can but the issue is more about having to make thousands of API calls, 1 per device and I do want to check all of them. The API doesn't have a way to query multiple device's datasource instances or batch requests together. Not a big deal but wanted to see if anyone knows of any "tricks" with the LM API for batching.

  • Cole_McDonald's avatar
    Cole_McDonald
    Icon for Professor rankProfessor
    6 years ago

    If you use start-job for each of those requests within a batchscript, you could get them to run in parallel rather than having to wait to loop.

    Not sure that that would work on something more complex than a simple data grab.  I haven't played with them too much, but they should allow a little post processing in the script before returning results.

    Caveat: They seem quite aggressive if you have a lot of devices on the same collector as they open a socket for every instance discovered.  They're easier on LM as they do a single script run... but they instantly hit a windows environment with x requests all at the same time.  We've had quite a few issues with resource exhaustion as a result as all of our available ports are eaten up on the collector if the devices aren't balanced in the collector group properly (hence the other balancer efforts I'm working on in other threads).

  • Mike_Moniz's avatar
    Mike_Moniz
    Icon for Professor rankProfessor
    6 years ago

    Thanks. I think I would just more quickly run into the rate limit cap if I ran API calls in parallel.

    The caveat wouldn't really apply here since this is not a DataSource or running on the collector and this is directly querying the LM API and not devices.

  • Cole_McDonald's avatar
    Cole_McDonald
    Icon for Professor rankProfessor
    6 years ago

    I actually use dataSources to run timed scripts on a collector by targeting just the collector and having the REST API in the script gather info only on the devices reporting to that particular collector.  Once triggered, it's just the same as running a scheduled task on the collector, except that you have access to all of the ##Properties## as well... so creds can be stored there for different environments and whatnot.