Datasource to monitor Windows Services/Processes automatically?

From my understanding, the native WMI-based checks will make a new WMI call for each instance, so 1 WMI call for each windows service and process, hence why you see 12k of them. There are a lot of types of checks that work that way, but there is one option that will let you make one WMI call per device (if you can get all the data in one call) and extract  in bulk for all instances at once: BATCHSCRIPT. I'm not sure if it would completely help in your situation, but if you switch from native WMI to using something like a PowerShell or Groovy BatchScript, you can send one WMI query to the server and get data for all services/processes at once. Scripts do cause more load on the collector than most native checks, but 150 script instances (75*2) are likely less load then 12k WMI instances.. Actually I think the collector does WMI queries via powershell anyway, not 100% sure about that, so even less of a concern.

You can still keep the old WMI AD method and just move Collector Attributes to use batchscript.

Take a look at these two datasources: https://github.com/sweenig/lmcommunity/tree/master/ProcessMonitoring. I just realized i haven't done any documentation on that part of the repo, so give me a few minutes and i'll commit some instructions.

It doesn't change how many resources are used to monitor a process, but it does do what i think you were referring to in the nightmare scenario above. They let you specify an include and an exclude filter as properties on the device level or on the group level. So, you can just provide a regex expression to dictate what you want to include and what you want to exclude in active discovery.

Alright documentation on that DS has been updated in the repo. Take a look and see if it helps.

Hello @Stuart Weenig,

If I understood it properly, I think we did that already with a custom property to actually filter 'noisy' services using a regex expression (on a group/device level). That helps to create exceptions indeed.
However, that leaves us with the same issue still. Due to the reasons I mentioned above.

Seems like what it's coming down to is that you are trying to monitor more stuff than the current collector resources can handle. Only two options really: reduce collection (stricter filters) or increase collector size.  You're already excluding manual and disabled services, right?

Ooo, that's a great point. Yes, the groovy based WMI query could do a single call and grab all the data. You'd have to parse through it and print each bit out, but that's easily doable with a for loop.

And keeping discovery using the WMI method is an excellent option.

3 minutes ago, Stuart Weenig said:

Yes, the groovy based WMI query could do a single call and grab all the data.

Can you do WMI using groovy? I looked around before but didn't find an example. I try to use groovy when I can, although WMI is only supported on Windows collectors anyway.

1. Hello guys,

Thanks a lot for the feedback & ideas @Mike Moniz & @Stuart Weenig!

We really enjoyed Mike suggestion of doing a script to actually poll the data in bulk - per device (instead of 1 query per service/process instance).
After reading some documentation & understanding how to use WMI on groovy (with help of some OutOfBox datasources) we ended up by rebuilding those 2 datasources (services & processes) using groovy (making use of the WMI class).

They're collecting exactly the same metrics as the out of the box ones & after 2/3 hours in production those really reduced the collector CPU/Memory usage - used 1 of our clients as a pilot

Previously the usage was constantly at >98%, now it's monitoring the same amount of services/processes & using 60-70% of CPU.

It increased the groovy instances exponentially - examples below

However, this doesn't seem to affect the collector resources usage that much.

We're also no longer seeing the WMI timeouts on the WMI instance runs. We guess this is really making a difference.
I know it's too soon to say this solved it, but by judging how the resource usage looks, this seems to make a big difference.

Also, the list of services changes, that's why we ended up using a property.

I'll share both datasources with you guys in a few minutes (I'll just attach the .XML file(s) to this thread) in case this is useful for anyone.

Regards,

Not at all, feel free to use them.
I was thinking on submitting a thread in LM Exchange containing those 2 datasources but I didn't wanted to create repeated threads (like I did a couple of days ago ?).

Let me know if we should submit a post within LM Exchange containing those 2 datasources for future record or, if this one is enough for the sharing purpose.

Regards,

2 minutes ago, Mike Moniz said:

 

I've had problems attempting to do that in the past outside of LM but haven't really resolved it (kinda gave up at the time). I think you need to query the value twice over a set period and also take into account the number of logical cores in the system has. But even doing that I intermittently was getting weird results like 72025954.98% cpu. You'll see some various discussions on replicating task manager per-process cpu % via wmi on google.

It would be great if there was a good solution to this though.

Yeah I've also been googling about this & found a bunch of threads, however, I got kinda confuse. Did some tests & the results were definitely not accurate. 
Hoping @Stuart Weenig to make it work!

It would definitely be useful!