Regarding CPU and Memory and Disk Performance Data Collection for multiple servers | Product Discussions

usnishguha · 2021-04-30T03:04:42Z

Hi Everyone, Can anyone please let me know if they are aware of any method/script that can be used to get the CPU, Memory and Disk related performance data in csv for a list of servers between a specific start time and end time. Thanks in advance.

Cole McDonald

·4 years ago

My query params were wrong... I'd set them as a default for getting lists of resources past the device limit... buried them in a function... then forgot about them. Out of sight, out of mind. That function calls my send-request function (modified from the api docs here). Once I switched to my send-request function directly... I started getting results. Like I said previously... It's always something simple.

Cole McDonald

·4 years ago

I do all of my work in Powershell at this point... makes it easier to tie the different toolsets together (although, not nearly as easy as Applescript does... but that's a different portion of my career).

Henry Steinhauer

·4 years ago

@Cole McDonald I agree it would make sense that once you have the unique instance ID you should be able to pull the information with that ID alone.

I'm creating the same type of reference utilization table based on the time series information and need to better understand the usage over the prime time day and the overnight window. One item I need to exclude is the obvious backup window time frame. That should be full usage of the circuit or interface and should not be factored in with 'normal' utilization. But still accounted for in terms of high usage during that interval.

Did you end up doing this work in Powershell? Or did you finish it off in Python? Curious as to how you finished up the project.

We have 90+ remote facilities each with dual WAN links and also an Internet connection now for the OS365 and other direct attached SAAS applications. Those application no longer use the Hub Spoke design for application use and have complicated the overall summation of utilization.

LM User

·4 years ago

I think it's the 427972 that's not right. Hard to say without navigating your IDs directly.

Here's what I'd do (in code or in postman). If this is what you're already doing, ignore:

/device/devices -> find the "id" of the device of interest (here, i'll call it $device_id)

/device/devices/$($device_id)/devicedatasources -> find the "id" (not the dataSourceId) of the datasource of interest (here, i'll call it $DS_id)

/device/devices/$($device_id)/devicedatasources/$($DS_id)/instances -> find the "id" of the instance of interest (here, i'll call it $instance_id)

/device/devices/$($device_id)/devicedatasources/$($DS_id)/instances/$($instance_id)/data -> should have data here from the last hour

For what it's worth, i was skeptical about postman too. I'm old school CLI as well. But i quickly got addicted. The other thing i got addicted to was the Python SDK. Combining it with my custom LM wrapper, the code becomes:

<span class="kwd">from</span><span class="pln"> lm </span><span class="kwd">import</span><span class="pln"> lm
data </span><span class="pun">=</span><span class="pln"> lm</span><span class="pun">.</span><span class="pln">get_device_datasource_instance_data</span><span class="pun">(</span><span class="lit">9</span><span class="pun">,</span><span class="lit">9512</span><span class="pun">,</span><span class="lit">146237749</span><span class="pun">)</span>

Where 9 is the $device_id, 9512 is the $DS_id, and 146237749 is the $instance_id.

Cole McDonald

·4 years ago

I'll be getting postman running tomorrow... my time was up and had to move to another effort. Thank you for pointing the tool out to me. It looks really useful. and thanks for poking at my code. I'll verify I have the right IDs for each of those pieces as well. (Funny Note... I typed this up yesterday and didn't hit submit). I've verified that I have the right pieces and parts. You're examining the JSON in your examples... I use Powershell ISE and explore the return data using the variables. I have verified that I'm using the deviceDataSource rather than the dataSource:

<span class="pln">PS C:\Users\cole.mcdonald> $inst_CPU

id                       : 8976881
dataSourceId             : 5725412
deviceDataSourceId       : 427972
groupId                  : 538636
groupName                : @default
name                     : Perfmon_CPU-_Total
displayName              : _Total
description              : 
lockDescription          : False
deviceId                 : 5378</span>

and my call was:

<span class="pln">/device/devices/5378/devicedatasources/427972/instances/8976881/data</span>

(I switched devices to verify that it wasn't just empty) I've tried grabbing data from both the instance directly as well as the graph data. I'm going to dig into postman... but looking at it, I don't know that it will necessarily show me anything different than I'm getting using straight powershell... and I generally dislike GUI'd applications for this sort of thing as I find them horribly inefficient (I'm old and grew up on CLI).

Here's my Postman return:

<span class="pln">{
    "dataSourceName": "Perfmon_CPU",
    "dataPoints": [
        "PercentProcessorTimeCounter",
        "PercentProcessorTime"
    ],
    "values": [],
    "time": [],
    "nextPageParams": ""
}</span>

So it looks like it's succeeding in the call, just no data returned. Odd, because the graph shows that there should be historical data there to grab.

LM User

·4 years ago

Would suggest you get postman up and running and vet out all the calls you are trying to make before you try to code them in. That will eliminate the question of whether it's something about the call or Posh. Just a suggestion.

That said, when i did your call, the $($DS_CPU.id) variable is the id (9512), not the dataSourceId (24131557):

And my $($inst_CPU.id) was the id (146237749), not any other id. That one stumped me for a while until i learned that lesson.

So, my final call was {{url}}/device/devices/9/devicedatasources/9512/instances/146237749/data

Where my $($device.id) is 9, $$DS_CPU.id) is 9512 (id) and my $($inst_CPU.id) was 146237749.

Cole McDonald

·4 years ago

3 minutes ago, Stuart Weenig said:

What do you get when you execute the same call through a href="https://communities.logicmonitor.com/topic/1964-accessing-the-logicmonitor-rest-api-with-postman-and-lmv1-api-token-authentication/?tab=comments#comment-4808" rel="">Postman?

No idea... I've never used it... I've only ever used the PS functions I'm currently using... including to get other data previously. Although I'm not sure if I was using V1 or V2 at that time.

LM User

·4 years ago

What do you get when you execute the same call through a href="https://communities.logicmonitor.com/topic/1964-accessing-the-logicmonitor-rest-api-with-postman-and-lmv1-api-token-authentication/?tab=comments#comment-4808" rel="">Postman?

Cole McDonald

·4 years ago

i'm trying to get to this data using Powershell... no errors, it successfully grabs the device, DS, and instances using the same functions... but the data returns an empty array for data that is present in the LM resources interface... thoughts?

$resourcePath = "/device/devices/$($device.id)/devicedatasources/$($DS_CPU.id)/instances/$($inst_CPU.id)/data"

my output shows me this:

Gathering /device/devices/6966/devicedatasources/503122/instances/9311756/data

No errors are thrown during the run. It's confounding me a bit.

LM User

·5 years ago

I'd need to see your code in order to help you troubleshoot it. Please remove any API keys/tokens before posting.

usnishguha

OP5 years ago

Hi @Stuart Weenig, I was able to get past the "certifi" module error after installing it. But can you please let me know about this problem,

Traceback (most recent call last):
File "<stdin>", line 1, in <module>
NameError: name 'lm' is not defined
>>>

*seems it doesn't find lm module.

usnishguha

OP5 years ago

@Stuart Weenig, thanks a lot for your detailed answer. I am struggling to some extent with Python SDK ( I am new to this). I have tried to download the tar.gz and extracted it. Then I tried running a sample from here: https://www.logicmonitor.com/support/rest-api-developers-guide/logicmonitor-sdks. But it's giving me this error,

ModuleNotFoundError: No module named 'certifi'

I am probably doing it wrong. Is there any url which helps in starting with the SDKs?

Thanks in advance.

usnishguha

OP5 years ago

yes @Stuart Weenig, exactly like that. Can you please help me to find any sample API scripts in which can referred for this purpose? Thanks in advance.

LM User

·5 years ago

Like if you're polling at 1 minute but you want to pull the data with a resolution of every 5 minutes? Closest thing might be the graph data api endpoints.

usnishguha

OP5 years ago

Hi @Stuart Weenig, not really this. I was looking to know if there's anyway we can export the raw data from the data source but with custom interval (different from the polling interval of the datasource). Can this be done via API?

LM User

·5 years ago

Like this (but obviously CSV)?

LM User

·5 years ago

Ok, so you want to aggregate data over the selected timeframe to get an average, sum, or percentile for each device?

usnishguha

OP5 years ago

Thanks @Stuart Weenig, I have earlier checked the option, but what I was looking for is the detailed data with a custom interval (greater than the polling interval of the datasource). Can you please help with that?

LM User

·5 years ago

Yes, a Resource Metric Trend report would do this for you.