Forum Discussion

kk_1's avatar
kk_1
Icon for Neophyte rankNeophyte
2 months ago

LogicMonitor.Api C# nuget limits size to 300?

Hi, We have just started using the LogicMonitor Api Nuget to collect data from LM.

I have noticed when we call either the GetAllDeviceDataSourcesAsync or GetAllAsync we only get 300 back. I have updated the code to pass in a filter with Take=1000, but the dependency logs still show the size as 300.

I have noticed that 300 appears to be hardcoded deep inside the code at https://github.com/panoramicdata/LogicMonitor.Api/blob/main/LogicMonitor.Api/LogicMonitorClient.cs#L218 and was wondering if anyone knew the reason for this hardcoding?

The issue is that with a size of 300 it means we are using 3x the calls that are possible (the max should be 1000) and this will mean we hit the https://www.logicmonitor.com/support/rest-api-developers-guide/overview/rest-api-rate-limit a lot quicker when we are querying for a lot of information.

Reading the guidance on https://www.logicmonitor.com/support/v3-swagger-documentation#:~:text=LogicMonitor%20REST%20API%20v3%20supports%20maximum%20size%20of%201000%20records.  says that a size of 1000 is supported and no where does it mention any reason we should be limited to 300.

Can I please have some help with either an understanding or a workaround?

  • Hello kk_1

    For clarity, the LogicMonitor.Api nuget package is NOT provided by LogicMonitor.  It's provided by my company Panoramic Data Limited (PDL).  It's MIT licensed, open source (https://github.com/panoramicdata/LogicMonitor.Api/) and so free to use.

    It is not possible for the client (library) to override REST service ("santaba")'s server-set maximum page sizes.  For example, if you ask for 10,000 the LogicMonitor will still only return the server-size maximum number of items.  There is nothing that we can do in the client Nuget to change the server's behaviour, so we reduce the requested page size back down to our understanding of the maximum in client-side code to ensure correct paging.

    Next issue - you state that the documentation claims a max page size of 1000 for the V3 API.  This is indeed news to me!  For years and years (V1 and V2), the maximum page size varied HUGELY between endpoints.  For some, the max size was 25, for most 300.  We had to put a lot of code in the library to adjust these based on the endpoint being called.

    However, like you, we value query performance!  So I have asked my developers to look at raising this to 1000 across the board.  Please be patient, as there is a LOT of production-critical code relying on successful paging, so it will take a while to validate the change before we can publish it.

    You can track the issue here: https://github.com/panoramicdata/LogicMonitor.Api/issues/10

  • Thank you David_Bond , yes I completely understand that this is PDL code and that this is technically a LM forum. My reason for posting on this forum was around the comments I saw in your release notes to say "We will answer your questions in this forum", I also wanted to know if there was any reason other than historical as to why this may have been limited. From your response it sounds like it is just historical. I really appreciate this code that you have provided for community use.

    Thank you also for logging the issue for this and looking to fix it, I will watch the issue link above to understand how it is progressing.

  • Hello kk_1

    For clarity, the LogicMonitor.Api nuget package is NOT provided by LogicMonitor.  It's provided by my company Panoramic Data Limited (PDL).  It's MIT licensed, open source (https://github.com/panoramicdata/LogicMonitor.Api/) and so free to use.

    It is not possible for the client (library) to override REST service ("santaba")'s server-set maximum page sizes.  For example, if you ask for 10,000 the LogicMonitor will still only return the server-size maximum number of items.  There is nothing that we can do in the client Nuget to change the server's behaviour, so we reduce the requested page size back down to our understanding of the maximum in client-side code to ensure correct paging.

    Next issue - you state that the documentation claims a max page size of 1000 for the V3 API.  This is indeed news to me!  For years and years (V1 and V2), the maximum page size varied HUGELY between endpoints.  For some, the max size was 25, for most 300.  We had to put a lot of code in the library to adjust these based on the endpoint being called.

    However, like you, we value query performance!  So I have asked my developers to look at raising this to 1000 across the board.  Please be patient, as there is a LOT of production-critical code relying on successful paging, so it will take a while to validate the change before we can publish it.

    You can track the issue here: https://github.com/panoramicdata/LogicMonitor.Api/issues/10

    • Mike_Moniz's avatar
      Mike_Moniz
      Icon for Professor rankProfessor

      I'm interested in what endpoints you still find have a max under 1000, outside of perhaps Alerts since it does paging differently. Atleast the most common endpoints have been 1000 for a long while, but I haven't touched the API in 3 years.