Forum Discussion

SteveBamford's avatar
SteveBamford
Icon for Neophyte rankNeophyte
2 months ago

Best Practices for API Calls in a datasource

Hi all,

Possibly the most random question of the week, when working in datasources where you are looking to utilize API calls what would you say is the maximum calls to make in what datasource typically I have worked on one data retrieval call per data source.

Why the question?

So Dell have withdrawn a number of fields from their Dashboard API in Dell ECS which means metrics such as CPU and memory need now to be retrieved from the flux API that is provided amongst a few other metrics which I may or may not need to provide to our infrastructure team.

To do this it looks like I may need to generate at least two flux queries one for CPU and one for memory this will result in two API calls.

So would you create a single data source for each metric or make the two calls within the datasource so you have a global stats data source for this sort of information.

Thanks in advance for your input.

 

1 Reply

  • jhupka's avatar
    jhupka
    Icon for Neophyte rankNeophyte
    2 months ago

    The answer to this is nuanced to me, and really depends on the amount of data you're getting versus the number of calls you're making. Also, how often is this data being collected, is it being grouped efficiently (or in a way that makes sense), etc.  You might also need to conform to a certain # of API calls during the day, and so need to gather lots of info in one call.  There's tons of considerations in this, and the easy answer to me is, "Just do it...see how it works."

    In your example, I probably would lean towards making separate calls for CPU and Memory simply because when you define a Datasource you also want set what the Resource Label and Group for how it gets displayed underneath a Device.  

    Now, if I wanted to grab all Memory info from an API, and have to make five calls to do it, then I would have no problems putting those five calls in one Datasource as long as it performs in a manner that matches how often I'm collecting the data.  E.g. I wouldn't want a Datasource collection run that takes 2 minutes to be executed every 5 minutes.  That seems a bit more logically grouped and efficient to me.

    At the end of the day these Datasource collection processes are running on your Collectors. So if you want to stay on top of poor performance make sure you're watching Collector data collection script execution metrics.