Forum Discussion

Jeroen_Gouma's avatar
5 years ago

How to make my datasource multi-instance

Hi,

I created a datasource which runs a script with a bunch of parameters to retreive acual cloud usage& costs for our customers.  Currently this script is running on the collector in the customers environment and working fine. 

But I would like to centralize these data collections 1 single place. I would like to have multiple instances from my script running on 1 system,. Based on the name of the instance I can retreive all the parameters from a parameter file.

How can I make multiple instances of my script?

Thanks a lot!

Jeroen

  • When you create a new Datasource you will have the option to check a "multi-instance" box. You can't easily change it after creating the DataSource as it's greyed out in the UI. You can either export and edit the XML directly or best to just create a new DataSource with that option enabled and copy over the settings/script. Once you enable that option you can optionally choose to use an Active Discovery script or not. If you disable this option, then you can use "add monitoring instance" on the device (resource tab) to manually add instances. If you enable Active Discovery, you can automate the addition of customer instances, assuming you have a source list you can query. Note that using BatchScript is generally more efficient than using plain Script when it can be used.

    Side note. It sounds like you might be an MSP? Keep in mind that DataSources will always run on the collector systems. So you don't want to collect data for one customer from another customer's collector. Best to do something like that from a collector you directly own. Unless by "customer" you mean your customer's customer.

  • Anonymous's avatar
    Anonymous

    Exactly right Mike.

    RE: Batchscript vs. Script

    It is generally correct that Batchscript runs more efficiently than script because the collector executes the script one time and the script collects the data for each and every instance with a single run. With Script, the collector runs the script once per instance. Script is sometimes/rarely/not often better in the case that the collection can't be done in bulk.

    For example, an API call has to be executed to collect the data and the API only offers the data when specifying the instance as part of the API resource (/api/{instanceID}/data). In this case, if you did a Batchscript, you would fetch the list of IDs and loop through each, executing the API call once per instance. Depending on the number of instances and the speed of the response from the API, this could end up timing out. This method executes the API calls serially.

    If you used Script instead, the Collector would execute once per instance, essentially running them in parallel. Each script would only be calling the API once so it should return much more quickly than running all fo them in a loop. You stand less of a chance of timing out in this case. However, care must be given to Collector performance since this method could significantly increase the number of tasks the Collector has to execute.

  • Thanks a lot for the swift reply. I noticed the multi-instance box, but indeed it was greyd-out. I will try this appoach and create a new version of the datasource, making it multi instance and see how I can query something that indicates the various instances. Beside that i will have to look into the BatchScript.

    We are indeed a MSP and the script will definitly run on one of our own collectors, providing the results only on the specific customer dashboards.

  • Anonymous's avatar
    Anonymous

    Good luck! The main thing to remember about Active Discovery is that it needs to return a list of instances, with a minimum of two values per instance: some kind of unique identifier and the displayName. Whatever unique identifier you choose will be addressable in the rest of the DataSource using the token ##WILDVALUE##.  ##WILDVALUE## is used to tie collected data back to the instances, so whatever you use for discovery should match what you use in polling. Whatever you choose as the displayName will be addressable in the rest of the DataSource using the token ##WILDALIAS##.