Forum Discussion

cvl's avatar
cvl
Icon for Neophyte rankNeophyte
4 months ago

datapoint added as a list?

Hello,

 

I have created a datasource, which polls every hour and extracts a list from a API. I was wondering if there was a way to add a list of elements extracted to a datapoint rather than the last element over writing the previous element during the datasource poll?

 

Regards

 

Conrad

 

 

  • Can you provide some more details and/or provide an example for what you are looking to do?

    Datapoints can only be a single number that is filled in at the time of running the DataSource. It can't be a list of numbers unless you try to encode it into a single number. But when you mean "list" do you mean there are multiple items that each have a number? Or do you mean you want to only query the API once but back fill data from the past hour? Or something else?

  • Hi Mike,

     

    what I mean is I get data like the below when I do a API call:

    Key 1,Datapoint 1, Datapoint2, Datapoint 3

    Key 2, Datapoint 1,Datapoint2,Datapoint 3

    Key 3,Datapoint 1, Datapoint 2 Datapoint 3

    I was wondering, if it was possible to add the values for datapoint1, to  a datapoint for all the keys? so when I look at the raw data I see all the values of the poll?

    • Mike_Moniz's avatar
      Mike_Moniz
      Icon for Professor rankProfessor

      So no, you can't do that. Datapoint value needs to be a single number.

      But can you provide a less generic example? There might be another way to do what you need but we would need to better understand the situation and data.

      For example perhaps you can have multiple Datapoints called "Datapoint1" and "Datapoint2" that each store the number (assuming the "list" is a fix size). Or you can multiple instances called "Key1.Datapoint1", "Key1.Datapoint2" or the like.

      • Mike_Moniz's avatar
        Mike_Moniz
        Icon for Professor rankProfessor

        Or perhaps if your Datapoint1 is the same for all the keys, perhaps you need to flip that around and make "Datapoint1" the instance and "key1", "key2", etc be the Datapoints.

  • Thanks Mike,

     

    I think I get , that it is not possible to have multiple values during a collection.

     

    Regards

     

    Conrad

  • Anonymous's avatar
    Anonymous

    It is possible to have multiple values during collection. But, your example is so generic, we're having a hard time understanding what is what to know how to explain what to map it to in LM.

    So this is your script output:

    Key 1,Datapoint 1, Datapoint2, Datapoint 3
    Key 2, Datapoint 1,Datapoint2,Datapoint 3
    Key 3,Datapoint 1, Datapoint 2 Datapoint 3

    Is your DS multi instance? Do you have active discovery setup to create the instances? Do the keys line up with instances?

    It appears that there are 3 "things" you want to monitor. In LM, these are called instances. So, you need a multi instance datasource because you have more than one thing that you want to monitor.

    For each thing, it appears you have three datapoints. To make this more specific, I'm going to apply this to a real world example: disk monitoring. Let's say that I have 3 disks (C:, D:, and E:), and there are three metrics for each disk (used%, free%, total_capacity in GB). Pretending to be you, i'd have a script that outputs something like what you see below. Let's pretend that this blob of text is stored in a variable called "output" in my script.

    println(output)

    C:,13,87,100
    D:,65,35,2000
    E:,1,99,500

    The first thing I'd need to setup is an active discovery, so that the three "things" i'm monitoring become instances in LM. An "instance" is something that LM knows about and tries to collect data for. My discovery script would need some additional modification in order to output what is required for LM to create the instances. The output format looks like this, with one line per thing:

    thingID##thingName

    Since my output is stored in a variable called "output" all I need to do is loop through that and print out the right thing in the right format for discovery (we're gonna do the same thing later for collection). What I'm showing here is Groovy, which is natively supported by LM. Your programming language will have syntactical differences, but the method is the same.

    output.eachLine{line->
        thingId = line.tokenize(",")[0]
        sanitizedThingId = thingId.replace(":","") //this is necessary because the wildvalue cannot contain =, :, \, #, space
        println(sanitizedThingId + "##" + thingId)
    }

    This will make the output of my script look like this instead:

    C##C:
    D##D:
    E##E:

    This is enough to register these three things as instances in LM. Now, we just need to look at collection. Collection is just a matter of outputting the data in a particular way. It needs to look like this, with one line per combination of thing and datapoint:

    thingId.datapointname: datapointvalue

    So, we need the output of collection to look like this:

    C.used: 13
    C.free: 87
    C.total_capacity: 100
    D.used: 65
    D.free: 35
    D.total_capacity: 2000
    E.used: 1
    E.free: 99
    E.total_capacity: 500
    

    So I'd go to my script again and loop through the output to get it to output something like what you see above.

    output.eachLine{line->
        (thingId, free, used, total_capacity) = line.tokenize(",")
        sanitizedThingId = thingId.replace(":","") //this is necessary because the wildvalue cannot contain =, :, \, #, space
        println(sanitizedThingId + ".free: " + free)
        println(sanitizedThingId + ".used: " + used)
        println(sanitizedThingId + ".total_capacity: " + total_capacity)
    }

    Then I need to add the free, used, and total capacity datapoints to my DS. When adding the datapoints, choose:

    • "Content the script writes to the standard output" as the "Datapoint source"
    • "Multi-line key-value pairs" as the "Interpret output with"
    • And for the key put "##WILDVALUE##.free", "##WILDVALUE##.used", "##WILDVALUE##.total_capacity", respectively.
  • Anonymous's avatar
    Anonymous

    I forgot to mention that this method is for "batchscript" type DS. I chose that type because it appears you have one script that returns all the data for all the things you want to monitor.

    "Script" would be used if your script would need to run once per thing, returning only the data for one thing each time it runs. Since that's not the case, batchscript is for you.