Is there a way to set custom property values from your groovy script in your datasource?

Next hiccup.  I have the script built out to the point where to GETs eventually pull in the various instance IDs attached to the device.  Works perfectly for one device.  The other.  I get another damn red text error.  

Although i believe this one may just be an error in how info is cached in the test box

The script failed, elapsed time: 0 seconds - micoresolutions.logicmonitor.comjava.net.UnknownHostException: micoresolutions.logicmonitor.com	at java.base/java.net.InetAddress$CachedAddresses.get(InetAddress.java:797)	at java.base/java.net.InetAddress.getAllByName0(InetAddress.java:1519)	at java.base/java.net.InetAddress.getAllByName(InetAddress.java:1378)	at java.base/java.net.InetAddress.getAllByName(InetAddress.java:1306)	at org.apache.http.impl.conn.SystemDefaultDnsResolver.resolve(SystemDefaultDnsResolver.java:45)	at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:112)	at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376)	at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393)	at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236)	at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186)	at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89)	at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110)	at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185)	at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83)	at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:108)	at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:56)	at org.apache.http.client.HttpClient$execute.call(Unknown Source)	at Script11.run(Script11.groovy:60)	

This is the whole thing so far.   Possible errors still.  im gonna try switching tot he debug window tester.

Ah, Groovy error messages. So helpful.

FYI, this:

datapull = new JsonSlurper().parseText(responseBody.toString());

datapull =   datapull.data.items.id.value.toString()

datapull = datapull.replace("[","")

datasouceid = datapull.replace("]","")

Can be just this:

datapull = new JsonSlurper().parseText(responseBody.toString())[0]

Are you closing something that doesn’t exist?

httpdsid.close()
httpinstanceid.close()

Where are these objects defined?

Gonna call it a day for now.  I think tomorrow I will be able to add the last bit of magic to get this all to work.

So the script works great for my main test device.  However the collector does keep throwing that error

“java.net.UnknownHostException: micoresolutions.logicmonitor.com”

When I run it on another device.  It is still the same account info so nothing should be different about it.

As a follow up, this specifically seems to only be affecting our other two virtual machines in our training environment here at the office

Is it micoresolutions or microresolutions?

hmmm they are monitored by their own collectors.  I am gonna go through them both to see if anything is different or maybe I messed something up in all my previous tests and failed scriptings.  

Well, I still haven’t been able to resolve this issue with a few of our training devices.  That aside, the modified script works as intended and I finally have a solution to my original problem posed in first message of this post.  

Huge thanks to Stuart for letting me bug him every step of the way.

So the solution was to use API calls within int he collector attribute script inside the mysql global performance datasource.  

I won’t share the whole script but I will post parts that seem relevant to my issue and solution.

A lot of extra imports were needed to get this working and before the base script starts i establish to GET requests to obtain info needed to get to my final GET and PUT request 

import groovy.sql.Sql

import com.santaba.agent.groovyapi.http.*;

import groovy.json.JsonSlurper;

import org.apache.http.HttpEntity

import org.apache.http.client.methods.CloseableHttpResponse

import org.apache.http.client.methods.HttpGet

import org.apache.http.client.methods.HttpPut

import org.apache.http.impl.client.CloseableHttpClient

import org.apache.http.impl.client.HttpClients

import org.apache.http.util.EntityUtils

import javax.crypto.Mac;

import javax.crypto.spec.SecretKeySpec;

import org.apache.commons.codec.binary.Hex;

import groovy.json.*;

import org.apache.http.entity.ContentType;

import org.apache.http.entity.StringEntity;

import org.apache.http.client.utils.URIBuilder










// ***  Establish API Creds  ***

def accessId = hostProps.get("lmaccess.id");

def accessKey = hostProps.get("lmaccess.key");

def account = 'micoresolutions';




// *** Get Datasource ID for device as string. ***

def resourcePath = '/device/devices/'+hostProps.get("system.deviceID")+'/devicedatasources'

def url = "https://" + account + ".logicmonitor.com" + "/santaba/rest" + resourcePath;



    //get current time

epoch = System.currentTimeMillis();



    //calculate signature

requestVars = "GET" + epoch + resourcePath;



hmac = Mac.getInstance("HmacSHA256");

secret = new SecretKeySpec(accessKey.getBytes(), "HmacSHA256");

hmac.init(secret);

hmac_signed = Hex.encodeHexString(hmac.doFinal(requestVars.getBytes()));

signature = hmac_signed.bytes.encodeBase64();



    // HTTP GET

CloseableHttpClient httpdsid = HttpClients.createDefault();

httpGet = new HttpGet(url);

    // Add custom filter values to GET Request

URI uri = new URIBuilder(httpGet.getURI())

      .addParameter("fields", "id")

      .addParameter("filter", "dataSourceName:Microsoft_SQLServer_GlobalPerformance")

      .build();

      httpGet.setURI(uri);



httpGet.addHeader("Authorization" , "LMv1 " + accessId + ":" + signature + ":" + epoch);

response = httpdsid.execute(httpGet);

responseBody = EntityUtils.toString(response.getEntity());

    // Turn data into id string that can be inserted into next Request

datapull = new JsonSlurper().parseText(responseBody.toString());

datapull =   datapull.data.items.id.value.toString()

datapull = datapull.replace("[","")

datasouceid = datapull.replace("]","")



httpdsid.close()



//  *** get instance ID for each instance on device ***



 resourcePath = "/device/devices/"+hostProps.get("system.deviceID")+"/devicedatasources/${datasouceid}/instances"

 url = "https://" + account + ".logicmonitor.com" + "/santaba/rest" + resourcePath;



    //get current time

epoch = System.currentTimeMillis();



    //calculate signature

requestVars = "GET" + epoch + resourcePath;



hmac = Mac.getInstance("HmacSHA256");

secret = new SecretKeySpec(accessKey.getBytes(), "HmacSHA256");

hmac.init(secret);

hmac_signed = Hex.encodeHexString(hmac.doFinal(requestVars.getBytes()));

signature = hmac_signed.bytes.encodeBase64();



    // HTTP GET

CloseableHttpClient httpinstanceid = HttpClients.createDefault();

httpGet = new HttpGet(url);

httpGet.addHeader("Authorization" , "LMv1 " + accessId + ":" + signature + ":" + epoch);

response = httpinstanceid.execute(httpGet);

responseBody = EntityUtils.toString(response.getEntity());

    // Male array of instance ids for final get and put request

datapull = new JsonSlurper().parseText(responseBody.toString());

instanceids =   datapull.data.items.id



httpinstanceid.close()

My final resource path --- "/device/devices/"+hostProps.get("system.deviceID")+"/devicedatasources/${datasouceid}/instances/"+instanceids[i]

requires a deviceID, a datasourceID and an instanceID.  So the first get pulls the correct datasource ID and the second get uses that datasourceID to grab the instance IDs.

Then Once the collector begins its SQL query loop to find the values for all those datapoints, I insert my final GET and PUT, as well as using a custom SQL query script to find the blocker info I need.  

 if ("${counter_name}"=="Processesblocked")

                        {

// ***Follow up if statement that decides between which Put request to use based on whether there is in fact a block or not. ***

                            if ("${cntr_value}".toInteger() > 0)

                            {

// ***For loop to cycle through each instance on the device pulled from instance array***

                                for(i=0; i < instanceids.size; i++)

                                {



                                    resourcePath = "/device/devices/"+hostProps.get("system.deviceID")+"/devicedatasources/${datasouceid}/instances/"+instanceids[i]

                                    url = "https://" + account + ".logicmonitor.com" + "/santaba/rest" + resourcePath;



                                    //get current time

                                    epoch = System.currentTimeMillis();



                                    //calculate signature

                                    requestVars = "GET" + epoch + resourcePath;



                                    hmac = Mac.getInstance("HmacSHA256");

                                    secret = new SecretKeySpec(accessKey.getBytes(), "HmacSHA256");

                                    hmac.init(secret);

                                    hmac_signed = Hex.encodeHexString(hmac.doFinal(requestVars.getBytes()));

                                    signature = hmac_signed.bytes.encodeBase64();



                                    // HTTP GET

                                    CloseableHttpClient httpinprop = HttpClients.createDefault();

                                    httpGet = new HttpGet(url);

                                    httpGet.addHeader("Authorization" , "LMv1 " + accessId + ":" + signature + ":" + epoch);

                                    response = httpinprop.execute(httpGet);

                                    responseBody = EntityUtils.toString(response.getEntity());

                                    code = response.getStatusLine().getStatusCode();

                                    datapull = new JsonSlurper().parseText(responseBody.toString());

                                    instanceprops =   datapull.data

// ***if statement to ensure that the put request only applies to current SQL instance iteration by matching the instance displayname property with the SQL instancename ***

                                    if ("${instanceprops.displayName}"=="${instanceName}")

                                    {

                                       // ***SQL query to find blocker data***

                                        outputBlocks = runQuery("""SELECT SPID, BLOCKED, REPLACE (REPLACE (T.TEXT, CHAR(10), ' '), CHAR (13), ' ' ) AS BATCH INTO #T FROM sys.sysprocesses R CROSS APPLY sys.dm_exec_sql_text(R.SQL_HANDLE) T; WITH BLOCKERS (SPID, BLOCKED, LEVEL, BATCH) AS (SELECT SPID, BLOCKED, CAST (REPLICATE ('0', 4-LEN (CAST (SPID AS VARCHAR))) + CAST (SPID AS VARCHAR) AS VARCHAR (1000)) AS LEVEL, BATCH FROM #T R WHERE (BLOCKED = 0 OR BLOCKED = SPID) AND EXISTS (SELECT * FROM #T R2 WHERE R2.BLOCKED = R.SPID AND R2.BLOCKED <> R2.SPID) UNION ALL SELECT R.SPID, R.BLOCKED, CAST (BLOCKERS.LEVEL + RIGHT (CAST ((1000 + R.SPID) AS VARCHAR (100)), 4) AS VARCHAR (1000)) AS LEVEL, R.BATCH FROM #T AS R INNER JOIN BLOCKERS ON R.BLOCKED = BLOCKERS.SPID WHERE R.BLOCKED > 0 AND R.BLOCKED <> R.SPID ) SELECT N'    ' + REPLICATE (N'|         ', LEN (LEVEL)/4 - 1) + CASE WHEN (LEN(LEVEL)/4 - 1) = 0 THEN 'HEAD -  ' ELSE '|------  ' END + CAST (SPID AS NVARCHAR (10)) + N' ' + BATCH AS BLOCKING_TREE FROM BLOCKERS ORDER BY LEVEL ASC DROP TABLE #T;""", conn.connection, debug)



                                        blockerid = "${outputBlocks.data.blocking_tree}"



                                        instanceprops.customProperties = [[name:"blockerdetails", value:"${blockerid}"]]

                                        instanceprops = JsonOutput.toJson(instanceprops)

                                        StringEntity params = new StringEntity(instanceprops,ContentType.APPLICATION_JSON)



                                        epoch = System.currentTimeMillis(); //get current time



                                        requestVars = "PUT" + epoch + instanceprops + resourcePath;



                                        hmac = Mac.getInstance("HmacSHA256");

                                        secret = new SecretKeySpec(accessKey.getBytes(), "HmacSHA256");

                                        hmac.init(secret);

                                        hmac_signed = Hex.encodeHexString(hmac.doFinal(requestVars.getBytes()));

                                        signature = hmac_signed.bytes.encodeBase64();



                                        httpPut = new HttpPut(url);

                                        httpPut.addHeader("Authorization" , "LMv1 " + accessId + ":" + signature + ":" + epoch);

                                        httpPut.setHeader("Accept", "application/json");

                                        httpPut.setHeader("Content-type", "application/json");

                                        httpPut.setEntity(params);

                                        responsePut = httpinprop.execute(httpPut);

                                        responseBodyPut = EntityUtils.toString(responsePut.getEntity());

                                        codePut = responsePut.getStatusLine().getStatusCode();



                                    }

                                       



                                    httpinprop.close()

                                }



                            }

                            else

                            {

                                for(i=0; i < instanceids.size; i++)

                                {

                                   

                                    resourcePath = "/device/devices/"+hostProps.get("system.deviceID")+"/devicedatasources/${datasouceid}/instances/"+instanceids[i]

                                    url = "https://" + account + ".logicmonitor.com" + "/santaba/rest" + resourcePath;



                                    //get current time

                                    epoch = System.currentTimeMillis();



                                    //calculate signature

                                    requestVars = "GET" + epoch + resourcePath;



                                    hmac = Mac.getInstance("HmacSHA256");

                                    secret = new SecretKeySpec(accessKey.getBytes(), "HmacSHA256");

                                    hmac.init(secret);

                                    hmac_signed = Hex.encodeHexString(hmac.doFinal(requestVars.getBytes()));

                                    signature = hmac_signed.bytes.encodeBase64();



                                    // HTTP GET

                                    CloseableHttpClient httpinprop = HttpClients.createDefault();

                                    httpGet = new HttpGet(url);

                                    httpGet.addHeader("Authorization" , "LMv1 " + accessId + ":" + signature + ":" + epoch);

                                    response = httpinprop.execute(httpGet);

                                    responseBody = EntityUtils.toString(response.getEntity());

                                    code = response.getStatusLine().getStatusCode();

                                    datapull = new JsonSlurper().parseText(responseBody.toString());

                                    instanceprops =   datapull.data

// ***if statement to ensure that the put request only applies to current SQL instance iteration by matching the instance displayname property with the SQL instancename ***

                                    if ("${instanceprops.displayName}"=="${instanceName}")

                                    {

                                        blockerid = "No Blocks"



                                        instanceprops.customProperties = [[name:"blockerdetails", value:"${blockerid}"]]

                                        instanceprops = JsonOutput.toJson(instanceprops)

                                        StringEntity params = new StringEntity(instanceprops,ContentType.APPLICATION_JSON)



                                        epoch = System.currentTimeMillis(); //get current time



                                        requestVars = "PUT" + epoch + instanceprops + resourcePath;



                                        hmac = Mac.getInstance("HmacSHA256");

                                        secret = new SecretKeySpec(accessKey.getBytes(), "HmacSHA256");

                                        hmac.init(secret);

                                        hmac_signed = Hex.encodeHexString(hmac.doFinal(requestVars.getBytes()));

                                        signature = hmac_signed.bytes.encodeBase64();



                                        httpPut = new HttpPut(url);

                                        httpPut.addHeader("Authorization" , "LMv1 " + accessId + ":" + signature + ":" + epoch);

                                        httpPut.setHeader("Accept", "application/json");

                                        httpPut.setHeader("Content-type", "application/json");

                                        httpPut.setEntity(params);

                                        responsePut = httpinprop.execute(httpPut);

                                        responseBodyPut = EntityUtils.toString(responsePut.getEntity());

                                        codePut = responsePut.getStatusLine().getStatusCode();

                                    }

                                    httpinprop.close()

                                }

Now when the collector does it’s thing it will run that SQL query and per instance, update my custom property with either the blocker details or “no blocks”.

Actually, I dipped into your github and tested out your version already.  I love how it’s cleaned up but unfortunately, in regard to that issue with the two training devices, that unknownhostexception persists.  
 

It seems the issue has something to do with API calls in general, and if I can get anything out of support, I’ll post an update.  But for the purpose of the original post, I’d say mission accomplished.  

Good idea, and after testing it, it works fine through powershell and bash.  So looks like it is a collector issue.