Dell ECS System Level Statistics Data Sources | LM Exchange

SteveBamford · 2025-12-16T07:26:25.030000Z

Dell made some changes to their ECS offering in version 3.6 where system level statistics such as CPU, Memory and Network were removed from the dashboard API and Flux Queries needed to be used to retrieve the data, below are two discovery and collection scripts one for CPU and Memory and one for Network Level statistics that utilize the flux query to retrieve the relevant metrics.Important note: this is for all versions above 3.6 of the Dell EMC ECS Solution all versions before are supported fully by the existing LogicMonitor out of the box packages.CPU and Memory Statistics:The following collection and discovery scripts retrieve the CPU and Memory statistics from the flux query API I would recommend keeping the collection frequency at 5 minutes.Discovery Script:/******************************************************************************* * Dell ECS Flux Query CPU and Memory Discovery Script ******************************************************************************/ import groovy.json.JsonSlurper import groovy.json.JsonOutput import groovy.json.JsonBuilder import java.util.concurrent.Executors import java.util.concurrent.TimeUnit import com.santaba.agent.groovyapi.http.*; import com.santaba.agent.util.Settings hostname = hostProps.get("system.hostname") user = hostProps.get("ecs.user") pass = hostProps.get("ecs.pass") collectorplatform = hostProps.get("system.collectorplatform") debug = false def success = false def token = login() // End Templines if (token) { //Retrieve data for all nodes for CPU and Memory def encoded_instance_props_array = [] //Use the flux call for getting the CPU to retrieve the Node information. Future Enhancement find a call for just the Nodes rather than the metrics call. def CPUresponse = getNode(token) if (debug) println "CPU Response: "+ CPUresponse // Work through table to retrieve the Node Name and Id to build out the instance level properties. // Internal Note used the methods in "Arista Campus PSU Collection Script" def CPUJson = new JsonSlurper().parseText(CPUresponse) if (debug) println "\n\n CPU Values: "+CPUJson.Series.Values[0] CPUJson.Series.Values[0].each { nodeEntry -> if (debug) println "In Table" if (debug) println "Node Data "+nodeEntry def nodeId = nodeEntry[9] def nodeName = nodeEntry[8] wildvalue = nodeId wildalias = nodeName description = "${nodeId}/${nodeName}" def instance_props = [ "auto.node.id": nodeId, "auto.node.name": nodeName ] encoded_instance_props_array = instance_props.collect() { property, value -> URLEncoder.encode(property.toString()) + "=" + URLEncoder.encode(value.toString()) } println "${wildvalue}##${wildalias}##${description}####${encoded_instance_props_array.join("&")}" } } else if (debug) { println "Bad API response: ${response}"} return 0 def login() { if (debug) println "in login" // Fetch new token using Basic authentication, set in cache file and return if (debug) println "Checking provided ${user} creds at /login.json..." def userCredentials = "${user}:${pass}" def basicAuthStringEnc = new String(Base64.getEncoder().encode(userCredentials.getBytes())) def loginUrl = "https://${hostname}:4443/login.json".toURL() def loginConnection = loginUrl.openConnection() loginConnection.setRequestProperty("Authorization", "Basic " + basicAuthStringEnc) def loginResponseBody = loginConnection.getInputStream()?.text def loginResponseCode = loginConnection.getResponseCode() def loginResponseToken = loginConnection.getHeaderField("X-SDS-AUTH-TOKEN") if (debug) println loginResponseCode if (loginResponseCode == 200 && loginResponseToken) { if (debug) println "Retrieved token: ${loginResponseToken}" return loginResponseToken } else { println "STATUS CODE:\n${loginResponseCode}\n\nRESPONSE:\n${loginResponseBody}" println "Unable to fetch token with ${user} creds at /login.json" } println "Something unknown went wrong when logging in" } def getNode(token) { def slurper = new JsonSlurper() def dataUrl = "https://"+hostname+":4443/flux/api/external/v2/query" if (debug) println "Trying to fetch data from ${dataUrl}" //def flux = 'from(bucket:\"monitoring_op\") |> range(start: -5m) |> filter(fn: (r) => r._measurement == \"cpu\" and r.cpu == \"cpu-total\" and r._field == \"usage_idle\" and r.host == \"'+hostname+'\")' def flux = 'from(bucket:\"monitoring_op\") |> range(start: -5m) |> filter(fn: (r) => r._measurement == \"cpu\" and r.cpu == \"cpu-total\" and r._field == \"usage_idle\")' def jsonBody = groovy.json.JsonOutput.toJson(["query":flux]) if (debug) println "Raw JSON Body "+jsonBody if (debug) println "Json Body "+JsonOutput.prettyPrint(jsonBody)+" Type "+jsonBody.getClass() def dataHeader = ["X-SDS-AUTH-TOKEN": token,"Content-Type":"application/json","Accept":"application/json"] if (debug) println("Sent Header: "+dataHeader) // Now we can retrieve the data. def httpClient = Client.open (hostname,4443); httpClient.post(dataUrl,jsonBody,dataHeader); if ( !(httpClient.getStatusCode() =~ /200/)) { println "Failed to retrieve data "+httpClient.getStatusCode() println "Header: "+httpClient.getHeader return(1) } String dataContent = httpClient.getResponseBody() if (debug) println "Status Code "+httpClient.getStatusCode() if (debug) println "Data in response Body "+dataContent //return slurper.parseText(dataContent) return dataContent }Collection Script:/******************************************************************************* * Dell ECS Flux Query CPU and Memory ******************************************************************************/ import groovy.json.JsonSlurper import groovy.json.JsonOutput import groovy.json.JsonBuilder import java.util.concurrent.Executors import java.util.concurrent.TimeUnit import com.santaba.agent.groovyapi.http.*; import com.santaba.agent.util.Settings hostname = hostProps.get("system.hostname") user = hostProps.get("ecs.user") pass = hostProps.get("ecs.pass") collectorplatform = hostProps.get("system.collectorplatform") debug = true def success = false def token = login() // End Templines if (token) { //Retrieve data for all nodes for CPU and Memory def encoded_instance_props_array = [] def CPUresponse = getCPU(token) def MEMresponse = getMemory(token) if (debug) println "CPU Response: "+ CPUresponse if (debug) println "Mem Response: "+ MEMresponse // Work through table to retrieve the Node Name and Id to build out the instance level properties. // Internal Note used the methods in "Arista Campus PSU Collection Script" //Process CPU Metrics def CPUJson = new JsonSlurper().parseText(CPUresponse) if (debug) println "\n\n CPU Values: "+CPUJson.Series.Values[0] CPUJson.Series.Values[0].each { nodeEntry -> if (debug) println "Node Data "+nodeEntry def idleCPU = Float.valueOf(nodeEntry[4]) def usedCPU = 100 - idleCPU def nodeId = nodeEntry[9] def nodeName = nodeEntry[8] wildvalue = nodeId wildalias = nodeName description = "${nodeId}/${nodeName}" println "${wildvalue}.idle_cpu=${idleCPU}" println "${wildvalue}.used_cpu=${usedCPU}" } // Process Memory Metrics def MEMJson = new JsonSlurper().parseText(MEMresponse) if (debug) println "\n\n Mem Values: "+MEMJson.Series.Values[0] MEMJson.Series.Values[0].each { nodeEntry -> def fieldValue = nodeEntry[4] def fieldName = nodeEntry[5] def nodeId = nodeEntry[8] def nodeName = nodeEntry[7] wildvalue = nodeId wildalias = nodeName description = "${nodeId}/${nodeName}" println "${wildvalue}.${fieldName}=${fieldValue}" } } else if (debug) { println "Bad API response: ${response}"} return 0 def login() { if (debug) println "in login" // Fetch new token using Basic authentication, set in cache file and return if (debug) println "Checking provided ${user} creds at /login.json..." def userCredentials = "${user}:${pass}" def basicAuthStringEnc = new String(Base64.getEncoder().encode(userCredentials.getBytes())) def loginUrl = "https://${hostname}:4443/login.json".toURL() def loginConnection = loginUrl.openConnection() loginConnection.setRequestProperty("Authorization", "Basic " + basicAuthStringEnc) def loginResponseBody = loginConnection.getInputStream()?.text def loginResponseCode = loginConnection.getResponseCode() def loginResponseToken = loginConnection.getHeaderField("X-SDS-AUTH-TOKEN") if (debug) println loginResponseCode if (loginResponseCode == 200 && loginResponseToken) { if (debug) println "Retrieved token: ${loginResponseToken}" return loginResponseToken } else { println "STATUS CODE:\n${loginResponseCode}\n\nRESPONSE:\n${loginResponseBody}" println "Unable to fetch token with ${user} creds at /login.json" } println "Something unknown went wrong when logging in" } def getCPU(token) { def slurper = new JsonSlurper() def dataUrl = "https://"+hostname+":4443/flux/api/external/v2/query" if (debug) println "Trying to fetch data from ${dataUrl}" def flux = 'from(bucket:\"monitoring_op\") |> range(start: -5m) |> filter(fn: (r) => r._measurement == \"cpu\" and r.cpu == \"cpu-total\" and r._field == \"usage_idle\")' def jsonBody = groovy.json.JsonOutput.toJson(["query":flux]) if (debug) println "Raw JSON Body "+jsonBody if (debug) println "Json Body "+JsonOutput.prettyPrint(jsonBody)+" Type "+jsonBody.getClass() def dataHeader = ["X-SDS-AUTH-TOKEN": token,"Content-Type":"application/json","Accept":"application/json"] if (debug) println("Sent Header: "+dataHeader) // Now we can retrieve the data. def httpClient = Client.open (hostname,4443); httpClient.post(dataUrl,jsonBody,dataHeader); if ( !(httpClient.getStatusCode() =~ /200/)) { println "Failed to retrieve data "+httpClient.getStatusCode() println "Header: "+httpClient.getHeader return(1) } String dataContent = httpClient.getResponseBody() if (debug) println "Status Code "+httpClient.getStatusCode() if (debug) println "Data in response Body "+dataContent return dataContent } def getMemory(token) { def slurper = new JsonSlurper() def dataUrl = "https://"+hostname+":4443/flux/api/external/v2/query" if (debug) println "Trying to fetch data from ${dataUrl}" //def flux = 'from(bucket:\"monitoring_op\") |> range(start: -5m) |> filter(fn: (r) => r._measurement == \"mem\" and r._field == \"available_percent\")' def flux = 'from(bucket:\"monitoring_op\") |> range(start: -5m) |> filter(fn: (r) => r._measurement == \"mem\")' def jsonBody = groovy.json.JsonOutput.toJson(["query":flux]) if (debug) println "Raw JSON Body "+jsonBody if (debug) println "Json Body "+JsonOutput.prettyPrint(jsonBody)+" Type "+jsonBody.getClass() def dataHeader = ["X-SDS-AUTH-TOKEN": token,"Content-Type":"application/json","Accept":"application/json"] if (debug) println("Sent Header: "+dataHeader) // Now we can retrieve the data. def httpClient = Client.open (hostname,4443); httpClient.post(dataUrl,jsonBody,dataHeader); if ( !(httpClient.getStatusCode() =~ /200/)) { println "Failed to retrieve data "+httpClient.getStatusCode() println "Header: "+httpClient.getHeader return(1) } String dataContent = httpClient.getResponseBody() if (debug) println "Status Code "+httpClient.getStatusCode() if (debug) println "Data in response Body "+dataContent return dataContent }Networking Statistics:See the link https://community.logicmonitor.com/discussions/lm-exchange/dell-ecs-network-statistics-version-3-6-flux-query/19817for the network statistics as I ran out of space.Additional comments and notes:One of the biggest challenges I had solving this change from the Dashboard API to the Flux API was that I was receiving a HTTP 401 initially I thought this was the flux query however it turned out to be the saving of the token to the file as per the original data sources, once I removed this and made it the same as my Python script which worked with out issue I resolved this issue.I have an additional request for the Latency statistics, I will share these in a separate post once done.Hope this helps.

MoreDiscussions

Dell ECS System Level Statistics Data Sources | LM Exchange | LogicMonitor Community