Silent Log Source Detection
We are working on migrating some items to LM Logs and one thing that came up was the lack of Silent Log Source detection. Log Usage is a PushModule and only updates when data is pushed to it. So unlike a traditional DataSource, you cannot alert on No Data or 0 events for X period of time. With that in mind, we wrote our own, which I am sharing here instead of in the LM Exchange as it will require a bit of changing on your own. So, this could be 100% portable, but I purposefully didn't do that to save on API calls. That this DataSource will do, is make two api calls per device, back to the LogicMonitor portal. The first is to retrieve the instances of LogUsage, which is defined in line 22. That is where you would need to change the ID to match what is in your portal. Then after it retrieves the instance of LogUsage, it queries data for said instance. That returns a JSON structure like this. { "dataPoints": [ "size_in_bytes", "event_received" ], "dataSourceName": "LogUsage", "nextPageParams": "start=1739704983&end=1741035719", "time": [ 1741110300000 ], "values": [ [ 25942, 16 ] } There will generally be more items in time and in values. We grab the first element, so [0] of the time array as that is the latest timestamp. Then we calculate the difference between now and that timestamp in hours. Then we return the difference in hours, or a NaN. /******************************************************************************* * © 2007-2024 - LogicMonitor, Inc. All rights reserved. ******************************************************************************/ import groovy.json.JsonSlurper import com.santaba.agent.util.Settings import com.santaba.agent.live.LiveHostSet import org.apache.commons.codec.binary.Hex import javax.crypto.Mac import javax.crypto.spec.SecretKeySpec String apiId = hostProps.get("lmaccess.id") ?: hostProps.get("logicmonitor.access.id") String apiKey = hostProps.get("lmaccess.key") ?: hostProps.get("logicmonitor.access.key") def portalName = hostProps.get("lmaccount") ?: Settings.getSetting(Settings.AGENT_COMPANY) String deviceid = hostProps.get("system.deviceId") Map proxyInfo = getProxyInfo() def fields = 'id,dataSourceId,deviceDataSourceId,name,lastCollectedTime,lastUpdatedTime,deviceDataSourceId' def apipath = "/device/devices/" + deviceid + "/instances" def apifilter = 'dataSourceId:43806042' def deviceinstances = apiGetMany(portalName, apiId, apiKey, apipath, proxyInfo, ['size':1000, 'fields': fields, 'filter': apifilter]) instanceid = deviceinstances[0]['id'] devicedatasourceid = deviceinstances[0]['deviceDataSourceId'] def instancepath = "/device/devices/" + deviceid + "/devicedatasources/" + devicedatasourceid + "/instances/" + instanceid + '/data' def instancedata = apiGet(portalName, apiId, apiKey, instancepath, proxyInfo, ['period':720]) def now = System.currentTimeMillis() if (instancedata['time'][0] != null && instancedata['time'][0] > 0) { def diffHours = ((instancedata['time'][0] - now) / (1000 * 60 * 60)).toDouble().round(2) println "hours_since_log=${diffHours}" } else { def diffHours = "NaN" println "hours_since_log=${diffHours}" } // If script gets to this point, collector should consider this device alive keepAlive(hostProps) return 0 /* Paginated GET method. Returns a list of objects. */ List apiGetMany(portalName, apiId, apiKey, endPoint, proxyInfo, Map args=[:]) { def pageSize = args.get('size', 1000) // Default the page size to 1000 if not specified. List items = [] args['size'] = pageSize def pageCount = 0 while (true) { pageCount += 1 // Updated the args args['size'] = pageSize args['offset'] = items.size() def response = apiGet(portalName, apiId, apiKey, endPoint, proxyInfo, args) if (response.get("errmsg", "OK") != "OK") { throw new Exception("Santaba returned errormsg: ${response?.errmsg}") } items.addAll(response.items) // If we recieved less than we asked for it means we are done if (response.items.size() < pageSize) break } return items } /* Simple GET, returns a parsed json payload. No processing. */ def apiGet(portalName, apiId, apiKey, endPoint, proxyInfo, Map args=[:]) { def request = rawGet(portalName, apiId, apiKey, endPoint, proxyInfo, args) if (request.getResponseCode() == 200) { def payload = new JsonSlurper().parseText(request.content.text) return payload } else { throw new Exception("Server return HTTP code ${request.getResponseCode()}") } } /* Raw GET method. */ def rawGet(portalName, apiId, apiKey, endPoint, proxyInfo, Map args=[:]) { def auth = generateAuth(apiId, apiKey, endPoint) def headers = ["Authorization": auth, "Content-Type": "application/json", "X-Version":"3", "External-User":"true"] def url = "https://${portalName}.logicmonitor.com/santaba/rest${endPoint}" if (args) { def encodedArgs = [] args.each{ k,v -> encodedArgs << "${k}=${java.net.URLEncoder.encode(v.toString(), "UTF-8")}" } url += "?${encodedArgs.join('&')}" } def request if (proxyInfo.enabled) { request = url.toURL().openConnection(proxyInfo.proxy) } else { request = url.toURL().openConnection() } request.setRequestMethod("GET") request.setDoOutput(true) headers.each{ k,v -> request.addRequestProperty(k, v) } return request } /* Generate auth for API calls. */ static String generateAuth(id, key, path) { Long epoch_time = System.currentTimeMillis() Mac hmac = Mac.getInstance("HmacSHA256") hmac.init(new SecretKeySpec(key.getBytes(), "HmacSHA256")) def signature = Hex.encodeHexString(hmac.doFinal("GET${epoch_time}${path}".getBytes())).bytes.encodeBase64() return "LMv1 ${id}:${signature}:${epoch_time}" } /* Helper method to remind the collector this device is not dead */ def keepAlive(hostProps) { // Update the liveHost set so tell the collector we are happy. hostId = hostProps.get("system.deviceId").toInteger() def liveHostSet = LiveHostSet.getInstance() liveHostSet.flag(hostId) } /** * Get collector proxy settings * @return Map with proxy settings, empty map if proxy not set. */ Map getProxyInfo() { // Each property must be evaluated for null to determine whether to use collected value or fallback value // Elvis operator does not play nice with booleans // default to true in absence of property to use collectorProxy as determinant Boolean deviceProxy = hostProps.get("proxy.enable")?.toBoolean() deviceProxy = (deviceProxy != null) ? deviceProxy : true // if settings are not present, value should be false Boolean collectorProxy = Settings.getSetting("proxy.enable")?.toBoolean() collectorProxy = (collectorProxy != null) ? collectorProxy : false Map proxyInfo = [:] if (deviceProxy && collectorProxy) { proxyInfo = [ enabled : true, host : hostProps.get("proxy.host") ?: Settings.getSetting("proxy.host"), port : hostProps.get("proxy.port") ?: Settings.getSetting("proxy.port") ?: 3128, user : Settings.getSetting("proxy.user"), pass : Settings.getSetting("proxy.pass") ] proxyInfo["proxy"] = new Proxy(Proxy.Type.HTTP, new InetSocketAddress(proxyInfo.host, proxyInfo.port.toInteger())) } return proxyInfo }
Anyone know of a way to monitor for SNapshot age on a Hyper-V machine?
We have some checks that monitor our VMWare system for Snapshots. Once they are over 72 hour old, we get an alert. I haven’t been able to find a way to do the same thing for our Hyper-V servers. Does anyone happen to know if that’s possible? I didn’t see anything in the Exchange but just through I’d ‘check in case someone knew of anything. I can pull the data with Powershell via the get-vm|get-vmsnapshot command. Not sure if that’s usable in LM somehow. Thanks.How can I tell what's been customized on a DataSource?
Hi, When I look at the toolbox area, the datasources have an icon that tells me if it's been Customized. How can I tell What has been customized if there aren't any notes in the Local History section? Is there a way to do a comparison between the original somehow if there's no update available for me to compare to? Thanks.New UI Impact Series - Datapoint & Log Analysis
LogicMonitor's new Datapoint Analysis and Log Analysis features are revolutionary tools designed to transform how IT professionals approach troubleshooting. These AI-powered features provide automated insights that significantly reduce the time and effort required to identify and resolve critical issues. By leveraging advanced correlation techniques and sentiment analysis, these tools offer a streamlined approach to problem-solving, allowing teams to quickly pinpoint the root cause of alerts and minimize downtime. Their user-friendly design ensures that even less experienced team members can use them effectively, empowering the entire team and boosting efficiency. Datapoint Analysis stands out with its ability to correlate metrics across various DataSources, creating a comprehensive correlation score. This score is calculated by: Other datapoints from the same instance The same datapoint on other instances within the same resource The same datapoint on other resources that share the same collector ID This comprehensive approach eliminates hours of manual investigation, instilling confidence in the analysis results. Log Analysis uses sentiment and keyword analysis to distill large volumes of log data into concise, actionable summaries. Its interactive visualizations allow for quick data refinement without complex queries, providing an intuitive interface for log exploration. So, what’s the impact for you? Datapoint and Log Analysis significantly reduce the MTTR for critical issues by offering rapid access to correlated metrics and summarized log insights. This reduction in resolution time not only improves system availability and performance but also relieves the stress and pressure on IT teams. These tools are seamlessly integrated throughout the LM Envision platform, accessible from alerts, dashboards, graphs, and the Resource Explorer, ensuring that users can leverage these powerful insights at any stage of their troubleshooting process. For organizations dealing with complex, distributed systems, these features represent a quantum leap in operational efficiency, enabling teams to maintain high system availability and performance with unprecedented ease and speed. Want to know more about Datapoint & Log Analysis? Check out these resources: Datapoint Analysis Datapoint Analysis Overview Datapoint Analysis Demonstration Video Log Analysis Accessing Log Analysis Log Analysis Overview Log Analysis Widgets Log Analysis Demonstration Video
DataSource PowerShell Active Discovery and Collector scripts
Hello LM Community! I am trying to configure a new DataSource that uses PowerShell scripts for both Active Discovery and Collector Attributes with the end goal being to populate the discovered devices/instances with data. My Active Discovery script is as follows (I've left out the functions that generate bearer tokens and make the MS Graph calls). $tenantID = '<tenantId>' $appID = '<appId>' $appSecret = '<appSecret>' $appCredentials = New-Object System.Management.Automation.PSCredential($appID, (ConvertTo-SecureString $appSecret -AsPlainText -Force)) $authToken = Get-MSGraphAuthToken -Credential $appCredentials -TenantID $tenantID $resourceURL = "https://graph.microsoft.com/v1.0/servicePrincipals" $applications = (Invoke-MSGraphQuery -method GET -Uri $resourceURL -token $authToken -Recursive).value | Where-Object {$_.preferredSingleSignOnMode -eq "saml"} foreach ($application in $applications) { Write-Host "$($application.appId)##$($application.appDisplayName)" } This successfully imports all of the instances as I would expect with their respective 'appId' and 'appDisplayName'. These can be seen under the Resources tab in LM. My Collector Attributes script is as follows (again functions left out): $tenantID = '<tenantId>' $appID = '<appId>' $appSecret = '<appSecret>' $global:appCredentials = New-Object System.Management.Automation.PSCredential($appID, (ConvertTo-SecureString $appSecret -AsPlainText -Force)) $global:authToken = Get-MSGraphAuthToken -Credential $appCredentials -TenantID $tenantID $resourceURL = "https://graph.microsoft.com/v1.0/servicePrincipals" $applications = (Invoke-MSGraphQuery -method GET -Uri $resourceURL -token $authToken -Recursive).value | Where-Object {$_.preferredSingleSignOnMode -eq "saml"} $todayDate = Get-Date foreach ($application in $applications) { $expirationDate = $application.keyCredentials.endDateTime[0] | Get-Date -Format yyyy-MM-dd $daysToExpiration = (New-TimeSpan -Start $todayDate -End $($application.keyCredentials.endDateTime[0])).Days $data = [PSCustomObject]@{ appId = $application.appId expirationDate = $expirationDate daysToExpiration = $daysToExpiration } | ConvertTo-Json $data } When I test this script, I select the Device to run it from and the Instance to run it against. The output contains the 'appId', 'expirationDate' and 'daysToExpiration' properties of all the instances, and not just the instance I want to match the latter two data points to. This would be as expected when looking at the script, but not what I want to achieve. I don't understand how the Collector script is supposed to match the values of the properties to their respective Instances. Could someone please explain this to me? I then want to interpret the output with a JSON parser, so I have some JSON Datapoints configured to pull in the desired 'expirationDate' and 'daysToExpiration'. For example: Interpreter: JSON JSON Path: $.daysToExpiration This appears to only work for one of the properties but not both. I have read other articles and documentation that mentions key pairs, but ideally I would like to stick with JSON.
Tesla Motors LogicModule Suite
I previously published a datasource for Tesla Motors Battery Statistics - which presents compelling vehicle battery and charging information that is fetched from the Tesla REST API. To complement those efforts, I've written a few other Tesla Motors LogicModules that return a variety of different, but still interesting, datapoints - including a ConfigSource that displays configuration information about the vehicle itself (are the doors locked? Is the sunroof open?) The following is a list of all the Tesla Motors LogicModules now available (see the above-linked post for additional info on how this all works.) DataSource 'Battery Statistics' tracks battery and charger performance and health metrics Tesla Motors Battery Statistics previously posted to the Exchange but included here for sake of keeping everything together.) The datasource name is TeslaMotors_BatteryStatistics and has lmLocator DXLLKY. DataSource 'Climate Statistics' tracks inside and outside temperatures, as well as driver and passenger temperature settings. The datasource name is TeslaMotors_ClimateStatistics and has lmLocator YZRWXC. ConfigSource 'Car Configuration' collects textual configuration data, cleans it up and makes it easily readable (screenshot attached.) The configsource name is TeslaMotors_Configuration and has lmLocator GRY9AE. DataSource 'Location Data' tracks compass heading, latitude and longitude, and power. The datasource name is TeslaMotors_LocationData and has lmLocator AYWYWA. DataSource 'Odometer Reading' does exactly what you might expect. The datasource name is TeslaMotors_BatteryStatistics and has lmLocator HHJRDparam not found error in datapoint mapping
Hello, I am new to LogicMonitor and need some help. I was working on creating a datasource where the groovy script returns line based output like: QueueDepth.Queue_Name_1=24 QueueDepth.Queue_Name_2=20 The datapoints are created like: Queue_Name_1 → Key- ##WILDVALUE##.Queue_Name_1 Queue_Name_2 → Key- ##WILDVALUE##.Queue_Name_2 However, when the data gets polled, it is unable to retrieve it: NaN param not found in output - (method=namevalue, param=.Queue_Name_1) Can someone please let me know what I am missing here. Regards, ShivamBGP Description
The Datasource BGP- shows per BGP instance a Description based on snmp OID 1.3.6.1.2.1.15.3.1.1 which is the bgp peer identifier. That is not the actual description that's configured in the switch. Here is an example of BGP configurations. address-family ipv4 vrf t01-WAN bgp router-id 11.226.11.2 network 10.251.0.0 mask 255.255.255.0 network 10.251.1.0 mask 255.255.255.0 network 10.251.2.0 mask 255.255.255.0 neighbor 10.228.3.53 remote-as 65500 neighbor 10.228.3.53 description t01 wan koppeling naar rack z1.3.23 neighbor 10.228.3.53 password *********** neighbor 10.228.3.53 activate neighbor 10.228.3.53 soft-reconfiguration inbound neighbor 10.228.3.53 prefix-list t01-WAN-IP-BLOKKEN out neighbor 10.228.3.53 maximum-prefix 200 50 exit-address-family address-family ipv4 vrf a07-GTT-sip bgp router-id 11.226.11.2 network 10.100.128.16 mask 255.255.255.240 neighbor 10.100.128.1 remote-as 65135 neighbor 10.100.128.1 description a07 gtt euro fiber neighbor 10.100.128.1 password ********* neighbor 10.100.128.1 activate neighbor 10.100.128.1 send-community neighbor 10.100.128.1 prefix-list a07-TO-GTT out exit-address-family A snmpwalk on SNMP OID 1.3.6.1.2.1.15.3 doesn't show the neighbor description. How can we get the bgp neighbor description into the LogicMonitor BGP instance description ?
