Silent Log Source Detection
We are working on migrating some items to LM Logs and one thing that came up was the lack of Silent Log Source detection. Log Usage is a PushModule and only updates when data is pushed to it. So unlike a traditional DataSource, you cannot alert on No Data or 0 events for X period of time. With that in mind, we wrote our own, which I am sharing here instead of in the LM Exchange as it will require a bit of changing on your own. So, this could be 100% portable, but I purposefully didn't do that to save on API calls. That this DataSource will do, is make two api calls per device, back to the LogicMonitor portal. The first is to retrieve the instances of LogUsage, which is defined in line 22. That is where you would need to change the ID to match what is in your portal. Then after it retrieves the instance of LogUsage, it queries data for said instance. That returns a JSON structure like this. { "dataPoints": [ "size_in_bytes", "event_received" ], "dataSourceName": "LogUsage", "nextPageParams": "start=1739704983&end=1741035719", "time": [ 1741110300000 ], "values": [ [ 25942, 16 ] } There will generally be more items in time and in values. We grab the first element, so [0] of the time array as that is the latest timestamp. Then we calculate the difference between now and that timestamp in hours. Then we return the difference in hours, or a NaN. /******************************************************************************* * © 2007-2024 - LogicMonitor, Inc. All rights reserved. ******************************************************************************/ import groovy.json.JsonSlurper import com.santaba.agent.util.Settings import com.santaba.agent.live.LiveHostSet import org.apache.commons.codec.binary.Hex import javax.crypto.Mac import javax.crypto.spec.SecretKeySpec String apiId = hostProps.get("lmaccess.id") ?: hostProps.get("logicmonitor.access.id") String apiKey = hostProps.get("lmaccess.key") ?: hostProps.get("logicmonitor.access.key") def portalName = hostProps.get("lmaccount") ?: Settings.getSetting(Settings.AGENT_COMPANY) String deviceid = hostProps.get("system.deviceId") Map proxyInfo = getProxyInfo() def fields = 'id,dataSourceId,deviceDataSourceId,name,lastCollectedTime,lastUpdatedTime,deviceDataSourceId' def apipath = "/device/devices/" + deviceid + "/instances" def apifilter = 'dataSourceId:43806042' def deviceinstances = apiGetMany(portalName, apiId, apiKey, apipath, proxyInfo, ['size':1000, 'fields': fields, 'filter': apifilter]) instanceid = deviceinstances[0]['id'] devicedatasourceid = deviceinstances[0]['deviceDataSourceId'] def instancepath = "/device/devices/" + deviceid + "/devicedatasources/" + devicedatasourceid + "/instances/" + instanceid + '/data' def instancedata = apiGet(portalName, apiId, apiKey, instancepath, proxyInfo, ['period':720]) def now = System.currentTimeMillis() if (instancedata['time'][0] != null && instancedata['time'][0] > 0) { def diffHours = ((instancedata['time'][0] - now) / (1000 * 60 * 60)).toDouble().round(2) println "hours_since_log=${diffHours}" } else { def diffHours = "NaN" println "hours_since_log=${diffHours}" } // If script gets to this point, collector should consider this device alive keepAlive(hostProps) return 0 /* Paginated GET method. Returns a list of objects. */ List apiGetMany(portalName, apiId, apiKey, endPoint, proxyInfo, Map args=[:]) { def pageSize = args.get('size', 1000) // Default the page size to 1000 if not specified. List items = [] args['size'] = pageSize def pageCount = 0 while (true) { pageCount += 1 // Updated the args args['size'] = pageSize args['offset'] = items.size() def response = apiGet(portalName, apiId, apiKey, endPoint, proxyInfo, args) if (response.get("errmsg", "OK") != "OK") { throw new Exception("Santaba returned errormsg: ${response?.errmsg}") } items.addAll(response.items) // If we recieved less than we asked for it means we are done if (response.items.size() < pageSize) break } return items } /* Simple GET, returns a parsed json payload. No processing. */ def apiGet(portalName, apiId, apiKey, endPoint, proxyInfo, Map args=[:]) { def request = rawGet(portalName, apiId, apiKey, endPoint, proxyInfo, args) if (request.getResponseCode() == 200) { def payload = new JsonSlurper().parseText(request.content.text) return payload } else { throw new Exception("Server return HTTP code ${request.getResponseCode()}") } } /* Raw GET method. */ def rawGet(portalName, apiId, apiKey, endPoint, proxyInfo, Map args=[:]) { def auth = generateAuth(apiId, apiKey, endPoint) def headers = ["Authorization": auth, "Content-Type": "application/json", "X-Version":"3", "External-User":"true"] def url = "https://${portalName}.logicmonitor.com/santaba/rest${endPoint}" if (args) { def encodedArgs = [] args.each{ k,v -> encodedArgs << "${k}=${java.net.URLEncoder.encode(v.toString(), "UTF-8")}" } url += "?${encodedArgs.join('&')}" } def request if (proxyInfo.enabled) { request = url.toURL().openConnection(proxyInfo.proxy) } else { request = url.toURL().openConnection() } request.setRequestMethod("GET") request.setDoOutput(true) headers.each{ k,v -> request.addRequestProperty(k, v) } return request } /* Generate auth for API calls. */ static String generateAuth(id, key, path) { Long epoch_time = System.currentTimeMillis() Mac hmac = Mac.getInstance("HmacSHA256") hmac.init(new SecretKeySpec(key.getBytes(), "HmacSHA256")) def signature = Hex.encodeHexString(hmac.doFinal("GET${epoch_time}${path}".getBytes())).bytes.encodeBase64() return "LMv1 ${id}:${signature}:${epoch_time}" } /* Helper method to remind the collector this device is not dead */ def keepAlive(hostProps) { // Update the liveHost set so tell the collector we are happy. hostId = hostProps.get("system.deviceId").toInteger() def liveHostSet = LiveHostSet.getInstance() liveHostSet.flag(hostId) } /** * Get collector proxy settings * @return Map with proxy settings, empty map if proxy not set. */ Map getProxyInfo() { // Each property must be evaluated for null to determine whether to use collected value or fallback value // Elvis operator does not play nice with booleans // default to true in absence of property to use collectorProxy as determinant Boolean deviceProxy = hostProps.get("proxy.enable")?.toBoolean() deviceProxy = (deviceProxy != null) ? deviceProxy : true // if settings are not present, value should be false Boolean collectorProxy = Settings.getSetting("proxy.enable")?.toBoolean() collectorProxy = (collectorProxy != null) ? collectorProxy : false Map proxyInfo = [:] if (deviceProxy && collectorProxy) { proxyInfo = [ enabled : true, host : hostProps.get("proxy.host") ?: Settings.getSetting("proxy.host"), port : hostProps.get("proxy.port") ?: Settings.getSetting("proxy.port") ?: 3128, user : Settings.getSetting("proxy.user"), pass : Settings.getSetting("proxy.pass") ] proxyInfo["proxy"] = new Proxy(Proxy.Type.HTTP, new InetSocketAddress(proxyInfo.host, proxyInfo.port.toInteger())) } return proxyInfo }Apr 24 - Product Power Hour: Logs Leveled Up!
Product Power Hour: Logs Leveled Up Date: Thursday, April 24 at 10 AM CT Join us for this month’s Product Power Hour, a monthly series brought to you by the LM Community, Product team, and Training & Enablement! This session dives into the latest innovations in LogicMonitor Logs, with a spotlight on AI-powered capabilities that enhance log analysis and accelerate root cause detection. Featuring Guest Speakers: Patrick Sites, Product Management Richard Brooke, Product Trainer Suzanne Shaw, Sr. Manager, Community & Advocacy Sky Donnell, Community Product Specialist What You’ll Learn: AI-Powered Log Intelligence: Discover how our new AI-driven features surface insights faster, reduce noise, and help your team make smarter decisions in real time. Improved Capabilities: Get a tour of the latest updates in LogicMonitor Logs, from intuitive log correlation to enhanced search and filtering. Integration Tips: Learn how logs fit seamlessly into the LM platform and how to set up your environment for success. Best Practices: We’ll debunk common myths and showcase some of the most underutilized (but powerful) features in action. Live Q&A: Bring your questions—our product experts are here to answer them live and help you unlock the full value of Logs. 🔗 Register Here - Recap and recording will be shared to all who register Note: Times may rotate monthly for global accessibility. All sessions are recorded and shared with registered participants.March 2025 Logs for Lunch Recap: Catching Anomalies in the Cloud & Deeper Insights with Logs
Overview This month’s Logs for Lunch session brought together IT professionals to explore Catching Anomalies in the Cloud & Deeper Insights with Logs, highlighting how LM Logs can streamline troubleshooting and proactive monitoring. Our experts explored real-world use cases, demonstrating how logs provide deeper visibility into diagnosing an anomaly in your cloud storage permissions before it becomes a major problem! Register to watch the replay! Summary: Overview This webinar, hosted by Tom Chavez, introduces Logs for Lunch, a new webinar series for 2025. The session covers the importance of logs in troubleshooting and monitoring cloud and infrastructure issues, featuring two live demos by Nick Doane (Sr. Sales Engineer) and David Femino (Product Team). The discussion highlights LogicMonitor’s (LM) log analysis tools, their AI-powered anomaly detection, and their ability to streamline IT operations. Key Takeaways 1. Challenges in Log Management Logs are typically owned by security teams, making it difficult for operations teams to access them. Organizations often deal with log overload, multiple tools, and lack of a unified view. Querying logs across different tools requires expertise in various query languages. LM Logs simplifies log access and analysis by integrating with LM Envision. 2. How LM Logs Helps Unifies log data across cloud, on-prem, and network devices. Reduces troubleshooting time by up to 80% through centralized log analysis. AI-powered anomaly detection filters out noise and focuses on unusual patterns. Streamlined access for Ops teams—no need to rely on security teams for logs. Hot storage for all logs, ensuring quick and easy searches. Flexible retention options (7 days to a year) based on business needs. Demo 1: Troubleshooting Cloud Issues with LM Logs (Nick Doane) Nick demonstrates how LM Logs’ anomaly detection can quickly pinpoint and resolve issues: Scenario: A Frontend Application Failure A 404 error surge is detected in an Azure-based app. LM Logs automatically correlates error alerts with related log anomalies. AI filters 20M logs down to 71 anomalies, surfacing the most critical entries. A user, John Johnson, is identified as having made a change in Azure storage. The public network access setting was disabled, causing 404 errors. Resolution: Quick identification of who changed what using log analysis and anomaly detection. No need for query language expertise—LM Logs automates log filtering. Faster root cause identification without escalating to higher-level engineers. Demo 2: Advanced Log Querying & Aggregation (David Femino) David explores how to extract insights from logs using LM Logs’ built-in query features: Log Querying Simplified Type-ahead key-value filtering (e.g., resource:name). Automatic log metadata extraction (log levels, syslog facilities, etc.). Visual filtering—click-to-query without learning complex query languages. Advanced Log Analytics with Operators Count logs by resource to identify "top talkers." Aggregate log sizes, track storage usage over time. Time bucketing: Group log data into 24-hour windows for trends. Parsing log fields dynamically (e.g., extracting Access Keys from AWS logs). AI-powered anomaly detection reduces the need for manual data sifting. Final Notes & Upcoming Events Live Q&A: Attendees ask about log storage, AI filtering, and query best practices. Elevate User Conference: Dallas (April), Sydney (May), London (June) Includes demos, training, and networking. Next Webinar: Patrick Sites (Product Architect) will showcase new LM Logs features. Takeaway: LM Logs is built to simplify log analysis, provide quick insights, and enhance operational efficiency without requiring deep technical expertise in querying logs. Questions: Q: Will any portion of elevate be available online? A: Yes, all Elevate sessions will be recorded for playback for free. Q: Where can I find more info about advanced search operators? A: Advanced search operators are covered here: https://www.logicmonitor.com/support/advanced-search-operators Q: Any other resources? A: I also love this resource, the Logs Search Cheatsheet: https://www.logicmonitor.com/support/lm-logs/logs-query-language/logs-search-cheatsheet Example queries can be imported from our git repo as well. ehttps://github.com/logicmonitor/log-queries What’s Next? Next Logs for Lunch: Accelerating Troubleshooting with Logs April 9, 2025 12noon CT / 10am PT / 11am MT / 1pm ET Register and get more information here Elevate Community Conference: Join us in Dallas, TX (April 30), Sydney, AUS (May 29), and London, UK (June 25), to gain strategic insights, hands-on product experience, and exclusive networking opportunities. Elevate 2025 will showcase the latest innovations in AI-powered observability, empowering enterprises to optimize their modern data centers. Find more details and registration links here! What Came Before? Did you miss the Logs for Lunch presentations and demos from January and February? Catch up on the great topics covered earlier this year, including our opening meeting of 2025 in January with a LM Logs overview and demo of Troubleshooting a Tomcat app, and the second meeting in February focused on Network Observability and Wireless Connectivity.February 2025 Logs for Lunch Recap: Network Observability & Wireless Connectivity
Overview This month’s Logs for Lunch session brought together IT professionals to explore Network Observability & Wireless Connectivity, highlighting how LM Logs can streamline troubleshooting and proactive monitoring. Our experts explored real-world use cases, demonstrating how logs provide deeper visibility into network performance, security events, and infrastructure health. Whether managing a growing wireless network or optimizing log intelligence, this session was packed with actionable insights to elevate your monitoring strategy. The Demo Making Wireless Networks More Predictable: We explored how log intelligence can help identify and resolve connectivity issues before they impact users. Proactive Troubleshooting with LM Logs: Discover how to correlate logs with performance metrics for faster incident resolution and enhanced root cause analysis. Security & Compliance Insights: Learn how to leverage log data for better security monitoring, detecting anomalies in network behavior. Enhancing Network Observability: Unveiling best practices for visualizing wireless connectivity issues with logs and metrics in a single pane of glass. Customer Success Stories: Real-world applications showcasing how teams are using LM Logs to optimize network health and troubleshoot at scale. Q&A Q: How can LM Logs help with wireless troubleshooting? A: LM Logs provide real-time insights into network performance, helping to correlate log data with connectivity metrics, device health, and historical trends. Q: Can LM Logs be used for security monitoring? A: Absolutely! Logs can highlight unexpected login attempts, firewall policy violations, and network anomalies, making them a key tool for security and compliance teams. Q: How do I integrate LM Logs with my current monitoring setup? A: LM Logs work seamlessly with existing dashboards and alerting workflows, allowing you to combine performance metrics, topology maps, and log data in one place. Q: What’s the best way to filter and analyze large volumes of logs? A: Utilize log search, filters, and anomaly detection features to pinpoint the most relevant data, reducing noise and making troubleshooting more efficient. Customer Call-outs “The ability to see connectivity issues correlated with logs in real-time is a game-changer.” “Security monitoring with logs is something we’ve needed, and this session really showed us how to implement it.” “We’ve been struggling with intermittent wireless issues, and now we have a solid strategy to tackle them.” What’s Next? Virtual User Groups: Join us for our first LM Community Virtual User Group series, where you'll hear from fellow LogicMonitor customers about their hybrid observability journey. Register for your preferred region below! LM User Group | AMER East - Mar 20 LM User Group | AMER West - Mar 20 LM User Group | APAC - Mar 27 LM User Group | EMEA - Mar 27 Elevate Community Conference: Join us in Dallas, TX, Sydney, AUS, and London, UK, to gain strategic insights, hands-on product experience, and exclusive networking opportunities. Elevate 2025 will showcase the latest innovations in AI-powered observability, empowering enterprises to optimize their modern data centers. Find more details and registration links here! Stay tuned for more insights and opportunities to enhance your monitoring capabilities with LM Logs. Missed this session? Watch the full recording below ⤵️January 2025 Logs for Lunch Recap: Transforming Log Intelligence
We launched our 2025 Logs for Lunch series with a bang, diving deep into how LM Logs is transforming how teams tackle troubleshooting. If you missed it, don't worry - here's a quick recap. Making Log Troubleshooting Less Painful Most of us don't exactly jump for joy when we have to dig through logs. But LogicMonitor is changing that game. The standout feature? An AI-powered system that spots unusual patterns automatically - no complex queries needed. This innovative approach has helped organizations reduce their troubleshooting time by up to 80%, significantly improving operational efficiency. The Demo The technical demonstration showcased real-world applications, featuring: Streamlined alert-to-resolution workflow "Show Patterns" feature for identifying recurring issues Automated alert creation based on log patterns Seamless integration between metrics and logs The demo walked through diagnosing a web server issue, illustrating how complex problems can be resolved with minimal clicks and without extensive logging expertise. Q&A People had questions, and we got answers! Here are the ones that got everyone's attention: Q: I'm new to this - where should I start? A: Start with what you know - if you're already monitoring network devices or Windows servers in LogicMonitor, that's your sweet spot. These are usually the easiest to set up and start getting value from right away. Q: How does pricing work? A: LM Logs is an add-on to LM Envision, and it's pretty straightforward: you pay based on how much data you're logging and how long you want to keep it. Whether you need 7 days or a full year of retention, they've got you covered. Q: How do I keep track of usage? A: There's a neat dashboard that shows your monthly usage, trends, and even which systems are your "top talkers" - super helpful for keeping things under control. What's Next? Mark your calendar for the next Logs for Lunch session on February 12th, 2025, at 12 pm CT, where we're tackling troubleshooting wireless networks. Save your spot by registering today. Keep an eye out in the Community for upcoming exciting product launches! Check out our official LM Logs page here for a deeper dive into logs.
Webhook Event Collection & Cisco Meraki
As a Cisco Meraki Strategic Technology Partner, we are always looking for ways to make our integration the best that it can be, so you can get the most out of your investments with Cisco and LogicMonitor. So, today we kicked off R&D planning for [safe harbor statement] the ability to collect webhook events from Cisco Meraki with the following objectives. Mitigate Cisco Meraki Dashboard API rate limiting. Enable [near] real-time alerts for things like camera motion, IoT sensor measurement threshold breach (or automation button press), power supply failure... Facilite sending webhook events from Cisco Meraki to LogicMonitor I have the following assumptions. Customers want to be alerted on most, but not all webhook events. Customers want to have multiple inbound webhook configurations, i.e. for different tenants/customers or different Cisco Meraki organizations. Cisco Meraki is the first but not the only platform that customers will want to use to send webhook events to LogicMonitor. If you had a magic wand and could make such an integration do exactly what you wanted, what would be your number one ask? Thank you!
How to redirect the output of the groovy script to the collector log file using groovy script?
In my groovy script, I want to redirect the output from the groovy script into the collectors log file? What should be the groovy code, to redirect the output to the collectors log file? Can anyone help me here?
☁️ Monitor Azure Resource Events with LogicMonitor Logs
I have a strong preference for Microsoft Azure due to its exceptional capabilities! I recently wrote a blog post showcasing how to bring your resource events to the LogicMonitor platform. This way, you can set up alerts for critical business operations, such as when a new user is added to your Active Directory (Entra), or when a file is deleted from your blob storage. I hope you find it as helpful as I did! Monitor Azure Resource Events with LogicMonitor Logs Do you use LogicMonitor or any other monitoring platform to address unique use cases? Share your stories with us!
