Keimond
Joined 7 years ago
User Profile
User Widgets
Contributions
Re: Has anybody noticed the flaw in LogSource logic?
You say, “identify logs by the device itself”. Great. How? @Cameron Compton I’m alsowondering how this happens.. LM Support tells me that it’s through the resource mapping in the lmlog source . . . but again before this lmlog source is even used, LM has to know what system the log came from for the AppliesTo section to work correctly…are we using the AppliesTo wrong ? Should be applying the lmlog source to the collector in which the logs are coming in at ? I have this set up, which is identical to what’s in a working collector configuration lmlogs.syslog.property.name=system.sysname lmlogs.syslog.hostname.format=HOSTNAME …but the lmlog source will not apply to the device so I’m at a loss.3Views0likes0CommentsRe: LM Logs multiple capture group parsing
At least you can get lmlogs to work lol…we (myself and now threesupport guys) have yet to get it working. I’ve taken a step back and just made on lmlog source that applies to one resource and told to map HOSTNAME to system.sysname which is identical to what’s in the collector configuration…lmlog source is supposed to take precedence over the collector configuration but so far it doesn’t look that way. #1 I don’t suppose you’d be willing to share all of the lines you have in your collector configuration with the word “lmlog” or “syslog” #2 What does your resource mapping look like in your lmlog source ? #3 I am very interested in the multiple capture group but their documentation is lacking, would you be able to post an example of that ? :) Thank you !!!8Views0likes0CommentsLinuxNewProcesses DataSource -- Auto discovery and key off of HOST-RESOURCES-MIB::hrSWRunName
Hello all! I just wanted to share my edits. I never could get LinuxNewProcesses to work for my needs.. but we really wanted it to also have auto discovery and automatically add a list of toolsets that we have deployed across the board. I did this LONG ago and my wildvalue was the PID…but that’s dangerous and I ended up creating thousands of entries in the LM database because my processes (thousands of them) were always changing. . . .this takes a different approach and keys off of the process name. #1 You just need to have a property defined with a comma separated list These names need to be from “HOST-RESOURCES-MIB::hrSWRunName” #2 My polling is every minute but don’t alert unless it’s been down for an hour…for my scenario, I do this on purpose because some of my applications run for about 5 minutes and then aren’t kicked off again for another 10…so adjust as needed :) The status is under a security review right now.. I’ll post the lmLocator if it makes it! Otherwise here’s the autodiscovery.. the collection script wont’ work and you’ll have to modify it import com.santaba.agent.groovyapi.snmp.Snmp; def OID_NAME = ".1.3.6.1.2.1.25.4.2.1.2"; def host = hostProps.get("system.hostname"); def services = hostProps.get("linux.services").split(','); Map<String, String> result = Snmp.walkAsMap(host, OID_NAME, null) result.forEach({ index,value->index = index; value = value; for (service in services) { if (value ==~ /${service}/) { def CMD_OID = ".1.3.6.1.2.1.25.4.2.1.4." + index; def service_cmd = Snmp.get(host, CMD_OID); def desc = index + " | " + service_cmd; out.println value + "##" + value + "##" + desc } } }) Script: Line 89: if ("${name}" == "${processPath}") {91Views19likes3CommentsGlobal Delay setting and maybe flapping trigger ?
Hello, We happen to run across another problem last week that might be a nice feature and the delay is probably pretty easy to implement. Sometimes a threshold might be triggered but then clear within X minutes. While these might be useful to know... I don't want my ticket system to get an email for these... so we tried to set an escalation delay of one hour (using a null escalation path for the 1st step as suggest in documentation)... but the problem is unless someone goes in and ack's the alert within the our of course the system is going to email again. Unless we write a custom API, we have no way to gracefully get these tickets to the ticket system without duplicates being created. So it would be SUPER nice to have a delay per priority so that we could say do not sent send any alerts unless the alert has lasted for longer than X minutes. I know I can do that in the datasource per datapoint but that's a lot of changes to go through.. which is why I would love a "global" setting :)/emoticons/smile@2x.png 2x" title=":)" width="20" /> Also noting for anyone using OpsGenie (us) or PagerDuty.. I'm pretty sure the delay could be set there... so really the global delay is just needed if you are using email/SMS. After writing this I realized wait a minute that brings up the idea of a flapping alert... for example if I were to set my GLOBAL delay for 2 hours on errors... ok cool I now am only going to get alerted if the error lasts that long... but maybe I still want to be alerted if it has been bouncing for X number of times per X minutes... that way my global delay isn't going to ignore "flapping" that we probably should know about :)/emoticons/smile@2x.png 2x" title=":)" width="20" />3Views1like0CommentsRe: xml and xpaths
Thanks again, I just wanted to show my final output as I added the boot-time and config-time that I wanted as well. /etc/snmp/scripts/runstats.sh boottime:79395 conftime:79395 ina:3098500 ina6:8 inaaaa:1107344 inany:69885 incname:43835 inmx:147801 innaptr:23 inns:1 inptr:2260082 insoa:65773 inspf:2244 insrv:97583 intxt:146317 rsnx:16075 outa:1605153 outa6: outaaaa:385541 outany: outcname:18206 outmx:212 outnaptr: outns:75 outptr:123973 outsoa:209 outspf: outsrv:1227 outtxt:288 rsnx:16075 rsfail:477 rserr:455 rsipv4qs:1875597 rsipv4rr:1866673 rsmismatch:1 rsqr:349108 rsqt:9010 rsrtt10:707770 rsrtt100500:76413 rsrtt10100:1082061 rsrtt1600:6 rsrtt500800:300 rsrtt8001600:15 sockopen:1876127 sockclosed:1876115 sockbf:169 consest:1863133 recverr:108 My modified /etc/snmp/script/runstats.sh is #!/bin/sh rm -rf /var/named/named.stats rndc stats stats=$(cat /var/named/named.stats | /etc/snmp/scripts/dnsstats.pl 2>/dev/null) now=$(date +%s) host=localhost port=8653 xmlstats=$(curl -s http://${host}:${port} 2>/dev/null | xml2) boottime=$(date -d $(echo "${xmlstats}" | egrep -i 'boot-time' | cut -d= -f2) +%s) conftime=$(date -d $(echo "${xmlstats}" | egrep -i 'config-time' | cut -d= -f2) +%s) boottimeseconds=$(echo "${now} - ${boottime}" | bc) conftimeseconds=$(echo "${now} - ${conftime}" | bc) echo "boottime:${boottimeseconds} conftime:${conftimeseconds} ${stats}" Note to anyone out there that also might see this, adding boot-time and config-timedoes require this in your named.conf statistics-channels { inet 127.0.0.1 port 8653 allow { 127.0.0.1; }; };2Views0likes0CommentsRe: xml and xpaths
And of course after I post this I decide to try one more thing and it works.. /statistics/server/counters[@type="qtype"]//counter[1]/text() I understand why this is working but what if I don't always know if for example the "name=A" is going to be the first element ? Which led me to try /statistics/server/counters[@type="qtype"]//counter[@name="A"]/text() Which does seem to work, so back to building all of the datapoints !2Views0likes0Commentsxml and xpaths
Hey all I'm having troubles figuring this out.. so I have set up a datasource that pulls the xml directly from a bind statistics web server and that's working but I can't get the values to show up in the datapoints. given this small snippet... <statistics> <server> <boot-time>2021-05-20T18:36:19.674Z</boot-time> <config-time>2021-05-20T18:36:19.842Z</config-time> <current-time>2021-05-20T21:43:29.263Z</current-time> <counters type="qtype"> <counter name="A">448606</counter> <counter name="PTR">395691</counter> </counters> </server> </statistics> When I try "/statistics/server/counters/counter/@name='A'", I just get a value of True back. When I try "/statistics/server/counters/counter/@name='A'/text()", I get NAN. Any ideas on what I can try ? I've been looking at xpath examples all day and haven't been able to figure out what's going on... Additionally if anyone is interested I'll share the datasource after I'm done making it.. I was able to make a complex datasource for the boot/config times with groovy doing something like this: stats = new XmlSlurper().parseText(body); rawDate = stats.'**'.find { it.name() == 'boot-time' }.text(); Date fd = Date.parse( "yyyy-MM-dd'T'HH:mm:ss.SSS", rawDate ); today = new Date(); timeDiff = (today.time - fd.time)/1000; return (timeDiff); Then of course if we want we can set up a threshold to let us know if a dns server was reloaded (config-time) or rebooted (boot-time)28Views0likes5CommentsRe: Tomcat/JMX monitoring with SSL
So I gave up on SSL as it turns out the behavior is not what I thought it was and we do not need it. I do have it working great with authentication which is all I need. Now I just have to figure out why our test server works and our production server doesn't ! my setenv.sh file sources a file set up by ansible... CATALINA_OPTS="-Dcom.sun.management.jmxremote=true \ -Dcom.sun.management.jmxremote.port={{ jira_lm_jmxport }} \ -Dcom.sun.management.jmxremote.ssl=false \ -Dcom.sun.management.jmxremote.authenticate=true \ -Dcom.sun.management.jmxremote.password.file={{ jira_lm_jmxpath }}/jmxremote.password \ -Dcom.sun.management.jmxremote.access.file={{ jira_lm_jmxpath }}/jmxremote.access \ -Djava.rmi.server.hostname={{ ansible_default_ipv4.address }}" I've run tcpdump on the production server and wireshark back on my windows collector and verified that the two are in fact talking to each other on the jira_lm_jmxport The error indicates that it's timing out. I tried increasing the timeout for jmx on the collector and was still getting a timeout but perhaps I need to try longer... On Friday I'll be restarting the service after adding these two lines to see if that helps after reading some posts -Dcom.sun.management.jmxremote.rmi.port={{ jira_lm_jmxport }} \ -Dcom.sun.management.jmxremote.local.only=false \3Views0likes0Comments