Forum Discussion

nageeb's avatar
2 years ago

New user - looking for information on parsing fields from Syslog message field

We’re just starting to get LogicMonitor setup on our network and, mainly to test the collector, I setup one of our HAProxy instances to forward it’s syslog to the collector and can confirm that the l...
  • JJumpp's avatar
    2 years ago

    A LogSource can break out all of these fields using the Regex Dynamic Group method.  You can create multiple fields in a single line/expression.  
    https://www.logicmonitor.com/support/syslog-logsource-configuration#h-configuration-options

    Before anything, make sure your collector is the right version to use LogSources and chose the “Enable LM Logs” option for the resource if the logs are being sent to a different collector than the one collecting regular monitoring metrics.

    Then build your Regex.
    It would start something like 
    ([a-zA-Z0-9_ ]*\d+:\d+:\d+) (\S+) (\S+) .+:?.+
    and keep breaking down each section you need to parse into different groups.  Some characters may need to be escaped to be accepted by the LogSource, but once you have it in, it works pretty well.  
    May take a few minutes to kick in, but you should start seeing them soon.  Line up the keys that correspond to each group in your expression in a comma separated list, no spaces (log_time,server,socket, etc)

    May sure none of your fields are named something that could conflict with the standard format, like “timestamp” or “source” since that may break your ingestion.  
    For troubleshooting, temporarily turn on your collector logging for syslog to debug (https://www.logicmonitor.com/support/collectors/collector-management/collector-logging) then go into your debug facility and try
    !tail ..\logs\wrapper.log 1000 
    to see any errors that may be associated with your logs.  

    Make sure to turn off that debug level when you are done.  

    Hope this helps!