Forum Discussion

Anonymous's avatar
Anonymous
2 years ago

LM Logs multiple capture group parsing

Ok, this is cool. I have some log data that has structured data in it (some text, then a python list of strings). I had started building out a parse statement for each member of the list, then thought I’d try just making multiple capture groups and naming multiple variables after the as fanboy. Turns out it completely works. It parses each capture group into the corresponding column with a single parse statement. I was halfway through writing out a feature request when I figured I’d give it a try only to discover that it completely works. Nice job LM Logs guys.

LM Logs
parse
regex
  • Anonymous's avatar
    Anonymous
    9 months ago

    I have two logsources for syslog and it’s not very easy to understand why, but it has something to do with the source IP address and resolvability or something.

    All Syslog (IP):

    • Method = IP
    • Key = system.ips

    All Syslog (Hostname)

    • Method = IP
    • Key = system.hostname

    I’d have to a pcap on two different collectors to see the actual difference between them incoming since LM doesn’t surface any details about what actually came in (would be nice to see the raw data by turning up a debug level somewhere). 

    All i know is if i change the applies to for either of these logsources so that they include the devices that should belong to the other logsource, the mapping suddenly, completely fails. They don’t get mapped to any devices at all.

    Theoretically, these two logsources could be combined into one if you’re using collector v35.100+. This is because there’s a new “OR” option in the resource mapping section that should allow both of these mapping to exist side by side and allow one to work when the other fails. I haven’t been able to test yet because we’re not on that collector version across the board yet (it’s not GA yet, just early access/beta).

  • Keimond's avatar
    Keimond
    Icon for Neophyte rankNeophyte
    9 months ago

    At least you can get lmlogs to work lol… we (myself and now three support guys) have yet to get it working.

    I’ve taken a step back and just made on lmlog source that applies to one resource and told to map HOSTNAME to system.sysname which is identical to what’s in the collector configuration… lmlog source is supposed to take precedence over the collector configuration but so far it doesn’t look that way.

    #1
    I don’t suppose you’d be willing to share all of the lines you have in your collector configuration with the word “lmlog” or “syslog”

    #2
    What does your resource mapping look like in your lmlog source ?

    #3 I am very interested in the multiple capture group but their documentation is lacking, would you be able to post an example of that ? :)

    Thank you !!!