Forum Discussion

mnagel's avatar
mnagel
Icon for Professor rankProfessor
5 years ago

better windows event sources

We discarded the default modules for Windows events long ago after realizing their filtering was unusable (events are identified by event source AND event ID, not just event ID as assumed by the default modules). Our modules use a regex matching both event source and ID to fix, and we reference multiple properties so there can be filters defined generally and for specific cases.  This allows higher level values to be overridden if needed, or to extend those with lower level values, as needed. I recently updated these to add 2 more filter properties so we can extend or override with better granularity (labeled universal, org, global and local).

  • Exchange: R7JXYE
  • System: FAAYZ7
  • Application: 94ML93

There is more detail in the technical notes (as much as I could fit before hitting undocumented and obscure field length restrictions). These were just marked for public sharing, so will need security review as they are using Groovy.

One more point -- we do have some global hardcoded filters in at least one of the modules. If that is a problem for anyone, we could add a new property to enable those, leaving them disabled by default.

  • Mike_Rodrigues's avatar
    Mike_Rodrigues
    Icon for Product Manager rankProduct Manager
    5 years ago

    Awesome stuff @mnagel, they've all passed Security Review.

    Thanks for sharing.

    The monitoring team gives me gruff about the tech notes limit too. We have an item to bump the field length, but there is a decent amount of stuff ahead of it.

    The monitoring team actually wants to have something like Github README pages in the LMX, maybe at the package level. They write all the docs in Markdown before passing them to the Documentation team.

  • mnagel's avatar
    mnagel
    Icon for Professor rankProfessor
    5 years ago

    @Michael Rodrigues Cool, thanks!  Now I just need to figure out what to do about my new discovery on how all Windows Event types operate as table scans. Weirdly, we have never had an obvious impact from this until today after all this time (it may explain some issues we have had before, though) 

    Basically, someone had a server spewing ~120MB of events in the selection window and now I know more than I did before today about how this data is collected. It would be nice to be able to narrow the query up front than repeatedly pull all logs and then filter them!  Ticket 212001 if you want to poke your head in :).

  • mnagel's avatar
    mnagel
    Icon for Professor rankProfessor
    5 years ago
    1 hour ago, mnagel said:

    @Michael Rodrigues Cool, thanks!  Now I just need to figure out what to do about my new discovery on how all Windows Event types operate as table scans. Weirdly, we have never had an obvious impact from this until today after all this time (it may explain some issues we have had before, though) 

    Basically, someone had a server spewing ~120MB of events in the selection window and now I know more than I did before today about how this data is collected. It would be nice to be able to narrow the query up front than repeatedly pull all logs and then filter them!  Ticket 212001 if you want to poke your head in :).

    Do you happen to know if the event log data is fetched once per ES or is it done once and then filtered via each?  The latter would be best, but this is not discussed anywhere I can find.  Even so, if one log is growing fast and we want to skip it, the default method has no way to do that.  We are considering a Groovy or PowerShell replacement to restrict data pulled prior to filtering.