add key/value store (redis or similar)

Question

It is becoming very clear we cannot rely on parameters in LM to drive scripts, either because some tokens are mysteriously unavailable for use as parameters (discovered only when assuming they should be), or because tokens have limitations on length that preclude using them for data passed to&nbsp;logicmodules.&nbsp; Please consider integrating a distributed key/value store like redis into LM, with data replicated among collectors.&nbsp; This would help with access to configuration data as well as cross-run results within or across datasources.&nbsp; Ideally this would work natively with Groovy and PowerShell.

mosh · Answer

Yes, a&nbsp;central key/value repository would be useful, with universal access to keys via scripting in particular.

patrickatl · Answer

Mnagel - Love this idea, but curious where you had issues with tokens being unavailable for use as parameters.&nbsp; Looking to determine if I'm also "assuming they should be" and need to be corrected.&nbsp;

mnagel · Answer

One obvious missing token I have raised to support several times is WILDVALUE -- it is not possible to reference this in alerts, which means you cannot say in an alert which input value triggered the alert.&nbsp; And of course, not a bug, but a feature request, which I hear frequently.&nbsp; You also cannot pass but a few specific tokens&nbsp;into PowerShell scripts, and the limitations are not well documented.

This specific issue is related more to recent security monitoring we have been asked to implement.&nbsp; It is not necessarily the correct tool, but then the whole point of using LM is to avoid a plethora of tools.&nbsp; When trying to encode the expected security settings for a Windows folder into a field, we found there are size limits so had to just use fields to index hardcoded values in the script.&nbsp; A key/value store would help.&nbsp; It would also help deal with monitoring for changes in values, like group membership, etc.&nbsp; I can roll out my own and figure out how to get that sync'ed on all the collectors, or it could be a service provided by the collectors themselves (preferable).

mnagel · Answer

Any movement on this? I have another use case where caching within LM would be very helpful -- API result caching.&nbsp; Cisco has an API to lookup vulnerabilities based on platform and version.&nbsp; They also have daily call limits.&nbsp; There is no reason to call on the same platform/version more than once per day, but the "LM way" would be to setup something akin to the MS Patch DS @Mike Suding wrote, so this needs to be bound to AD per host.&nbsp; I can hack around the issue by writing files, but it already bugs me that module scripts can even write files to begin with.&nbsp; Actually doing it makes me remember how dangerous that is&nbsp;and it makes me sad.&nbsp; Having an integrated cache system like this would also allow all collectors in the cache group to leverage the data rather than each caching separately (and increasing the API use rate).&nbsp; Other relevant places API result caching would be useful is for weather lookups for a location -- this only needs to be done once per location every so often (30 minutes perhaps).&nbsp; Not every device in a location needs to actually look up the weather for its location, but you want it to look like that so each device has the status bound.&nbsp; And those services have limits (and costs for increasing limits).&nbsp; Maybe redis is not the specific solution, but something to achieve this would be very helpful.

mnagel · Answer

Another use case -- currently eventsources are of limited use due to lack of correlation/counting.&nbsp; For script-base eventsources, you could at least use a k/v store to detect the same event and extend its lifetime and update a counter in the k/v store.&nbsp; Not perfect, but it is impossible now so this would be an improvement.

Forum Discussion

add key/value store (redis or similar)

8 Replies

Related Content

BigNumber Widget or something similar that will use the Dashboard Timescale?

How to sort key/value pairs in Terraform provider

Redis- DataSource - Occasionally 'No Data'

Automate adding of multiple datapoints from multiline key-value pair output for Script datasource

function or similar for alert "applies to"

Recent Discussions

Add Predefined Resource Types

Rubrik Monitoring package - REST API being retired

Ability to display utilization % on Interface Bandwidth Report

Need to be able to select NOTHING for Alerting on website checks

Enhanced Azure Subscription Monitoring API