Updates to modules showing in repo but not modules toolbox

Why would this be? There were 5 updates to aws_billing_* modules today, but they only show up when i use the traditional repo update option. No combination of filters could show those in the module toolbox, except by clearing all filters and just searching by name. Even then, once found by name, they showed already up to date.

In addition, when trying to make a filter to show them, i was frustrated (once again) that the filters are filtering by values we cannot see:

The “Customized” field isn’t displayable on the table at all, or if it is, it’s displayed as an icon in the status column. However, the dropdown for the filter is not an icon, it’s text.
The status column shows an icon, but the filter values are text. Which icon corresponds to which text?
For these 5 modules, the “Support” column was blank, even though these modules come from LM so should be “official”? How can you filter by “Support” == null?

It’s all very Duolingo: make mistakes and wander your way through until you figure it out?

LM User

Posted 3 years ago·Last reply 2 years ago

16 comments

LM User

OP2 years ago

You won’t see modules you don’t have in the Toolbox, so the Exchange is the only place to see new (uninstalled) modules.

TIL. I never really understood why there should be a separate toolbox from the exchange. Would rather have one searchable/filterable list to rule them all. Even your own filtering options for the same kind of thing have different names depending on whether you’re in the exchange or the toolbox. I have not been on the exchange page in over a year. Do you have usage stats on it? Do people actually use it? Why does the exchange even list the installed modules?

You may want to supplement these by looking only at Official modules, but if you have Community modules and those have updates, they’re probably worth looking at too.

Community stuff cannot be trusted to receive updates, so I never use it. The exception is if there’s a github repo that i can follow.

Currently, there is no way to differentiate in use from not in use, or to see which side customizations were made on (you vs us) with in the diff. We do have all this data though. 195 will see the rollout of the “In Use / Not in Use” column for some modules types, and it’s more advanced than the old school AppliesTo check that Import from Repository uses (and which you technically can tediously recreate by testing AppliesTo on each module). There will be a filter for this as well (and you can sort by how in use it is).

Is that going to happen simultaneous to losing the audit module functionality? What will the in use/not in use be based on? Actual instance counts? Could we just get the instance counts and resource counts in columns? Is the ability to run an adhoc “test appliesto” and/or “test discovery” still on the roadmap?

You can see “Customized” status today to know that you customized a given module from our defaults, but again, that doesn’t extend to the field by field diff (though we have that info and hope to show it in the UI).

This will help immensely with the actual heavy lifting of merging in updates. Currently I just look at the diffs in my tool (which diffs all fields not just preservable ones and decodes the code and diffs line by line). Understanding the decisions the ME team makes without any context can sometimes be difficult.

It’s going to be difficult for my team to allow access to thew new APIs (we don’t control API access, but we do build the LMX APIs), but if you’re cool with something like a CSV export, I’m pretty confident we’ll be able to create something you can use to get back to a similar (if not better) flow.

Obviously today, I built my own module toolbox to help me analyze the diffs. Having a CSV/JSON endpoint i can hit in a script will allow me to still update my changelog. Once all this mess is actually working, I’d love to talk about enhancing the change log so i don’t have to maintain my own.

Last week (in the context of the API difficulties), one of the BE engineers actually recommended just providing a simple export for power users who wanted the info but didn’t want to deal with our tooling/ui/flow.

Yes, that would help ease the transition until the kinks are worked out of the module toolbox workflow.

Michael Rodrigues

·2 years ago

@Stuart Weenig

Here’s how those filters map to those conditions (just click Clear and enter the relevant filters for Toolbox or Exchange):

Up to Date
- Toolbox
  - Status: Up to Date
- Exchange
  - Installed & Up-to-Date
New
- Toolbox
  - You won’t see modules you don’t have in the Toolbox, so the Exchange is the only place to see new (uninstalled) modules.
- Exchange
  - Status: Not Installed
Updated
- Toolbox
  - Status: Outdated
- Exchange
  - Status: Installed & Update Available

You may want to supplement these by looking only at Official modules, but if you have Community modules and those have updates, they’re probably worth looking at too.

Currently, there is no way to differentiate in use from not in use, or to see which side customizations were made on (you vs us) with in the diff. We do have all this data though. 195 will see the rollout of the “In Use / Not in Use” column for some modules types, and it’s more advanced than the old school AppliesTo check that Import from Repository uses (and which you technically can tediously recreate by testing AppliesTo on each module). There will be a filter for this as well (and you can sort by how in use it is).

You can see “Customized” status today to know that you customized a given module from our defaults, but again, that doesn’t extend to the field by field diff (though we have that info and hope to show it in the UI).

It’s going to be difficult for my team to allow access to thew new APIs (we don’t control API access, but we do build the LMX APIs), but if you’re cool with something like a CSV export, I’m pretty confident we’ll be able to create something you can use to get back to a similar (if not better) flow.

Last week (in the context of the API difficulties), one of the BE engineers actually recommended just providing a simple export for power users who wanted the info but didn’t want to deal with our tooling/ui/flow.

LM User

OP2 years ago

I think what might help would be to know how to construct filters in the module toolbox that help classify the modules according to the level of attention required. In my case these classifications also map to:

Up to date - no attention required
New - cursory attention required (ability to back out needed)
Updated Not in Use - cursory attention required (ability to back out needed)
Updated In Use - that hasn’t been customized by me, moderate attention required
Updated In Use - that has been customized by me, heavy attention required

I currently tell the difference between the last two using some custom software I’ve built that parses out and identifies the differences between the repo and the currently installed modules. It also displays the update history, which easily identifies a module that has been customized vs. updated from repo. @Michael Rodrigues, I know you and i have talked about the ability for the toolbox to track this not just on a module level, but on an attribute level. That should help.

Here’s my workflow. If this workflow breaks, i can either build a new workflow or abandon doing the work.

New
1. make a note of the date/time (my script preps this list so i can bulk copy/paste into my changelog system)
2. pull them in
Updated not in use
1. make a note of the date/time and the old version of the DS in JSON (so i can roll back if needed) (my script preps this list so i can bulk copy/paste into my changelog system)
2. pull them in
Updated In use - not customized:
1. make a note of the date/time and the old version of the DS in JSON (so i can roll back if needed)
2. check if any changes would cause instances to disappear, new alerts to open, or customizations to be overwritten (decide go/no-go)
3. pull them in
Updated in use - customized:
1. make a note of the date/time and the old version of the DS in JSON (so i can roll back if needed)
2. check if any changes would cause instances to disappear, new alerts to open, or customizations to be overwritten (decide go/no-go)
3. pull into sandbox. Copy script(s) to DS editor in prod and test extensively (decide go/no-go)
4. copy changes from sandbox to prod without overriding prod customizations (most often not fields yet preservable in module toolbox sadly)
5. mark prod DS as audited so it no longer shows up as having an update

Elsewhere here we’ve asked for the ability to download tables in CSV format. Keeping my change log up to date is easy today because my script fetches all the data from the api and formats it in a way that’s easy for me to pull into my changelog. I can’t do that once the functionality is only available in the module toolbox. Having the ability to download the table from the module toolbox would alleviate that issue, especially if i can figure out the filters to replicate the four classifications above.

If a new interface is being designed and it makes a workflow harder or impossible, what’s the point of the update? I get that ease of manufacture is a goal of UIv4, but that can’t be at the expense of usability. Otherwise you’re making an upgrade that costs the customer without providing a corresponding value.

Michael Rodrigues

·2 years ago

@Stuart Weenig I’ve got the backend team looking into the update “no differences” issue. I see us getting this fixed before the old stuff goes away.

I understand the fear of “having the rug pulled out”. The LMX team understands how important these flows are for LogicMonitor customers, and we want them to be solid before they become your only option. Some of these changes are very difficult to make/maintain in parallel (Audit/Skip being one, and RBAC being another), which is why we pulled Audit out earlier. We haven’t seen any issues with “Skip” functionality in the Modules section.

LM User

OP2 years ago

Can we please resolve the large number of modules that appear to need updates but when installing the update it says there are no differences? Can we do that before we pull the rug out from under the existing working workflows at least?

Michael Rodrigues

·2 years ago

Audit in UIv3 will be going away in 194. This is to avoid further divergence with UIv4’s skipped modules.

We recommend using the “Skip” functionality in the Toolbox to replace the older Audit functionality.

There’s no plan to provide API endpoints for module counts by state, but we can reevaluate this based on feedback.

194 will also see the rollout of “in use / not in use” counts and status for some LogicModules (others coming in later releases).

LM User

OP3 years ago

We may actually bump up the timeline of disabling “Audit” in UIv3 due to the migration becoming quite a bit more difficult as time goes on and it diverges further from “Skip”.

This would break our workflow since we couldn’t make it work until the new toolbox has parity.

I’ll also still need some API endpoints (documented please) that will allow me to get counts into a datasource of how many modules are in the different states. These alerts and dashboard widgets are what prompt me to go in and check for modules that need to be updated.

LM User

OP3 years ago

The migration will be automated, so feel free to continue to Audit instead of Skip if that is your workflow. The current plan is to do this at the same time UIv3 for Audit becomes inaccessible, preventing you from being in the situation you describe above.

Presumably that would happen once you reach some sort of feature parity between the old system and the new? I’ve found so many discrepancies that I’ve stopped even going to the module toolbox for anything, but I’m worried of another debacle like the Alerts page where the old will be ripped out while the new is still in buggy enough to be considered alpha by anyone but LM.

Michael Rodrigues

·3 years ago

We’re working on making the Skipped filter (and others) easier to use. We’ve also improved the responsiveness of the whole Modules area which you should see soon.

The migration will be automated, so feel free to continue to Audit instead of Skip if that is your workflow. The current plan is to do this at the same time UIv3 for Audit becomes inaccessible, preventing you from being in the situation you describe above.

LM User

OP3 years ago

Ok, I gave it a college try today. I set my filter to:

Skipped = “Show only skipped” (don’t get me started on why this isn’t the same kind of dropdown as “customized”)
Status = “Outdated”

This seemed to get me the list of modules that i have audited in the repo. There were a couple that showed updates, so i checked them out and they were innocuous so i applied the update. Guess what shows up in the repo as now having an update? The ones I updated through the toolbox. So, the source code for the toolbox and the source code for the repo are different.

How are we supposed to transition to the new system if using the new system throws alerts in the old system (which we’re still using)?

LM User

OP3 years ago

At the least, can you guys let us know when you plan to mark as skipped everything that is currently audited? That would let us know when we should stop auditing and start skipping. As mentioned before, I have 73 audited modules; marking each one as skipped manually would be a big time suck.

Also, when that expected time comes, hopefully it can be more accurate than the “your portal will be upgraded to v.NNN on Monday” message; it’s almost never right, it might be Monday or Tuesday or the next Saturday before it’s upgraded.

LM User

OP3 years ago

If it can be migrated, can it be automated so that i can continue with my current processes until the toolbox is a bit more workable?

Michael Rodrigues

·3 years ago

@Stuart Weenig we’re planning on disabling the old Audit system and migrating those audits to the Skip system in Toolbox. We had previously migrated Audits but we failed to disable it afterwards.

Audit and Skip solve the same use case of hiding updates you’re not interested in.

I suggest using Skip in the Toolbox and not using Audit in UIv3, though again, it will be migrated.

LM User

OP3 years ago

It turns out (based on some conversations with @Michael Rodrigues that Auditing a datasource through the old method is a completely different and totally separate action from “skipping” an update in the module toolbox. Meaning that all 73 of our audited datasources will eventually have to be skipped in the module toolbox.

I just hope they don’t remove functionality that we’re using until the replacement is usable. To be clear, I believe the module toolbox is not usable currently.

Joe Williams

·3 years ago

We reported similar results to our CSM a month or so ago. We were asked why we refuse to use the newer system and we brought forth the reported difference between the two.

We also see datasources marked for update that aren’t customized and are up to date at times in the newer system.

Also our lack of confidence in the newer method as, at least in prior years, wouldn’t actually preserve the customizations made.