New VMware modules dropped | Product Discussions

LM User

OP2 years ago

Going on 7 months waiting for an answer from LM on this.

LM User

OP2 years ago

The these are the modules that are listed as N/A in the “Replaces” column, because they are updates rather than new modules.

addERI_vCenter
addERI_ESXi
VMware_LM_Troubleshooter

Reference:
https://www.logicmonitor.com/support/vmware-vsphere-monitoring#h-vmware-vsphere-logicmodules-in-package

Not what I was asking for during the AMA.

Patrick Rouse

·2 years ago

The these are the modules that are listed as N/A in the “Replaces” column, because they are updates rather than new modules.

addERI_vCenter
addERI_ESXi
VMware_LM_Troubleshooter

Reference:
https://www.logicmonitor.com/support/vmware-vsphere-monitoring#h-vmware-vsphere-logicmodules-in-package

Cole McDonald

·2 years ago

Here are some more details. We ended up having to build a dynamic group to turn off the “VMware ESXi *” set of dataSources. Unsure if the issue is a single DS or all hitting at the same time. we have 64 VMware hosts and 3 device PureStorage SAN under them. The utilization for the SAN devices tripled when these got turned on. I have a ticket in with LM to explore this issue with these. Went through the batchscripts with our virt team and nothing seemed out of place individually in terms of API/SDK calls being made. DS were installed at 2:12PM per the LM audit logs, performance started rising nearly immediately. My suspicion is that the wait queue for disk access started going up due to the volume of requests (10 DS x 64 hosts = 640 requests)

A handful of these are older DSs… most are from the new set.

Cole McDonald

·2 years ago

DO NOT USE THESE YET… I’m dropping a support ticket in with the issues we saw when we turned them on… purestorage array utilization tripled instantly on import!

Specifically the VMWare ESXi * set of them… not sure if it was one or multiple/cumulative yet.

LM User

OP2 years ago

Since i got no response to my question here, I’ve opened up a support case to try to get an answer as to why the difference exists. Once it’s explained and/or fixed, I can finally look at the other modules to see if they’re even worth my time to import.

LM User

OP2 years ago

I imported the VMware troubleshooter under a different name and ran them side by side for a few days. There is one instance discovered under the old module that isn’t discovered under the new module: Incorrect Statistics Level.

Is this because the new modules don’t care about the statistics level? Or does the new troubleshooter not check for that particular error (i.e. it doesn’t have feature parity with the module it’s replacing)?

LM User

OP2 years ago

I imported the VMware troubleshooter under a different name and ran them side by side for a few days. There is one instance discovered under the old module that isn’t discovered under the new module: Incorrect Statistics Level.

Is this because the new modules don’t care about the statistics level? Or does the new troubleshooter not check for that particular error (i.e. it doesn’t have feature parity with the module it’s replacing)?

Patrick Rouse

·2 years ago

For awareness, we’ve fixed an issue with the Enhanced Script NetScan script and updated it in the product docs. The latest script is available here:

https://www.logicmonitor.com/support/vmware-vsphere-monitoring

LM User

OP2 years ago

I imported the VMware troubleshooter under a different name and ran them side by side for a few days. There is one instance discovered under the old module that isn’t discovered under the new module: Incorrect Statistics Level.

Is this because the new modules don’t care about the statistics level? Or does the new troubleshooter not check for that particular error (i.e. it doesn’t have feature parity with the module it’s replacing)?

Patrick Rouse

·2 years ago

But that’s pretty much what your suggested workflow is: Import the new modules, customize them like the old ones, hope to hell nothing alerts while you’re customizing because the people receiving alerts will lose faith in LM, then disable the new ones (which can’t be done in bulk without losing historical data).

So, it sounds like there’s an opportunity to improve the process by including a “Disable New DataSources while customizing” ?

LM User

OP2 years ago

But that’s pretty much what your suggested workflow is: Import the new modules, customize them like the old ones, hope to hell nothing alerts while you’re customizing because the people receiving alerts will lose faith in LM, then disable the new ones (which can’t be done in bulk without losing historical data).

Patrick Rouse

·2 years ago

Let me illustrate the complexity of integrating these updates with existing stuff:

I have to have the exchange (not the module toolbox?) pulled up to know if a module is new or existing (could have just been signaled in the docs).
I have to have the documentation pulled up to know which modules replace which modules.
I have to have the module toolbox pulled up so I can pull in the changes.
I have to search through all my groups for custom thresholds that I have to recreate on the new modules, but I can’t do that until they’re imported. And searching for custom thresholds is made more complicated by the fact that on the alert tuning tab they are displayed by display name, not name. The documentation uses name (rightly so), so knowing which one is which requires me to also look up the module in the repo/toolbox. I can run a report, but those have become less and less reliable with the new schedule-now bug.
I also have to identify all the customizations that we’ve done to the existing modules and apply those to the new modules, which I can’t do until I’ve already pulled the new modules into prod, by which time, they’re already alerting with non-custom alert thresholds, trigger/clear interval settings, discovery customizations, etc..

Forgive me for being a little salty, but this isn’t something that we can just import and see how it goes. The existing modules weren’t tuned for reality/usability, so we had to make significant modifications to get them to work well in a real environment.

Hi, @Stuart Weenig. This is great feedback that we can use to improve our product documentation and in-product (LM Exchange/Toolbox).

Your words “this isn’t something that we can just import and see how it goes” are very important for others to read. If IT personnel don’t have a formal change control process (ITIL or otherwise) and follow the “install it and see how it goes” in a stable production environment, they are taking unnecessary risk that at some point with some product is going to have unintended consequences. Change Control is critically important.

Thanks again.

LM User

OP2 years ago

Let me illustrate the complexity of integrating these updates with existing stuff:

I have to have the exchange (not the module toolbox?) pulled up to know if a module is new or existing (could have just been signaled in the docs).
I have to have the documentation pulled up to know which modules replace which modules.
I have to have the module toolbox pulled up so I can pull in the changes.
I have to search through all my groups for custom thresholds that I have to recreate on the new modules, but I can’t do that until they’re imported. And searching for custom thresholds is made more complicated by the fact that on the alert tuning tab they are displayed by display name, not name. The documentation uses name (rightly so), so knowing which one is which requires me to also look up the module in the repo/toolbox. I can run a report, but those have become less and less reliable with the new schedule-now bug.
I also have to identify all the customizations that we’ve done to the existing modules and apply those to the new modules, which I can’t do until I’ve already pulled the new modules into prod, by which time, they’re already alerting with non-custom alert thresholds, trigger/clear interval settings, discovery customizations, etc..

Forgive me for being a little salty, but this isn’t something that we can just import and see how it goes. The existing modules weren’t tuned for reality/usability, so we had to make significant modifications to get them to work well in a real environment.

Patrick Rouse

·2 years ago

In LM Exchange, any of the new modules are version 1.0.

LM User

OP2 years ago

Any chance that distinction could be made? New modules are fine, but for existing modules, I have to check for customizations and custom thresholds applied at group levels. Knowing which ones are new and which ones are updates (which contradicts the you-can-safely-import-these-because-they-have-different-names message) will cut down on the amount of work I have to do to vet these modules.

Patrick Rouse

·2 years ago

In the modules table, if the “replaces” column is “n/a”, does that mean it’s a new module that didn’t exist before?

That or it's a module update rather than a new module. For example “VMware_LM_Troubleshooter”

LM User

OP2 years ago

In the modules table, if the “replaces” column is “n/a”, does that mean it’s a new module that didn’t exist before?

Patrick Rouse

·2 years ago

At the bottom of the product docs:

Each customers’ change control process is likely to be unique, but you may find value in this high-level (generalized) diagram.

If there are specific questions or scenarios you’d like us to add to the product documentation, please let me know and we’ll be happy to consider adding them.

Thanks @Stuart Weenig

LM User

OP2 years ago

So, no documented migration plan for those not starting from scratch?

Patrick Rouse

·2 years ago

We don’t have any plans to remove instance-based monitoring of ESXi-hosted VMs, but rather are focused on making it fast and frictionless for customers to add Resources for VMs that they determine need OS/workload monitoring.

Patrick Rouse

·2 years ago

Coming from a MSP as well, I agree with Stuart. Is the future plan to deprecate the current modules? And move to a non instance based one? If so that impacts us on our license counts, billing to clients, etc….

Which current modules? Thanks

Patrick Rouse

·2 years ago

The new dashboard are here:

https://github.com/logicmonitor/dashboards/tree/master/Virtualization/VMware%20(Current)

Thanks @Kerry DeVilbiss

Patrick Rouse

·2 years ago

The PropertySource:

Patrick Rouse

·2 years ago

if you have some logic in vCenters that determine what should be monitored as a resource, i.e. tags, naming conventions

As an MSP, we never have that luxury. For the same reason, we can’t use LM’s fancy interface alert filtering.

you don’t have to have a manual process to do this

I don’t have a manual process to do this. It’s sync’d in from our billing system of record.

Any chance you’re going to put out documentation on how to utilize the new datasources without using the netscan? If I import these today, will they start monitoring since I already have things like esx/vcenter/vsphere credentials already added as properties? What’s the migration process for existing customers? Will there be/is there part of this update a property source that will pull the tags so we can still do dynamic grouping based on tags?

From the new product docs → https://www.logicmonitor.com/support/vmware-vsphere-monitoring

Joe Williams

·2 years ago

Coming from a MSP as well, I agree with Stuart. Is the future plan to deprecate the current modules? And move to a non instance based one? If so that impacts us on our license counts, billing to clients, etc….

LM User

OP2 years ago

No, that’s not really the best answer and didn’t answer my question.

LM User

OP2 years ago

if you have some logic in vCenters that determine what should be monitored as a resource, i.e. tags, naming conventions

As an MSP, we never have that luxury. For the same reason, we can’t use LM’s fancy interface alert filtering.

you don’t have to have a manual process to do this

I don’t have a manual process to do this. It’s sync’d in from our billing system of record.

Any chance you’re going to put out documentation on how to utilize the new datasources without using the netscan? If I import these today, will they start monitoring since I already have things like esx/vcenter/vsphere credentials already added as properties? What’s the migration process for existing customers? Will there be/is there part of this update a property source that will pull the tags so we can still do dynamic grouping based on tags?

Patrick Rouse

·2 years ago

Hi, @Stuart Weenig. Migrating to the new modules will not convert instances into Resources. However, the Enhanced Script NetScan filters make it easy to add the ESXi hosts and Mission Critical VMs you want to monitor as Resources. You can setup filters that mirror your business logic and schedule the NetScan to query vCenter on (as frequent as) an hourly basis to add the exact resources you want, so what you deem to need monitoring gets monitored automatically.

So all the netscan really does is add the resources I already have in LM and create some group structure? Pretty sure we’re not going to use it then since adding devices in is associated with billing and we don’t just add stuff in because it exists. Is there a propertysource we can use to grab the additional properties the nestcan would set without all the stuff we don’t need?

Is there any benefit to importing the datasources if I’m not using the netscan?

That’s a huge netscan script and a lot of warnings about changing it. If you’re so worried about people changing it, why not make it a snippet and then the netscan script is 3-4 lines (import the snippet, run the snippet, output the data)?

Certainly, you do not have to use the NetScan. However, if you have some logic in vCenters that determine what should be monitored as a resource, i.e. tags, naming conventions…. the NetScan can use those to automatically add resources as they are created (in less than an hour if you set the NetScan to recurring) so you don’t have to have a manual process to do this. Some customers prohibit use of ICMP NetScans in their Data Centers, so they use a manual process to export information from vCenter, put it in a properly formatted CSV file, then import it into LM. Because this NetScan communicates with vCenter rather than scanning the network, it can automate this process. The organization part of the NetScan makes it easy for VMware admins to find resources, because it puts them in LM in a structure that mirrors what’s in vCenter. So, the VMware admin gets a folder structure that is intuitive to them, and the business can use Dynamic Resource Groups to place resources in LM where it makes sense to the business. For existing customers like yourselves, you’d have to decide if you want to use this method or not. It’s not required.

If the existing modules are working for you without issue and you don’t see an immediate need for any of the new features (shown in the screenshots above), then you can plan to migrate when it makes sense to you. However, if at some point you encounter an issue with a future vSphere 8.x compatibility issue or some other issue with the deprecated modules, your recourse would be to adopt the new/replacement module(s). The new modules are the only ones “supported” for vSphere 8.x.

As for the NetScan, we’re looking at ways to reduce the total steps required and to mitigate user error, but I don’t expect those to come in 2023. For now, it achieved our goals of organizing resources in an manner that’s intuitive for VMware Admins, automating vSphere device discover and onboarding, and reducing the typical time to onboard new hosts and VMs that customers want to monitor as Resources from as much as a week to an hour or less.

Dave Lee

·2 years ago

Pretty sure we’re not going to use it then since adding devices in is associated with billing and we don’t just add stuff in because it exists.

We’re probably in the same camp. I’m pretty sure that we get what we need in terms of host monitoring through vCenter. I can’t imagine that we would add the hosts individually into LM as there would be a cost associated with adding each host. We may just look at customising alerts on the existing datasources so that they take into account of host maintenance mode in the vCenter inventory (i.e. don’t throw an alert if a host is in maint mode).

LM User

OP2 years ago

Hi, @Stuart Weenig. Migrating to the new modules will not convert instances into Resources. However, the Enhanced Script NetScan filters make it easy to add the ESXi hosts and Mission Critical VMs you want to monitor as Resources. You can setup filters that mirror your business logic and schedule the NetScan to query vCenter on (as frequent as) an hourly basis to add the exact resources you want, so what you deem to need monitoring gets monitored automatically.

So all the netscan really does is add the resources I already have in LM and create some group structure? Pretty sure we’re not going to use it then since adding devices in is associated with billing and we don’t just add stuff in because it exists. Is there a propertysource we can use to grab the additional properties the nestcan would set without all the stuff we don’t need?

Is there any benefit to importing the datasources if I’m not using the netscan?

That’s a huge netscan script and a lot of warnings about changing it. If you’re so worried about people changing it, why not make it a snippet and then the netscan script is 3-4 lines (import the snippet, run the snippet, output the data)?

Patrick Rouse

·2 years ago

We’re part the way through onboarding a load of internal and customer VMware platform platforms into LM, so interested in this as well.

I’d be interested to understand the reason for releasing new modules over updating the existing? The commit history for the brand new ones mentioned ‘optimized for vSphere 8’ but the AppliesTo doesn’t seem to limit them to vSphere 8 vCenters/hosts, so loathed to import them just yet as I expect we’ll have a lot of duplication in data collection and alerts if we did.

Dave

Hi, @Dave Lee. Because the new VMware_vSphere* and VMware_vCenterAppliance* modules communicate with vCenter (via API), we recommend following your change control process for migrations rather than running the previous and new modules in parallel (which could put an unnecessary burden on vCenter).

To maintain historical data, make sure to disable previous datasources rather than changing the AppliesTo or deleting them.

Patrick Rouse

·2 years ago

Did anybody else notice the ~44 new and ~5 updated modules around VMware dropping in the last hour or so? Does anyone know how to implement these new modules? Since there was talk of making the instances into resources I don’t want to just bring them in without knowing how it’s going to mess with my device list (which is tightly bound to billing for us).

Hi, @Stuart Weenig. Migrating to the new modules will not convert instances into Resources. However, the Enhanced Script NetScan filters make it easy to add the ESXi hosts and Mission Critical VMs you want to monitor as Resources. You can setup filters that mirror your business logic and schedule the NetScan to query vCenter on (as frequent as) an hourly basis to add the exact resources you want, so what you deem to need monitoring gets monitored automatically.

Patrick Rouse

·2 years ago

That would be interesting… one of our pain points at the moment is putting vSphere hosts into SDT. We don’t actually monitor the hosts as such at the moment, just through the vCenter, so the vCenter is the only resource. As I understand it, if our operations team wants to SDT a host, we’d have to do it under each datasource it appears in under it’s parent vCenter.

If hosts and datastores etc show up a devices that can be separately SDT’d then that’d be great.

Likewise, we’re setting up a separate LM instance to do stuff like this in!

Dave

To accomplish this with the new modules and onboarding process, you can have the Enhanced Script NetScan add your desired vSphere Clusters (ESXi hosts) as Resources and disable the following DataSources:

VMware_vCenter_HostPerformance
VMware_vCenter_HostVSwitch
VMware_vCenter_HostInterfaces

After doing this, you can selectively SDT by DataCenter, Cluster, Resource Pool, or individual ESXi host.

Patrick Rouse

·2 years ago

Below are some highlights about the new VMware vSphere modules.

Patrick Rouse

·2 years ago

https://www.logicmonitor.com/support/vmware-vsphere-monitoring

LM User

OP2 years ago

I think that’s generally the advantage they’re trying to provide. Make things that should be resources into resources (not instances). The same is reportedly going to happen for Meraki (and maybe other controller based tech).

Dave Lee

·2 years ago

That would be interesting… one of our pain points at the moment is putting vSphere hosts into SDT. We don’t actually monitor the hosts as such at the moment, just through the vCenter, so the vCenter is the only resource. As I understand it, if our operations team wants to SDT a host, we’d have to do it under each datasource it appears in under it’s parent vCenter.

If hosts and datastores etc show up a devices that can be separately SDT’d then that’d be great.

Likewise, we’re setting up a separate LM instance to do stuff like this in!

Dave

LM User

OP2 years ago

Supposedly the new ones create resources instead of instances (not sure how that magic works), so that would allow you to do a soft migration maintaining the old stuff while getting the new stuff working.

If the QA on these modules matches the QA on the common configsources, these ain’t getting imported any time soon.

Dave Lee

·2 years ago

We’re part the way through onboarding a load of internal and customer VMware platform platforms into LM, so interested in this as well.

I’d be interested to understand the reason for releasing new modules over updating the existing? The commit history for the brand new ones mentioned ‘optimized for vSphere 8’ but the AppliesTo doesn’t seem to limit them to vSphere 8 vCenters/hosts, so loathed to import them just yet as I expect we’ll have a lot of duplication in data collection and alerts if we did.

Dave