Dynamic Device Groups - constrain membership to parent folder | Product Discussions

Dynamic Device Groups - constrain membership to parent folder

Hi,

We have the following hierarchy setup:

Our company is top level, e.g. LOGICMONITOR
|--Customer Name, e.g. GOOGLE, MICROSOFT
|----’All Devices’ static group (when we add new devices, we add them here)
|----Category named group, e.g. ‘Infrastructure’ static group (with no actual devices added here)
|------Function named group, e.g. ‘Active Directory’, ‘DNS Server’, etc. dynamic group with custom query (hasCategory("MicrosoftDomainController"), hasCategory("Windows_DNS") etc.)

So:

The problem with this, and it is partly a misunderstanding on my part, is that I expected the (nested) dynamic groups to automatically be constrained to their parent container.

How do I achieve this? The only thing I can think of is having the function-named group custom query be something like this:

join(system.groups,",")=~"GOOGLE/_All Devices" && hasCategory("MicrosoftDomainController")

But that means every dynamic group for every customer will need that, which is fine. But, is there a better, more native/default way?

Thanks

ldoodle

Posted 2 years ago·Last reply 2 years ago

28 comments

ldoodle

OP2 years ago

Unlike wmi.user / wmi.pass which I have at the collector group level, the esx and veeam properties don’t seem to work at the collector/collector group level so must be configured at the customer folder level.

the credentials of the user running the collector service will be used for WMI calls

Ah, that makes total sense. I didn’t even think of that. Our collector service account uses that same creds as wmi.user/wmi.pass

Right, so defining them on the collector/collector group doesn’t really do anything.

yeah. I just thought the collector service creds were just to get the collector going.

ldoodle

OP2 years ago

Interestingly, the ‘New UI’ preview Add Wizard seem to handle my scenario; it doesn’t fail on creds and then prompt for appropriate ones. It seems a bit ‘softer’ than old UI.

LM User

·2 years ago

Unlike wmi.user / wmi.pass which I have at the collector group level, the esx and veeam properties don’t seem to work at the collector/collector group level so must be configured at the customer folder level.

the credentials of the user running the collector service will be used for WMI calls

Ah, that makes total sense. I didn’t even think of that. Our collector service account uses that same creds as wmi.user/wmi.pass

Right, so defining them on the collector/collector group doesn’t really do anything.

ldoodle

OP2 years ago

Unlike wmi.user / wmi.pass which I have at the collector group level, the esx and veeam properties don’t seem to work at the collector/collector group level so must be configured at the customer folder level.

the credentials of the user running the collector service will be used for WMI calls

Ah, that makes total sense. I didn’t even think of that. Our collector service account uses that same creds as wmi.user/wmi.pass

Mike Moniz

·2 years ago

So as far as I know properties on the Settings Collector tab has no affect on the resources or any other pages, even if the server hosting the collector software is also in the resource tab (which btw is not required). So if you set wmi.user/pass on a collector it will do nothing. If it is working, it’s likely because the collector software will default to using the user/creds the Collector windows service has. Which I think is the preferred option in most cases so you don’t need to provided extractable windows creds into LM.

Hmm, I can’t think of a way around the issue you are mentioning though.

P.S. Does LM no longer attempt to set esx properties at the root level when using the wizard?

LM User

·2 years ago

Unlike wmi.user / wmi.pass which I have at the collector group level, the esx and veeam properties don’t seem to work at the collector/collector group level so must be configured at the customer folder level.

I’m not sure this is working the way you think. I’ve not seen properties defined on the collector/collector group inherited by the devices assigned to the collector/collector group. However, when you assign a device to a collector, if the device doesn’t have wmi.user/wmi.pass assigned (either on the device or on the resource group), the credentials of the user running the collector service will be used for WMI calls. So while you are specifying wmi.user and wmi.pass on the collector/collector group, and while things appear to be working, they may be working for a reason different than the one you think.

FWIW: when we add devices, the static group we add it into either contains the credentials the device will need or we specify the credentials as properties on the device.

ldoodle

OP2 years ago

I’ve just found a little issue with my overall setup. Since I now have a global _All Devices static group where every resource gets added, when adding resources that are say ESX hosts or Windows hosts running Veeam, it’s not going to know the creds for those until it’s been added and the PropertySource has run to apply the property that makes the resource be pulled into a customer group to then inherit these properties! So during the add wizard, I will have to manually specify them when prompted.

Unlike wmi.user / wmi.pass which I have at the collector group level, the esx and veeam properties don’t seem to work at the collector/collector group level so must be configured at the customer folder level.

Is there anyway around this, excluding adding resources directly in the customer folder again?

LM User

·2 years ago

One thing you may want to also keep in mind is supporting multiple collector groups per customer. Since collector groups also group together fail overs, you may have to create multiple ones if you don’t want to failover any customer collector to just any other customer collector. Especially across multiple sites which might not have connections between them or you don’t want to failover to another site over vpn.

Yep, but only for ABCGs. If you’re manually defining the failover collector, collector group has no impact.

Mike Moniz

·2 years ago

LM User

·2 years ago

However, you can also post them in the Feature Request section of the LM Community:
https://community.logicmonitor.com/feature-requests-5

Yeah, if you haven’t already done so, please submit this directly through the product. Posting it on the community is good because users like myself will copy and paste your feature request directly into the feature request submission form in the product. However, the product team does not ingest feature requests submitted through the community, only those submitted through the product itself. Dare i say they don’t even read them?

Joseph Brett

·2 years ago

Hey @ldoodle

Yea, so you can do it via your portal like this:

However, you can also post them in the Feature Request section of the LM Community:

https://community.logicmonitor.com/feature-requests-5

ldoodle

OP2 years ago

Had a bit of panic as I could see it was working when testing but none of my dynamic groups were being populated. Then remembered on the other side of the pond you guys use z in organisation 😂

@LMjosephBrett how do we submit feature requests? I have 2:

Ability to create UDFs for importing in to scripts, e.g. within LogicModules, alongside AppliesTo Functions have a Script Functions group where we can put things like @Stuart Weenig LM_API function. Then in a PropertySource for example we can use the import tag, or an equivalent
I still think collectorgroupid and collectorgroupdesc default properties should be a thing, and should be applied to every resource add the point of adding them, like collectorid and collectordesc are

Thanks again for all the replies and help.

LM User

·2 years ago

Ha, this ain’t my first rodeo. Developing it would be even faster if LM supported python natively.

Yes there is a way of defining it only once and importing it: collector snippets. However, LM doesn’t allow mere mortals like you and me to manage snippets. It’s exclusive to LM devs. Submit a feature request for it so i’m not the only one.

You can go the route of creating a full java library with that function in it and you could install that library into the groovy path on each collector.

Or you can just include it at the bottom of each module you want to use it in (what I do).

ldoodle

OP2 years ago

Wowsers, that’s amazing! Thank you so much.

Now… is it possible to define the LM_API function somewhere else for re-use, rather than embedding it in each script?

LM User

·2 years ago

So it seems tenant id would only work if each customer resourced is statically placed in a customer nested folder in the first place.

Yes, the relationship between the device and the customer will have to be defined by you somewhere.

It does seem a shame (maybe omission) that you can’t map a collector to its collector group and in turn the collector group name, since for us the collector group name is the exact acronym we need.

You can do this. You just need to make an API call to grab the collector groups. Unintuitively, that wouldn’t be a call to /setting/collector/groups as that doesn’t return the IDs of the collectors in the group. Instead, do a get to /setting/collector/collectors/:id. This will give you back the collector’s group name (among other things, but we can specify that we only want collectorGroupName).

So, it would be something like this:

import com.santaba.agent.util.Settings<br/>import groovy.json.*<br/>import javax.crypto.Mac<br/>import javax.crypto.spec.SecretKeySpec<br/>import org.apache.commons.codec.binary.Hex<br/>import org.apache.http.client.methods.*<br/>import org.apache.http.entity.*<br/>import org.apache.http.HttpEntity<br/>import org.apache.http.impl.client.*<br/>import org.apache.http.util.EntityUtils<br/><br/>deviceCollectorId = hostProps.get("system.collectorId")<br/>collector = LM_API("GET","/setting/collector/collectors/${deviceCollectorId}","lmaccess",[:],["fields": "collectorGroupName"])<br/>if (collector.code == 200){<br/>    println("organization=${collector?.body?.collectorGroupName}")<br/>}<br/>return 0<br/><br/>def LM_API(httpVerb, endpointPath, creds="lmaccess", data = [:], queryParams=[:]){<br/>    def api_id = hostProps.get("${creds}.id")<br/>    def api_key = hostProps.get("${creds}.key")<br/>    def api_company = hostProps.get("${creds}.company") ?: Settings.getSetting(Settings.AGENT_COMPANY)<br/>    def queryParamsString = queryParams.collect{k,v->"${k}=${v}"}.join("&")<br/>    def url = "https://${api_company}.logicmonitor.com/santaba/rest${endpointPath}?${queryParamsString}"<br/>    epoch_time = System.currentTimeMillis()<br/>    def json_data = JsonOutput.toJson(data)<br/>    if ((httpVerb == "GET") || (httpVerb == "DELETE")){requestVars = httpVerb + epoch_time + endpointPath}<br/>    else {requestVars = httpVerb + epoch_time + json_data + endpointPath}<br/>    hmac = Mac.getInstance("HmacSHA256")<br/>    secret = new SecretKeySpec(api_key.getBytes(), "HmacSHA256")<br/>    hmac.init(secret)<br/>    hmac_signed = Hex.encodeHexString(hmac.doFinal(requestVars.getBytes()))<br/>    signature = hmac_signed.bytes.encodeBase64()<br/>    CloseableHttpClient httpclient = HttpClients.createDefault()<br/>    if (httpVerb == "GET"){http_request = new HttpGet(url)}<br/>    else if (httpVerb == "PATCH"){<br/>        http_request = new HttpPatch(url)<br/>        http_request.setEntity(new StringEntity(json_data, ContentType.APPLICATION_JSON))}<br/>    else if (httpVerb == "PUT"){<br/>        http_request = new HttpPut(url)<br/>        http_request.setEntity(new StringEntity(json_data, ContentType.APPLICATION_JSON))}<br/>    else if (httpVerb == "POST"){<br/>        http_request = new HttpPost(url)<br/>        http_request.setEntity(new StringEntity(json_data, ContentType.APPLICATION_JSON))}<br/>    else {http_request = null}<br/>    if (http_request){<br/>        http_request.setHeader("Authorization" , "LMv1 " + api_id + ":" + signature + ":" + epoch_time)<br/>        http_request.setHeader("Accept", "application/json")<br/>        http_request.setHeader("Content-type", "application/json")<br/>        http_request.setHeader("X-Version", "3")<br/>        response = httpclient.execute(http_request)<br/>        body = EntityUtils.toString(response.getEntity())<br/>        code = response.getStatusLine().getStatusCode()<br/>        return ["body":new JsonSlurper().parseText(body), "code":code]}<br/>    else {return ["body":"", "code": -1]}<br/>}

You’d need to set properties at the root for lmaccess.id, lmaccess.key and optionally, lmaccess.company. If you wanted to generate an API credential set specifically to this task (and you should), just change the prefix in line 13 from “lmaccess” to whatever you want to prefix the properties with.

This outputs a property called auto.organization with the value of the collector group name of the collector identified by system.collectorId.

ldoodle

OP2 years ago

Yeah that’s what I thought. But, if my customer _All Devices group is now a dynamic group, resources won’t land anywhere inside the customer parent folder now to pick up that tenant id value.

So it seems tenant id would only work if each customer resourced is statically placed in a customer nested folder in the first place.

I initially interpreted the suggestion of tenant id that LM picks the value from somewhere automatically (like the collector that’s polling it).

New structure:

_All Devices static group ← everything gets added here
|--GOOGLE
|-----_All Devices dynamic group ← this previously populated based on system.collectordesc value, but I changed the propertysource (shown below) so now use system.collectorid

Current plan would be to just add additional case statements to cover multiple collectors and when adding new customers.

It does seem a shame (maybe omission) that you can’t map a collector to its collector group and in turn the collector group name, since for us the collector group name is the exact acronym we need.

It seems like a resource-level built in system.collectorgroup (collectorgroupid and collectorgroupdesc) property (just like collectorid and collectordesc) would quite easily solve this. Then I could simplify my propertysource to just return collectorgroupdesc and not need any logic at all. If I rename the collectorgroup, next time the propertysource is run devices would pick up that update.

LM User

·2 years ago

In portal settings, you specify the name of the property that identifies the tenant. Then you set that property on the device, either directly or by specifying it as a group property that the device will inherit. Then some automagic happens to set the value of that property as the value of system.tenant.identifier.

ldoodle

OP2 years ago

No I haven’t.

In Portal Settings it’s already set at ‘tenant.identifier’. How do I then use it on a folder?

If I applied it to the master _All Devices group, does it just work out what the tenantid is for a given resource and populate the appropriate value?

LM User

·2 years ago

Have you looked into using the tenantid? After setting it in both the portal settings and on an folder, devices will inherit it, then a propertysource auto sets it as a system property.

You can then use system.tenant.identifier as your property to match on.

That’s probably the best way. It wasn’t available when we built out our group structure so we didn’t use it. But if i were to rebuild, that would be the way I’d go.

Joe Williams

·2 years ago

Have you looked into using the tenantid? After setting it in both the portal settings and on an folder, devices will inherit it, then a propertysource auto sets it as a system property.

You can then use system.tenant.identifier as your property to match on.

ldoodle

OP2 years ago

e.g. each collector has an ID, so can we use the ID to find which collector group it is in, then use the collector group name instead?

Syntactically I’m thinking of SQL join clauses - could something like this be achieved in a PropertySource to apply the auto.organisation value instead of what I’ve done above?

propertysource run through all resources in _All Devices group
resource system.collectorid value
join to collector resource on ID value
join to collector group
apply collector group name as auto.organisation value

Then the dynamic query would become auto.organisation == “<CUSTOMERACRONYM>”

ldoodle

OP2 years ago

Thanks for those replies. After a bit of testing I did work out that you can’t use inherited properties, and some of your suggestions I tried and do work.

But I did something a bit different (which seem close to @Stuart Weenig config), since I wanted to make much more use of dynamic grouping… not sure if this is good or bad.

Initially, each customer device went in an _All Devices static group, inside their parent folder. I’ve changed this so now every single device is stored in a single _All Devices group, outside of any customer folder. This means staticgroups lookup no longer works because all devices are in one place
Each customer parent group I added a custom property - ‘organisation’, which contains the customer prefix (acronym)
The customer group now has a dynamic _All Devices group, and this currently is using system.collectordesc =~ “<CollectorVMNamePrefix>” (question below on this) to pull in appropriate devices for each cutsomer. We build the VMs for the collector so have full control over the name, e.g. ‘<CUSTOMERACRONYM>COLLECTOR01’, ‘<CUSTOMERACRONYM>COLLECTOR02’
I then created a custom PropertySource that has AppliesTo filter of the new singular _All Devices group, looks up the customer parent group’s organisation property value, and adds an auto.organisation property to every resource using the retrieved organisation value
Now in my dynamic groups I’m using auto.organisaiton == “<CUSTOMERACRONYM>” && hasCategory(“MSSQL”) for example

This is, so far, working really well. However, my only concern...

On point 3, we use Auto Balanced Collector Groups by default with no exception. Rather than using system.collectordesc =~ “<CUSTOMERACRONYM>” in the customer’s _All Devices dynamic group query, is there anyway we can do it based on the collector group name instead? I cannot see a property like system.collectorgroup for example. We have system.collectorid but I wanted to avoid that since we have multiple collectors in each customer so the query would have to be using something like multiple ORs to capture all IDs. And if we replace a collector with a new one, we get a new ID so we’d have to keep that query up to date.

We may also come across a customer that has a rigid naming convention for VMs, or a customer that has multiple locations and VM prefixes contain the location prefix etc. etc.

So if we could base this query on something else instead that tells us which collector group a device belongs to that would make sense.

LM User

·2 years ago

We have almost the same group structure. What we do is make sure that every device belongs to one static group. We do this with automation. That static group allows its members to inherit the customer id property. Then we have a property source that takes that inherited property and makes it a device level property. Then each of our dynamic groups under our customers constrain on whatever their original applies to is, plus the device level customer id property.

Joseph Brett

·2 years ago

Hey @ldoodle

So, for dynamic groups when it comes to properties, you cannot use inherited properties or categories as this could lead to circular references. We don’t allow for circular references. However, if you have a static group for all of your individual customer devices you can use that.

Using your initial example, you could use a join statement like this:

join(system.staticgroups,",")=~"Google/All Devices"

This will need to be used in every dynamic group under that customers group to ensure it’s querying only that customers devices. But it should allow you to accomplish what you are looking to do.

In the event that you didn’t see this support article already, this doc goes over appliesTo scripting:

https://www.logicmonitor.com/support/terminology-syntax/scripting-support/what-is-lms

ldoodle

OP2 years ago

I’m playing with properties at the parent group level. These are applying properly, but why is this happening...

Doesn’t work in Dynamic Group custom query:

Works in AppliesTo function:

ldoodle

OP2 years ago

Ah, can use system.staticgroups instead.

ldoodle

OP2 years ago

Actually, that doesn’t work. I was testing the query in the AppliesTo function, but when trying to use it in dynamic group custom query I get “Only properties of the host itself, not inherited properties, can be used in dynamic groups.”

1	SUYASH GAIKWAD
2	RR Robin Ratcliffe
3	AH Alejandra Herrera
4	HS Hunter Sparks
5	LT Lily Trbovic
6	AG Asher Guinzy