Forum Discussion

zwheeler-jm's avatar
zwheeler-jm
Icon for Neophyte rankNeophyte
6 months ago

Bulk Removal of System.Categories Value

We accidentally added the value 'ZertoAppliance' to the system.categories field for 800+ devices due to a poorly written PropertySource.  And because we have DataSources and EventSources that apply based on that PropertySource we had about 800 non-Zerto devices trying to grab Zerto data.

LogicMonitor does not automatically remove a value from system.categories field if the PropertySource is no longer valid.  The only option to bulk remove this value in this situation is via the API.

Below is a compilation of calls to the API using Powershell that corrected this issue for us.  Hopefully it helps the community, even if simply giving examples of calling the API with Powershell.

Disclaimer: Run at your own risk - this does not have any 'pause-continue' logic in it.

Adding links to the KBs that were helpful to me in writing this solution.  You will notice a lot of the code below is simply copied and pasted from the KBs.

 

<# Use TLS 1.2 #>
[Net.ServicePointManager]::SecurityProtocol = [Net.SecurityProtocolType]::Tls12

<# Account Info #>
$accessId = 'myAccessId'
$accessKey = 'myAccessKey'
$company = 'myCompany'

<# Request Details #>
$httpVerb = 'GET'
$resourcePath = '/device/devices'
$queryParams = '?fields=customProperties,name,id&filter=customProperties.name:system.categories,customProperties.value~ZertoAppliance&size=1000'

<# Construct URL #>
$url = 'https://' + $company + '.logicmonitor.com/santaba/rest' + $resourcePath + $queryParams

<# Get current time in milliseconds #>
$epoch = [Math]::Round((New-TimeSpan -start (Get-Date -Date "1/1/1970") -end (Get-Date).ToUniversalTime()).TotalMilliseconds)

<# Concatenate Request Details #>
$requestVars = $httpVerb + $epoch + $resourcePath

<# Construct Signature #>
$hmac = New-Object System.Security.Cryptography.HMACSHA256
$hmac.Key = [Text.Encoding]::UTF8.GetBytes($accessKey)
$signatureBytes = $hmac.ComputeHash([Text.Encoding]::UTF8.GetBytes($requestVars))
$signatureHex = [System.BitConverter]::ToString($signatureBytes) -replace '-'
$signature = [System.Convert]::ToBase64String([System.Text.Encoding]::UTF8.GetBytes($signatureHex.ToLower()))

<# Construct Headers #>
$auth = 'LMv1 ' + $accessId + ':' + $signature + ':' + $epoch
$headers = New-Object "System.Collections.Generic.Dictionary[[String],[String]]"
$headers.Add("Authorization",$auth)
$headers.Add("Content-Type",'application/json')

<# Make Request #>
$response = Invoke-RestMethod -Uri $url -Method $httpVerb -Header $headers

<# Print status and body of response #>
$status = $response.status
$data = $response.data.items

<# Set value in system.categories to be removed #>
$badString = 'ZertoAppliance'

<# Initialize an array of devices to be changed #>
$results = @()

<# Iterate through the response data #>
ForEach ($item in $data) {

    <# Get the current value of system.categories #>
    $currentValue = ($item.customProperties | ? {$_.name -eq 'system.categories'}).value

    <# Only add to results array if matches #>
    if ( $currentValue -match $badString ) {

        <# Rebuild value of system.categories #>
        $newValue = ($currentValue.split(',') | ? {$_ -ne $badString}) -join ","

        <# Create PSObject of device id, name, and values #>
        $result = New-Object PSObject -Property @{
            id = $item.id
            name = $item.name
            currentData = $currentValue
            newData = $newValue
        }

        <# Add PSObject to array of devices to be changed #>
        $results += $result

    } else {
        # Nothing to do because it's not mislabeled
        # Add verbose logging if you want output
    }
}

<# Dump the results array to validate the value is being removed #>
$results

<# Iterate through the array of devices to be changed #>
ForEach ($item in $results) {

    <# Request Info #>
    $httpVerb = 'PATCH'
    $resourcePath = '/device/devices/' + $item.id
    $queryParams = '?patchFields=customProperties&opType=replace'
    $data = '{"customProperties":[{"name":"system.categories","value":"' + $item.newData + '"}]}'

    <# Construct URL #>
    $url = 'https://' + $company + '.logicmonitor.com/santaba/rest' + $resourcePath + $queryParams

    <# Get current time in milliseconds #>
    $epoch = [Math]::Round((New-TimeSpan -start (Get-Date -Date "1/1/1970") -end (Get-Date).ToUniversalTime()).TotalMilliseconds)

    <# Concatenate Request Details #>
    $requestVars = $httpVerb + $epoch + $data + $resourcePath

    <# Construct Signature #>
    $hmac = New-Object System.Security.Cryptography.HMACSHA256
    $hmac.Key = [Text.Encoding]::UTF8.GetBytes($accessKey)
    $signatureBytes = $hmac.ComputeHash([Text.Encoding]::UTF8.GetBytes($requestVars))
    $signatureHex = [System.BitConverter]::ToString($signatureBytes) -replace '-'
    $signature = [System.Convert]::ToBase64String([System.Text.Encoding]::UTF8.GetBytes($signatureHex.ToLower()))

    <# Construct Headers #>
    $auth = 'LMv1 ' + $accessId + ':' + $signature + ':' + $epoch
    $headers = New-Object "System.Collections.Generic.Dictionary[[String],[String]]"
    $headers.Add("Authorization",$auth)
    $headers.Add("Content-Type",'application/json')

    <# Make Request #>
    $response = Invoke-RestMethod -Uri $url -Method $httpVerb -Header $headers -Body $data

    <# Dump the result of each PATCH call #>
    $response
    
}
  • Anonymous's avatar
    Anonymous

    Cool, well done!

    You should make sure to put v=3 in your query params or X-Version: 3 in your headers to make sure you are using the current supported API version.

  • Going to drop an update that this script ran great for devices not in the cloud.  The calls to update our 100+ devices in Azure returned a 200 OK without changing the content of the system.categories field.

    I did try different find-replace or escape functions or even changing to v3 with no improvement yet.  The Swagger for v3 API also talks about system.categories being an exception.

        <# Request Info #>
        $httpVerb = 'PATCH'
        #$resourcePath = '/device/devices/' + $item.id #v1 query
        #$queryParams = '?patchFields=customProperties&opType=replace' # v1 query
        $resourcePath = '/device/devices/' + $item.id + '/properties/system.categories' #v3 query
        $queryParams = '?v=3' # v3 query
    
        #$data = '{"customProperties":[{"name":"system.categories","value":"' + $item.NewData + '"}]}' # v1 data payload
        $data = '{"value":"' + $item.newData + '"}'  # v3 data payload
    
        # v3 data mangling
        #$value = [uri]::EscapeDataString($item.NewData)
        #$value = $item.newData.replace('/','%2F')
        #$value = $item.newData.replace('/','%252F')
    
        #$data = '{"customProperties":[{"name":"system.categories","value":"' + $value + '"}]}' # v1 data payload
        #$data = '{"value":"' + $value + '"}' # v3 data payload

    The audit log in LogicMonitor has records of my calls but the middle is blank.  I did update my case with LogicMonitor Support - this is an interesting one.

    Update host US-NC:vm:myserver, , via API token xyz123

    • zwheeler-jm's avatar
      zwheeler-jm
      Icon for Neophyte rankNeophyte

      Can't edit  but I think it has to do with the slash in one of the categories - that's the reason I have been trying different solutions for the '/'.

      value
      -----
      ZertoAppliance,collectorDataSources,Azure/VirtualMachine