When an anomaly isn't an anomaly what could i do?
What can i do when anomaly detection wont work ( something that is seen on a regular basis, and dynamic threshold also wont help where it is within range? For example a drive on a server gets filled with data ( drive is normally cleared down on a daily basis ) but when someone decides to upload a larger than expected amount the drive hasn't been cleared or with other uploads throughout the day there isn't enough space. You are happy if the drive is above 80% during the night because if it hasn't cleared it can be dealt with in the morning ( no need to get anyone out of bed ) but if there is a rapid spike ( more than 2.5% growth in used space in a 30min period ) then they need an alert to get out of bed and fix / make enough room for the data. A possible solution is a datasourcethat will alert if the drive is over the 80% but only with that rapid growth. DataSource calls the api for the last 30min worth of data and calculates the growth rate. The below is the code for a C drive but the drive letter can be changed easily in the code below, same with the 2.5% and the 80% values, they could also be parameterised for different ranges on different devices. <# Use TLS 1.2 #> [Net.ServicePointManager]::SecurityProtocol = [Net.SecurityProtocolType]::Tls12 <# account info #> $accessId = '##apiaccessid.key##' $accessKey = '##apiaccesskey.key##' $company = '##company##' $deviceId = "##system.deviceId##" <# request details #> $httpVerb = 'GET' $resourcePath = "/device/devices/$deviceId/devicedatasources" $queryParams = '?filter=dataSourceName:"WinVolumeUsage-"' <# Construct URL #> $url = 'https://' + $company + '.logicmonitor.com/santaba/rest' + $resourcePath + $queryParams <# Get current time in milliseconds #> $epoch = [Math]::Round((New-TimeSpan -start (Get-Date -Date "1/1/1970") -end (Get-Date).ToUniversalTime()).TotalMilliseconds) <# Concatenate Request Details #> $requestVars = $httpVerb + $epoch + $data + $resourcePath <# Construct Signature #> $hmac = New-Object System.Security.Cryptography.HMACSHA256 $hmac.Key = [Text.Encoding]::UTF8.GetBytes($accessKey) $signatureBytes = $hmac.ComputeHash([Text.Encoding]::UTF8.GetBytes($requestVars)) $signatureHex = [System.BitConverter]::ToString($signatureBytes) -replace '-' $signature = [System.Convert]::ToBase64String([System.Text.Encoding]::UTF8.GetBytes($signatureHex.ToLower())) <# Construct Headers #> $auth = 'LMv1 ' + $accessId + ':' + $signature + ':' + $epoch $headers = New-Object "System.Collections.Generic.Dictionary[[String],[String]]" $headers.Add("Authorization",$auth) $headers.Add("Content-Type",'application/json') $headers.Add("X-Version","3") <# Make Request #> $response = Invoke-RestMethod -Uri $url -Method $httpVerb -Header $headers <# Get Device DataSource ID #> $deviceDataSourceId = $response.items.id <# request details #> $httpVerb = 'GET' $resourcePath = "/device/devices/$deviceId/devicedatasources/$deviceDataSourceId/data" $queryParams = '' <# Construct URL #> $url = 'https://' + $company + '.logicmonitor.com/santaba/rest' + $resourcePath + $queryParams <# Get current time in milliseconds #> $epoch = [Math]::Round((New-TimeSpan -start (Get-Date -Date "1/1/1970") -end (Get-Date).ToUniversalTime()).TotalMilliseconds) <# Concatenate Request Details #> $requestVars = $httpVerb + $epoch + $data + $resourcePath <# Construct Signature #> $hmac = New-Object System.Security.Cryptography.HMACSHA256 $hmac.Key = [Text.Encoding]::UTF8.GetBytes($accessKey) $signatureBytes = $hmac.ComputeHash([Text.Encoding]::UTF8.GetBytes($requestVars)) $signatureHex = [System.BitConverter]::ToString($signatureBytes) -replace '-' $signature = [System.Convert]::ToBase64String([System.Text.Encoding]::UTF8.GetBytes($signatureHex.ToLower())) <# Construct Headers #> $auth = 'LMv1 ' + $accessId + ':' + $signature + ':' + $epoch $headers = New-Object "System.Collections.Generic.Dictionary[[String],[String]]" $headers.Add("Authorization",$auth) $headers.Add("Content-Type",'application/json') <# Make Request #> $response = Invoke-RestMethod -Uri $url -Method $httpVerb -Header $headers <# Print status and body of response #> $status = $response.status $body = $response.data | ConvertTo-Json -Depth 5 function Select-Nth { param([int]$N) $Input | Select-Object -First $N | Select-Object -Last 1 } $array1 = @($response.data.instances.'WinVolumeUsage-C:\'.values) $first = $array1[0] | Select-Nth 3 $last = $array1[19] |Select-Nth 3 $growth = $first - $last if (($growth -gt 2.5) -and ($first -ge 80)){ return 1 }else { return 2 } Hope this gives you some ideas to develop alerting further😁
DFSR Replication Backlog
XKJNGZ Uses Powershell to make WMI queries to get the current backlog file count for each outbound DFSR partner on each DFSR share. These queries can be expensive if the backlog is large, so the polling interval is set to 10 minutes. If there is no backlog, the script finishes quickly. No default alerting is set, but I would recommend adding a threshold to be notified of potential replicationissues. NOTE* - The collector must be able to reach both DFSR partners and will use the same credentials to make the queries for both.