Getting Datapoint data via REST API

I raised this issue with Support last week, as well - hadn't seen this post until just now.  Ticket 87499, for reference.

Datapoint filtering doesn't appear to be working in REST API calls with a query parameter like this
?datapoints=datapoint1,datapoint5

This just results in the first two datapoints in the datasource being returned, rather than the 1st and the 5th.

Okay, I see - and I am able to replicate the issue.  It appears to be a bug with the current implementation, sorry about that - we'll get that fixed with the next release!

19 hours ago, Sarah Terry said:

Hi Tom - If you can share the details of your specific request here, I can take a look. Thanks!

Sure... e.g.

resourcePath = '/device/devices/73023/devicedatasources/1573061/instances/1928726/data'

Two alternative queries:
queryParams = '?datapoints=requests_queue_depth_priority_higher' 

queryParams = '?datapoints=requests_sent_to_team_last_day'

Results - note the data is identical, also, "requests_queue_depth_priority_higher" should be a list of integers as per attached image:

Response Status: 200
Response Body: {
  "status" : 200,
  "errmsg" : "OK",
  "data" : {
    "dataSourceName" : "[can't say sorry]",
    "dataPoints" : [ "requests_queue_depth_priority_higher" ],
    "values" : [ [ 0.0249 ], [ 0.0337 ], [ 0.0155 ], [ 0.044 ], [ 0.0476 ], [ 0.0817 ], [ 0.0646 ], [ 0.0589 ], [ 0.0201 ], [ 0.0686 ], [ 0.0589 ], [ 0.0713 ], [ 0.0926 ], [ 0.0354 ], [ 0.034 ], [ 0.0418 ], [ 0.0785 ], [ 0.06 ], [ 0.0434 ], [ 0.0998 ], [ 0.1075 ], [ 0.0537 ], [ 0.0934 ], [ 0.0964 ], [ 0.1088 ], [ 0.1086 ], [ 0.1251 ], [ 0.1058 ], [ 0.0316 ], [ 0.0932 ] ],
    "time" : [ 1504198000000, 1504197880000, 1504197760000, 1504197640000, 1504197520000, 1504197400000, 1504197280000, 1504197160000, 1504197040000, 1504196920000, 1504196800000, 1504196680000, 1504196560000, 1504196440000, 1504196320000, 1504196200000, 1504196080000, 1504195960000, 1504195840000, 1504195720000, 1504195600000, 1504195480000, 1504195360000, 1504195240000, 1504195120000, 1504195000000, 1504194880000, 1504194760000, 1504194640000, 1504194520000 ]
  },
  "nextPageParams" : ""
}

Response Status: 200
Response Body: {
  "status" : 200,
  "errmsg" : "OK",
  "data" : {
    "dataSourceName" : "[can't say sorry]",
    "dataPoints" : [ "requests_sent_to_team_last_day" ],
    "values" : [ [ 0.0249 ], [ 0.0337 ], [ 0.0155 ], [ 0.044 ], [ 0.0476 ], [ 0.0817 ], [ 0.0646 ], [ 0.0589 ], [ 0.0201 ], [ 0.0686 ], [ 0.0589 ], [ 0.0713 ], [ 0.0926 ], [ 0.0354 ], [ 0.034 ], [ 0.0418 ], [ 0.0785 ], [ 0.06 ], [ 0.0434 ], [ 0.0998 ], [ 0.1075 ], [ 0.0537 ], [ 0.0934 ], [ 0.0964 ], [ 0.1088 ], [ 0.1086 ], [ 0.1251 ], [ 0.1058 ], [ 0.0316 ], [ 0.0932 ] ],
    "time" : [ 1504198000000, 1504197880000, 1504197760000, 1504197640000, 1504197520000, 1504197400000, 1504197280000, 1504197160000, 1504197040000, 1504196920000, 1504196800000, 1504196680000, 1504196560000, 1504196440000, 1504196320000, 1504196200000, 1504196080000, 1504195960000, 1504195840000, 1504195720000, 1504195600000, 1504195480000, 1504195360000, 1504195240000, 1504195120000, 1504195000000, 1504194880000, 1504194760000, 1504194640000, 1504194520000 ]
  },
  "nextPageParams" : ""
}

Hi Tom - If you can share the details of your specific request here, I can take a look. Thanks!

On 24/08/2017 at 8:36 PM, Sarah Terry said:

You can now specify a datapoints query parameter to limit the data returned in the request, like this: GET /device/devices/{id}/devicedatasources/{id}/instances/{id}/data?datapoints=datapoint1,datapoint5.  Does that satisfy both of your use cases?  

Documentation updated here: https://www.logicmonitor.com/support/rest-api-developers-guide/data/get-data/

@Mosh you should be able to already filter by epoch using the start and end parameters - is that what you meant by datapoint timestamp?

Hi Sarah,

Thank you for getting this pushed through so quickly - the Datapoint Filter shows me a reduced set of data which is a good start...unfortunately, it doesn't matter which datapoint I filter on, I always get the same data back even though "dataPoints" : [ "my_data_point" ], shows correctly in the result.

Should I raise this with LM support?

Thanks,

Tom.

I thought I had posted at one time (but obviously didn't), here's a PowerShell script that parses through the data that's returned so that you can have a good object to work with. Can see what's done here to possibly work in whatever language you want to program in. 

there's some work that I haven't finished on this (mainly if it doesn't include an instance ID), but thanks @Sarah Terry for the additional rest path as well, I'll be using that to narrow this function even more! 

function lm-getdatarest {
[CmdletBinding()]
Param (
      [Parameter(Position=0,Mandatory=$true)]
      [string]$DeviceID,
      [Parameter(Position=1,Mandatory=$true)]
      [string]$DatasourceID,
      [Parameter(Position=2,Mandatory=$false)]
      [string]$InstanceID
)

<# account info #>
$accessId = 'xxxxx'
$accessKey = 'xxxxx'
$company = 'xxxx'

<# request details #>
$httpVerb = 'GET'
if (!($InstanceID)) {$resourcePath = '/device/devices/' + $DeviceID + '/devicedatasources/' + $DatasourceID + '/data'}
if ($InstanceID) {$resourcePath = '/device/devices/' + $DeviceID + '/devicedatasources/' + $DatasourceID + '/instances/' + $InstanceID + '/data'}

<# Construct URL #>
$url = 'https://' + $company + '.logicmonitor.com/santaba/rest' + $resourcePath

<# Get current time in milliseconds #>
$epoch = [Math]::Round((New-TimeSpan -start (Get-Date -Date "1/1/1970") -end (Get-Date).ToUniversalTime()).TotalMilliseconds)

<# Concatenate Request Details #>
$requestVars = $httpVerb + $epoch + $resourcePath

<# Construct Signature #>
$hmac = New-Object System.Security.Cryptography.HMACSHA256
$hmac.Key = [Text.Encoding]::UTF8.GetBytes($accessKey)
$signatureBytes = $hmac.ComputeHash([Text.Encoding]::UTF8.GetBytes($requestVars))
$signatureHex = [System.BitConverter]::ToString($signatureBytes) -replace '-'
$signature = [System.Convert]::ToBase64String([System.Text.Encoding]::UTF8.GetBytes($signatureHex.ToLower()))

<# Construct Headers #>
$auth = 'LMv1 ' + $accessId + ':' + $signature + ':' + $epoch
$headers = New-Object "System.Collections.Generic.Dictionary[[String],[String]]"
$headers.Add("Authorization",$auth)
$headers.Add("Content-Type",'application/json')

<# Make Request #>
$response = Invoke-RestMethod -Uri $url -Method $httpVerb -Header $headers 

<# Print status and body of response #>
$body = $response.data
$data = @()
$a = 0
if ($InstanceID) {
    foreach ($iteration in $body.time) {
        $b= 0
        $epoch = [datetime]"1/1/1970"
        $date = $epoch.AddMilliseconds($iteration)
        foreach ($datapoint in $($body.values[$a])) {
            $detailprop = @{'value'=$datapoint;
                    'datapoint'=$($body.dataPoints[$b]);
                    'timestamp'=$date}
    
            $objectD = New-Object -TypeName PSObject -Prop $detailprop
            $data += $objectD
            $b++
            }   
        $a++
        }
    }
return $data

}#lm-getdatarest

#$output = lm-getdatarest "25991" "961171"

<# Example
datapoint       timestamp            value
---------       ---------            -----
average         6/14/2017 5:17:08 PM     0
maxrtt          6/14/2017 5:17:08 PM     0
minrtt          6/14/2017 5:17:08 PM     0
PingLossPercent 6/14/2017 5:17:08 PM     0
recvdpkts       6/14/2017 5:17:08 PM    10
sentpkts        6/14/2017 5:17:08 PM    10
#>

@Sarah Terry Actually using start and end timestamps does the job for me, thanks.

You can now specify a datapoints query parameter to limit the data returned in the request, like this: GET /device/devices/{id}/devicedatasources/{id}/instances/{id}/data?datapoints=datapoint1,datapoint5.  Does that satisfy both of your use cases?  

Documentation updated here: https://www.logicmonitor.com/support/rest-api-developers-guide/data/get-data/

@Mosh you should be able to already filter by epoch using the start and end parameters - is that what you meant by datapoint timestamp?

I'd like this too please, though I'd prefer to do it by DataPoint timestamp (epoch), which I think you already store in the history.

11 minutes ago, Sarah Terry said:

Hi Tom,

Thanks for posting - we should be able to add in a sub-resource for data specific to one datapoint, or at least offer a query parameter that restricts the datapoints data is returned for.  I'll get the ball rolling on that. 

Thanks,

Sarah

Great, thanks Sarah.