Listing monitored websites via SDK

The SDK documentation is light on what the domain field is, so reverting back to the Swagger docs, it looks like domain represents: "The domain of the service. This is the base URL of the service". The UI won't let you add or edit a website to remove that value, so i'm not sure why you'd be seeing that. 

What do you get when you do this?

len(api_response.items)

FWIW, using the SDK, i got 17 items under the 'total' object, and 17 items with a non-null domain field in the 'items' object. And i have 17 websites configured in my portal.

You might consider altering your actual call to bump up the size:

api_response = api_instance.get_website_list(size=1000)

Saying the documentation is light on what Domain means is an understatement.  Its light on any documentation.  I was just picking on the field Domain because all our websites will have a domain.  

I did increase Size (what ever that is supposed to mean) to some crazy number like 9999 and got what looks like all the Sites.  I can only guess that Size is the number of Websites per Subgroup.  I say that because if Size = (some number under 100) I will get back all the values of the Subgroups that only have one or 2 Sites in them and then a few from the our largest group which has over 100 Sites in it.

I cannot for the life of me figure out what Offset does.

As far as Field and Filter goes I have tried putting things in there but I only get back python errors for what ever I do.

Stuart, 

Try this.  Create a tree of groups and subgroups.  Put one website in each group/subgroup except one subgroup put 100 sites.  Then set offset=0 and size=0.  

If what I am seeing is correct what you will get back is all the contents of the groups/subgroups where you only have one site.  For the subgroup where you have 100 sites you will get only a few back (maybe half).

Now with the same configuration change size=1000 and you should get call the sites back across all the groups/subgroups.

Same config and leave size=1000 now start playing around with offset.  Try offset=10.  If offset and size here in this V2 request are the same as in the V1 request then offset=10 should skip past all the groups/subgroups with less then 10 sites.  But that did not happen for me.

19 hours ago, Stuart Weenig said:

The SDK documentation is light on what the domain field is, so reverting back to the Swagger docs, it looks like domain represents: "The domain of the service. This is the base URL of the service". The UI won't let you add or edit a website to remove that value, so i'm not sure why you'd be seeing that. 

What do you get when you do this?

len(api_response.items)

 

If collectorIds = '' # String |  (optional)
   fields = '' # String |  (optional)
   size = 5000 # Integer |  (optional)
   offset = 0 # Integer |  (optional)
   filter = '' # String |  (optional)

print(len(api_response.items)) returns 132 which just looking at the data seems right.

I've had mixed results with field and filter through the SDK. Works fine when accessing the API directly, but something about the SDK doesn't quite handle it right.

As to size and offset, they are related. Size is max number of items to fetch (regardless of group), and it defaults to 50. When you were calling without size, you were asking the API to get you the first 50 items. I've also had mixed results with the API concerning the "total" number returned not matching the actual number returned. This is why I asked what `len(api_response.items)`. That's the real way I know how many items I got back, since I can't trust the "total" number.

All of this has to do with pagination, which is pretty common when dealing with APIs. The API will only let you fetch a certain number of items at a time. This prevents a single request for 9,999,999 items from bogging down the system. LM's API does the same thing, defaulting to only returning the first 50 items unless you specify a size, which can be as high as 1000. So, when you specify a size of 1000, you will get a max of 1000 items back. If you only have 133, you'll get all 133. 

But what happens if you have 1500 items? That's where offset comes in. First of all, you need to recognize that you'll have to make multiple calls to the API, since you can fetch 1000 items max per request. For 1500 items, you'll need to do 2 requests. The first request would have a size of 1000 and an offset of 0 (the default for offset). The second request needs to specify a size of 1000. An offset of 1000 would tell LM to return not the first 1000 items, but the second 1000 items. 

Think of it this way: size=1000, offset=0 --> returns up to 1000 items starting with the 0th item. size=1000, offset=1000 --> returns up to 1000 items starting with the 1000th item. Size=1000, offset=2000 would return 1000 items starting with the 2000th item.

Example code of how to do pagination coming up...

Script that ignores pagination (problematic):

from __future__ import print_function
import time
import logicmonitor_sdk
from pprint import pprint
# Configure API key authorization: LMv1
creds = logicmonitor_sdk.Configuration()
creds.company = 'acmecorp'
creds.access_id = 'FM8L8nRHjx7YxBS3I7XD'
creds.access_key = 'xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx'
# create an instance of the API class
api = logicmonitor_sdk.LMApi(logicmonitor_sdk.ApiClient(creds))
try:
    # get alert list
    api_response = api.get_alert_list()
    pprint(api_response)
except logicmonitor_sdk.rest.ApiException as e:
    print("Exception when calling LMApi->getAlertList: %s\n" % e)

Script that uses pagination (will always fetch all items):

from __future__ import print_function
import time
import logicmonitor_sdk
from pprint import pprint
# Configure API key authorization: LMv1
creds = logicmonitor_sdk.Configuration()
creds.company = 'acmecorp'
creds.access_id = 'FM8L8nRHjx7YxBS3I7XD'
creds.access_key = 'xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx'
# create an instance of the API class
api = logicmonitor_sdk.LMApi(logicmonitor_sdk.ApiClient(creds))
try:
  end_found = False    # we'll set this to True once we have fetched the last page
  offset = 0           # which page to start at
  size = 50            # how many records to fetch per page
  results = {}         # dict to contain our results
  # start a loop that won't end until we've gotten all the records
  while not end_found:
    # fetch the data
    data = api.get_alert_list(size=size, offset=offset)
    # add the items just fetched to the main dictionary
    results.update({record.id : record for record in data.items})
    # figure out the next page to fetch
    offset += size
    # if we fetched fewer items than a full page, we've reached the end
    end_found = len(data.items) < size

  for k,v in results.items():
    print(f"{v.monitor_object_name}-{v.instance_name}-{v.data_point_name}")

5 minutes ago, systemgeek said:

print(len(api_response.items)) returns 132 which just looking at the data seems right.

Ok, that jives. With size=1000 (which is the max, your 5000 size is getting capped at 1000) and offset=0, you are requesting 1000 items starting with the 0th item, which gets you all 132 websites. Group membership doesn't play into what is coming back and what isn't. Group membership is indicated in the results, but it doesn't drive what is returned by the API.

Nice.  This is making more sense now.  

As for Fields and Filter not working across the SDK that makes little sense unless it means that LM puts out an API they do not use themselves.  Which to me is kinda stupid.  I am dealing with another vendor over their API and they are working very hard to fix it since they use the API on their site.

What ever...  Good to know I am not crazy and/or doing it wrong.

Thank you very much for clearing this up for me.