API ingest from UltraDNS into Logic Monitor

D

I am attempting to ingest data from UltraDNS to create an alarm in LM and have spoken with their support but I am seeming to run into a dead end. I am trying to create an API call into LM that would populate an usage report. I am able to gather that data from UltraDNS but have no idea how to push that data into LogicMonitor. UDNS uses a bearer token for authentication but when I spoke with LM support, they say that only accept basic token authentication. Has anyone created an API call like this or can anyone offer any assistance? If I don't have to reinvent the wheel, I would really rather not have to do so. Thank you in advance.

 

 

26 replies

A

Sounds like a couple things are going on. Let's clarify some stuff:

1) You have API access to UltraDNS and you may have built a script that pulls that data through the API. Now you're wondering how to get that data into LM?  If so, the answer is DataSource and you wouldn't do anything with the LM API. So, the question about our API token is irrelevant.

2) You have API access to UltraDNS which has alerts and you may have built a script that pulls those alerts? If so, the answer is an EventSource. Let's tackle these one at a time, because if you get the DataSource working, the EventSource will be easy.

D

Stuart, that may be the case. I'm fairly new to doing this in LM and was kind of given a quick run thru a few days ago. for the DataSource, as of this writing, all we can do is to ping the URL of where I am needing to pull data from. We found a groovy script to try to run that supposedly will GET the data, but it doesnt seem to work. Not sure exactly what the issue is. 

A

Ok, so the first thing I'd do is figure out how to get a script working that will pull the data and output it using print statements. That doesn't have to be groovy, it can be powershell if you're running windows collectors, or it can be any other language (like python) as long as the Collector has that language installed. It's pretty easy to use python scripts on your collector if you're running Linux Collectors. A little trickier on Windows, but not impossible.

Depending on the type of data you're wanting to fetch, it maybe single instance or multi-instance, script or batch script. All of that is secondary to actually having a script that will fetch the data and print it to the screen.

D

Stuart,

The script that we have is built around groovy, but is just something that is pieced together.  

// instantiate an http client object for the target system
ip="api.ultradns.com"
httpClient = HTTP.open(ip, 443);

user = hostProps.get("ultradns.user");
pass = hostProps.get("ultradns.pass");

// use an authentication API call to initiate a session
// specify the url to which we want to post
url = "https://"+ip+"/authorization/token";
def payload = '{"username":"myusername","typeId":"com.tintri.api.rest.vcommon.dto.rbac.
    RestApiCredentials","password":"mypassword"}';
 
// do the post
def postResponse = httpClient.post(url, payload,["Content-Type":"application/json"]);
// does the response indicate a successful authentication?
if ( !(httpClient.getStatusCode() =~ /200/) ) 
{
    // no -- report an error, and return a non-zero exit code
    println "authentication failure";
    return(1);
}
// we are now authenticated. Subsequent GETs with the httpClient will pass in the session cookie 
url="https://"+ip+"/reports/dns/usage_summary";
def getResponse=httpClient.get(url);
// print some data
println httpClient.getResponseBody();

With the params ultradns.user and ultradns.pass being values I pass into the script. I can show you what we have written on the postman side that is calling the data from Ultra that works if needed. Again, I apologize for being a novice at this but this is my first time working with an API into LogicMonitor. 

A

This is great progress. So, now you need to figure out how your data is going to be organized on the LM side. Will you be gathering stats about the resource itself? That might be the case, since you're going to be doing a get request to /reports/dns/usage_summary after getting authenticated. What does that API endpoint return? This will determine if your DataSource will be single instance or multi-instance. Let's look at some examples:

single instance: ping, RAM utilization, system uptime

multi-instance: hard disks, network interfaces, CPU cores

I don't really know what it is that you're trying to monitor, so I can't answer this for you. Once you answer this question, it will determine next steps.

D

 

Stuart,

What we are attempting to monitor is the amount of DNS queries we receive per month. The output of that report shows us rough numbers, but we start to incur costs after 65 million. We're trying to set up an alert that notifies us when we get to approx 62 million so that will be our threshold. The code I sent above presents me with all kinds of errors when I'm building it though so I'm not sure what I'm missing. If you'd like I can post you a screen shot of the errors I get when I try to test that code in LM.

A

Good idea. You can DM me the output if it's sensitive data.

D 
The embed groovy script syntax is illegal - startup failed:
Script303.groovy: 11: expecting ''', found '\r' @ line 11, column 88.
   tri.api.rest.vcommon.dto.rbac.
                                 ^

1 error

org.codehaus.groovy.control.MultipleCompilationErrorsException: startup failed:
Script303.groovy: 11: expecting ''', found '\r' @ line 11, column 88.
   tri.api.rest.vcommon.dto.rbac.
                                 ^

1 error

	at org.codehaus.groovy.control.ErrorCollector.failIfErrors(ErrorCollector.java:310)
	at org.codehaus.groovy.control.ErrorCollector.addFatalError(ErrorCollector.java:150)
	at org.codehaus.groovy.control.ErrorCollector.addError(ErrorCollector.java:120)
	at org.codehaus.groovy.control.ErrorCollector.addError(ErrorCollector.java:132)
	at org.codehaus.groovy.control.SourceUnit.addError(SourceUnit.java:349)
	at org.codehaus.groovy.antlr.AntlrParserPlugin.transformCSTIntoAST(AntlrParserPlugin.java:220)
	at org.codehaus.groovy.antlr.AntlrParserPlugin.parseCST(AntlrParserPlugin.java:191)
	at org.codehaus.groovy.control.SourceUnit.parse(SourceUnit.java:233)
	at org.codehaus.groovy.control.CompilationUnit$1.call(CompilationUnit.java:189)
	at org.codehaus.groovy.control.CompilationUnit.applyToSourceUnits(CompilationUnit.java:966)
	at org.codehaus.groovy.control.CompilationUnit.doPhaseOperation(CompilationUnit.java:626)
	at org.codehaus.groovy.control.CompilationUnit.processPhaseOperations(CompilationUnit.java:602)
	at org.codehaus.groovy.control.CompilationUnit.compile(CompilationUnit.java:579)
	at groovy.lang.GroovyClassLoader.doParseClass(GroovyClassLoader.java:323)
	at groovy.lang.GroovyClassLoader.parseClass(GroovyClassLoader.java:293)
	at groovy.lang.GroovyShell.parseClass(GroovyShell.java:677)
	at groovy.lang.GroovyShell.parse(GroovyShell.java:689)
	at groovy.lang.GroovyShell.parse(GroovyShell.java:725)
	at groovy.lang.GroovyShell.parse(GroovyShell.java:716)

Stuart,

Above are the error messages we receive when I run the test script. Thank you again for your assistance.

 

A

It would appear that groovy doesn't like that string being split across two different lines.

D

Ah, I see. I fixed that issue and now get this error: 

The script failed, elapsed time: 0 seconds - No such property: HTTP for class: Script2340
groovy.lang.MissingPropertyException: No such property: HTTP for class: Script2340
	at org.codehaus.groovy.runtime.ScriptBytecodeAdapter.unwrap(ScriptBytecodeAdapter.java:66)
	at org.codehaus.groovy.runtime.callsite.PogoGetPropertySite.getProperty(PogoGetPropertySite.java:51)
	at org.codehaus.groovy.runtime.callsite.AbstractCallSite.callGroovyObjectGetProperty(AbstractCallSite.java:310)
	at Script2340.run(Script2340.groovy:3)

A

Did you  

import com.santaba.agent.groovyapi.http.*;

at the beginning of your script? Looks like it doesn't like that you're trying to create an "HTTP.open()" object without importing that library. Is the above code the entirety of your script?

D

Ok I fixed that but now I'm getting an auth error. I verified my user name and password and those both work, but I believe that Ultra is looking for a bearer token to pass to it as part of the auth process.  

This is what I have to pass into Ultra as part of my auth process. This is my POST to them.  

"refresh_token": "xxxxxxxx",
    "access_token": "xxxxxxxxx",

Info obscured for security reasons.


A

Alright, it make sense for them not to allow basic authentication on their api (basic auth means username & password). You'll need to see if they have any details of how to authenticate with their API. Like, what needs to be in the headers, besides what you already have.

D

Stuart,

in my GET from them, I am passing a bearer token along with the param for our account name. Here's how I've updated my script: 

import com.santaba.agent.groovyapi.http.*;

// instantiate an http client object for the target system
ip="api.ultradns.com"
httpClient = HTTP.open(ip, 443);

user = hostProps.get("ultradns.user");
pass = hostProps.get("ultradns.pass");
account = hostProps.get("ultradns.accountName");

// use an authentication API call to initiate a session
// specify the url to which we want to post
url = "https://"+ip+"/authorization/token";
def payload = '{"username":"myusername","typeId":"com.tintri.api.rest.vcommon.dto.rbac.RestApiCredentials","password":"mypassword"}';
 
// do the post
def postResponse = httpClient.post(url, payload,["Content-Type":"application/json"]);
// does the response indicate a successful authentication?
if ( !(httpClient.getStatusCode() =~ /200/) ) 
{
    // no -- report an error, and return a non-zero exit code
    println "authentication failure";
    return(1);
}
// we are now authenticated. Subsequent GETs with the httpClient will pass in the session cookie 
url="https://"+ip+"/reports/dns/usage_summary";
def getResponse=httpClient.get(url);
// print some data
println httpClient.getResponseBody();


A

shouldn't the payload look like this? 

def payload = '{"username":user,"typeId":"com.tintri.api.rest.vcommon.dto.rbac.RestApiCredentials","password":pass}';

 

D

I tried that with that same auth failure. 

A

Are you running this in the Collector debug? If so, you can add a println statement after you define the payload to output it to make sure that everything is set properly in the payload.

D

No because this isn't a collector. I've reviewed the documentation on how to set up a collector for debug purposes and in this case, it may not apply to what I'm trying to do. 

A

Without a Collector, you won't be able to poll the data from UltraDNS into LogicMonitor. It's where this script should be running. The hostProps.get() method won't run on anything that's not a Collector. You want to be designing this script to run on the Collector so that you can get the data into LM and alert on it. Wasn't that the goal?

Michael Rodrigues
Rank
Userlevel 3
Badge +4

Your payload should probably actually look like this: 

"parameter=value&also=another"

With your keys and values substituted, of course.


 

D

The problem here being that I need to be able to gather this information without a collector. UltraDNS is expecting me to be able to pull this information from an API call, which I can do. I want to be able to ingest that information into LM with that call via my API and have LM ingest that data, but it appears that the methods of auth may or may not work between the two end points. 

Michael, 

For the payload, the parameter values would be my auth token I'm assuming? They do have an expiry on the initial bearer token but we do have the ability to supply a refresh token that extends beyond the expiry of the initial bearer token. 

Michael Rodrigues
Rank
Userlevel 3
Badge +4

payload = "username=literal_username&typeId=com.tintri.api.rest.vcommon.dto.rbac.RestApiCredentials&password=literal_pass"


From your description, I still can't figure out why/how you intend to do this without a collector.

The com.santaba.agent.groovyapi.http.* classes are provided by LogicMonitor and may be hard to get working outside of the collector.

A

The way you would "ingest" the data is to have that script run on a Collector. If running the collector is a problem, you could always run it in a container in the cloud. 

There are several advantages to this vs. trying to use the API to ingest the data:

1) your UltraDNS credentials can be safely stored in LogicMonitor without hard coding them into your script. 

2) Once you get authenticated to UltraDNS' API, you'd only need to output the data in a specific format, which isn't hard to do.

The authentication to UltraDNS' API would be the same whether your running it on the Collector or on your laptop or on another server you might be building. As @Michael Rodrigues mentioned, the Santaba http classes won't be easy to get working on another system, so you may as well develop it 

Once you get it working, you'd:

1) Add the UltraDNS API as a resource in LogicMonitor. Simply go to your resources page and click Add >> One Device >> Expert. 

a) Use "api.ultradns.com" as the "IP Address/DNS name"
b) Give it whatever Name you'd like it to display as. "UltraDNS API" would work fine.
c) Choose the Collector that will monitor it
d) Add it into any static group (optional)
e) Add a property called "system.categories" with a value of "NoPing" (optional, this will disable ping checks on the API)
f) Add a property called "ultradns.user" with the value of whatever your username is
g) Add a property called "ultradns.pass" with the value of whatever your password is
h) Add a proeperty called "ultradns.accountName" with the value of your UltraDNS account name

 

2) Create the UltraDNS DataSource. Go to Settings >> DataSources and click the Add >> DataSource option

a) Choose an appropriate Name, Displayed as, Description
b) Put in the Technical notes information about how this depends on the properties mentioned above existing on the device
c) In the AppliesTo, put "ultradns.accountName && ultradns.pass && ultradns.user". You could experiment with other expressions, but this one is the simplest.
d) Choose "Script" for the collector and choose your desired poll rate.
e) Modify your script, replacing the literal username and password with the hostProp.get() calls you have in the first posted copy of your script. This is what will bring the property values into the script at runtime.
f) Make sure your script output matches LM's scripted output format.
g) Paste your script into the "Groovy Script"
h) For each datapoint of your output, create a datapoint with the corresponding key
i) You can test your script at this point, but it should work the same as it did on your laptop.

I'm available to guide you through that process, but using a Collector is the way.

D

Ok so the first issue I have to overcome is that I have to perform a POST for the bearer token to a URL that I can provide via DM. Then once I get that POST response, that is my token (along with my userID, pwd and acct# that I have to pass as part of my GET (or can be in the body of the request, whichever way works best.) If I can get that to work, then the ingest of the data shouldnt be a huge issue. I hope that makes sense. UltraDNS doesn't just have a simple auth process when it comes to their API. I wish it were a simple un/pwd combo, but it's MFA, so they need the bearer token (or in our case ... a refresh Token) to pass to them for auth. 

A

Yeah, looking at their example Python, it looks like you need to post to "/v1/authorization/token" with your username, password, and "grant_type", which should be set to "password". It looks like it returns an access token and a refresh token. So, you'll need to grab the access_token it returns because you'll need it in subsequent calls.

After that, it looks like you can do a GET with the headers "Accept":"application/json" and "Authorization":"Bearer THE_BEARER_TOKEN_FROM_THE_POST".

Looks like it's possible they may send back an error even with a 200. If they do, it appears to mean that you need to do another post to /v1/authorization/token using the refresh token, which will give you a new access_token along with a new refresh_token. It's likely this won't ever happen because you will likely be doing a single GET right after obtaining the original access_token, i.e. before it expires. 

Reply


Terms & ConditionsCookie settings