REST API 503 errors
Hello everyone
We have been using various PowerShell scripts for years and have never had any problems with them. Now, since last week Monday, we are getting random 503 “server is busy” HTTP errors on some GET queries such as Devices, Device DataSources, Dashboards, etc.
There is no recognizable pattern to the errors.
Does this problem also occur for others using script automation via the REST API (occurs in V2 and V3)?
I have submitted a support request to LM, but progress is slow...
Have a nice day everyone
Dorian
Dorian Fuchs
Posted 2 years ago·Last reply 2 years ago
14 comments
Dorian Fuchs
OP2 years agoHello everyone
I have now received feedback from LM that on March 26, 2024 auto throttling for external API calls was activated and therefore the 503 HTTP errors are coming (nowhere mentioned in the releases note, etc.).
The feedback from LM on my question how we can handle such errors in the future was simply "script a catch and try again after a certain time".
What do you think about this? Is this our problem now if the API has resource bottlenecks? Is LM leaving us to our fate here? Should LM increase the resources of the API?
In my opinion, a 429 HTTP error would be the right "procedure".
1. for 429 errors, the person who makes a request can do something about it AND also do something against it.
2. in the case of 503 errors, the person making the request can usually do nothing about it if there is a high load on the API AND also do nothing against it.
Have a nice weekend soon
Dorian
LM User
·2 years agoAuto throttling?
What Joe_Williams posted earlier is what they mean by this.
A 429 would be the right thing to do as it explicitly means you've hit the rate limit. Sending back a 503 indicates that the server is having issues, not that there's anything wrong with our request.
Mike Moniz
·2 years agoLM has always had rate limiting and it's documented at https://www.logicmonitor.com/support/rest-api-developers-guide/overview/rest-api-rate-limit. Is this something different or is it no longer doing 429 anymore?
Joe Williams
·2 years agoI guess 1 question is, does your automation/scripts look at the headers to see your api limits? I don't know off the top of my head if when you hit your call limit, if it returns a 503, but you might be querying a little to fast and hitting the api limit.
LM User
·2 years agoI think that would be a 429, not a 503. 400-499 means you did something wrong. 500-599 means the server did something wrong.
Joe Williams
·2 years agoYeah, I wasn't sure what the status code would be as I have always coded around it. But this is LogicMonitor, I love it, use it, praise it, but it is LogicMonitor and logic doesn't always play a role ;)
Justin Dietz
·2 years agoi have several powershell scripts that run daily and no errors for me, the main ones backing up all my logicmodules
LM User
·2 years agoI run a lot of automation and haven't had any errors crop up.