HTTP protocol version and HTTP(S) datasource

As we all know, LogicMonitor has a monitoring to check if web server is alive or responding in a device. It uses datasource called HTTP(S) with a nice short note here:

https://www.logicmonitor.com/support/datasources/data-collection-methods/webpage-httphttps-data-collection/

However, there is a catch :). and explanation is on the way....

 

I am running an NGINX (with SSL). This datasource is supposed to correctly monitor the secured web server but it is not and there is an error code 5 and based on the note in datasource it means network error. What kind of error is that? It will demand a closer look.

(

Firstly, such error usually will lead to some checks if the web server is running fine and with LogicMonitor Collector debug, there is a tool to do just that:

!http

 

In the debug window, it shows that the web server has no issue.

being paranoid .. or in fact, a standard procedure for any devops/sysadmin(?), checking the web server service rightfully to be done as well and it is totally up and running.

Further to that, querying the web server from the collector server shell is another evidence that it is all well with the web server.

1. Checking from collector debug?
2. checking the web server?
3. query from collector shell to the web server?

This was very confusing at first until a more detailed investigation was done.

 

HTTP protocol version is the centre of this blunder.

The datasource is sending a request with HTTP version 1.0, while the server does not accept it and it throws error every time. The web server can be configured to accept both the older protocol 1.0 (which is still widely used) and 1.1, nevertheless, some of them only accept 1.1

Collector debug command !http in LogicMonitor, does not specifically request with 1.0,  therefore there is no error when such debug is run. The result shows as evidence:

It also leads to the finding that the web server return the query in protocol version 1.1

If the query specifies which protocol to use such as in datasource HTTPS, the web server will cough up that error:

Server2000:~$ curl -k -I --http1.0 https://test-server_nginx
curl: (52) Empty reply from server

 

and when the request is specified with 1.1, the result is just what it should be:

Server2000:~$ curl -k -I --http1.1 https://test_server_nginx
HTTP/1.1 200 OK
Server: nginx/1.10.0 (Ubuntu)
Date: Sun, 13 Aug 2017 09:59:35 GMT
Content-Type: text/html
Content-Length: 6031
Last-Modified: Wed, 10 May 2017 11:51:16 GMT
Connection: keep-alive
ETag: "5912feb4-178f"
Strict-Transport-Security: max-age=63072000; includeSubdomains
X-Frame-Options: DENY
X-Content-Type-Options: nosniff
Accept-Ranges: bytes

 

Therefore the datasource needs to be modified to deal with web servers that do not like the query with the earlier version of 1.0, with the following on the request part:

GET / HTTP/1.1 \nUser-Agent:LogicMonitor\nConnection: close\n\n

hence the monitoring will be corrected:

 

As a note, there is a configuration in NGINX whereby it will strict the web server to only respond to 1.1 protocol and reject 1.0 which is the following:

if ($server_protocol ~* "HTTP/1.0") {
             return 444;
}

 

"444" is the code that will be provided when the query is using 1.0, which means:

444 No ResponseUsed

to indicate that the server has returned no information to the client and closed the connection.495 SSL Certificate Error

The error in LogicMonitor will correspond to that error configuration in the web server and in this case when protocol 1.0 is used, it returns code 5.

 

For IIS, the restriction will be in web.config as follows, which is using URL Rewrite module in IIS:

<rules>
   <rule name="rulename1" patternSyntax="Wildcard" stopProcessing="true">
      <match url="*" />
         <conditions>
            <add input="{SERVER_PROTOCOL}" pattern="HTTP/1.0" />
         </conditions>
      <action type="AbortRequest" />
   </rule>
</rules>
</rewrite>
</system.webServer>