Generic HTTP(S) resource type discovery (template) | Product Discussions

Antony Hawkins · 2022-02-17T10:24:40Z

Detecting an SNMP-based resource type is easy enough - look for the sysoid, that'll tell you the vendor and sometimes a specific product family. However, the ever-increasing number of resource types that use an API or other approaches for monitoring means that all demand different credential properties. How then do you auto-detect a resource type amongst these things, to be able to categorise them and, if the correct credentials aren't present, flag these gaps? In some cases a PropertySource may apply only to resources with e.g. 'sometype.api.user' and 'sometype.api.pass', which minimises any waste of collector effort in running Active Discovery in DataSources, but still demands you set the right credentials for a resource before we'll check whether that resource needs those credentials. Hmm... However... even without credentials, a large number of resource types have an http(s) response that contains tell-tales as to what that thing is - sometimes even if it gives a non-200 code without credentials. Here then is a "universal" (not really) resource identifier that tries an unauthenticated http call (which will follow most redirects) and looks for relevant data in the headers, or page title, or page content. Note: This should be considered a template that is to be tailored per-technology. Do not use it as a universal detector, because it'll get complex and the risk of mis-identification increases with more complexity. For example, if you were to check the response headers for "Apache" and assume that a match means you've found a production web server, you'll mis-identify all those resource types that use a version of Apache to serve their admin page. What it does in template form: It makes an http call and captures the result, then inspects the content of the page title of the response for tell-tale content (that you define). If it finds no matches to whichever regexes you've defined, it looks in the headers similarly; failure here and it searches the full page content. Therefore, my suggestion is to clone this PropSource for each use case, and by examining the content of an !http debug call (use !http http://<hostname>/ initially, and evolve if necessary from there) determine whether any of the methods (header, title, or body content comparison) provide suitable tell-tales to reliably identify the resource type. All you have to do is define a suitable regex and what that should produce as a system.categories field value in the appropriate map, e.g.: // Can we identify from within the <title> field? def titleRegexesMap = [ /APIC/ : 'CiscoAPIC', /Pure\s*Storage/ : 'PureStorage', /Veritas\s+NetBackup/ : 'VeritasNetBackup', ]; // Can we identify from within the headers? // Note, will only be attempted if <title> look-ups fail // CARE! Note for example that a lot of different things might run Apache web server for an admin page. // Apache therefore (using /Server:\s*Apache/) would be a poor example to attempt. def headersRegexesMap = [ /ReadyNAS/ : 'ReadyNAS', /tandberg/ : 'Cisco_Expressway', ]; // Can we identify from rest of the body? // Note, will only be attempted if <title> and headers look-ups fail, as the full html content // might contain all sorts of content that could match. // This could be expensive. def bodyRegexesMap = [ /www\.nexsan\.com/ : 'Nexsan', /SolidFire/ : 'Solidfire', ]; In the try{} block of the code you'll find a block for each method; if you know you're only testing for the page title, you can delete or comment out the other two maps and other two code blocks for headers and body. This will make more sense when you see it. Honest. Clearly also from this point, you don't *have* to write to system.categories if you're in doubt, you could write to a new auto.property if that's preferable. You could equally use outputs to indicate "probably technology X"; you can further use this detection to apply troubleshooter DataSources highlighting an absence of credentials, further PropertySources to find more data about the resources, etc. PropertySource: http_discovery_template v1.4 Published with lmLocator: FTNZX2

MoreDiscussions

Generic HTTP(S) resource type discovery (template) | Product Discussions | LogicMonitor Community