DataSource Dependencies
With the issues we've been having with collector resource exhaustion, I've been thinking about ways to reduce the amount of dataSources that run on the collectors at any given time. It occurs to me that if there is a host down, all of the datasources are still trying to run against it and having to await timeout before releasing their resources on the collector. I'd like to submit that a Host down status should issue a partial SDT for the device that would prevent all but the host status datasources from running against that device. The host status change could then remove the SDT once it has cleared. This would prevent devices in environments that spin them up / down based on load rather than a schedule from taking up too many collector system resources during their down time. It would also help alleviate strain during larger outages while still providing just the actionable information necessary to address the situation.