Ping Failing from collector to device and back?

41 minutes ago, Michael Rodrigues said:

Collector PM here. Quick glance at tickets suggests this ICMP firewall issue realization never made it to the collector team. I do recall the similar issue with SNMP.

I'll bring this up with the collector team tomorrow night. While best not to ping through a FW, if all we have to do to fix it is randomize the ICMP ID, that seems like a reasonable ask. I'm finding old TS tickets that appear to be this same issue, whether it was recognized at the time or not.


Sarah Terry is still here, by the way, she is Senior Director of Product these days.

Thank you for jumping in!  I did not have any idea where to acquire a Sumo suit :).

Still working on this, hoping to have a better update soon.

On 4/1/2022 at 3:18 PM, Mike Moniz said:

Interesting, I haven't run into that yet myself. I typically have collectors located on the same network segment as the devices being monitored so not hitting firewalls, but some situations do go thru some. Is this specific to Windows or Linux Collectors?

We've already faced this internally with some customers too (where having 1 collector per subnet is not feasible - due to the size, licensing, etc...).
If I'm not mistaken the OS is irrelevant to this issue (since it's the way LM is working currently).

Any further updates here?

I'm also facing the same issue for Palo Alto FW.

Any updates on this? ?

I would not hold my breath.  I pushed my CSM at the time on this issue back in 2018 and they refused to consider it a bug, but a feature request.  I also brought that up recently with my current CSM and I got crickets.  I dutifully followed through, but since the feature request "system" is nearly worthless, nothing has been done.  I cannot begin to enumerate the number of embarrassing conversations with clients that start like "Why is LogicMonitor alarming about SNMP being down or a host being down when we can get to those devices just fine?"  The workaround is time-intensive (manual collector restarts) and the repeated data loss is unforgivable.  I don't know what possessed the developers to generate a fixed SNMP session ID or ICMP ID once when the collector starts rather than at each new get/walk or ping.  It is the ultimate in false optimization and causes unending problems for any but the smallest simplest network.  LM should be ashamed of letting this continuing to happen.

I looks like I’m seeing the same issues now with some customers where ping stops working until the collector is restarted. So add me to the list of affected people.

We have observered similar issues with Ping failing via the collector application but not from the OS it self. A restart would resolve the issue for awhile. What we found was adjusting the threadpool for Ping to fix our problem. It seems the collector would max out of threads for ping and then just start failing. After adjusting the threadpool count we didn’t encounter this issue again.

I suppose it is possible there are two different issues -- threadpool change requirements needed indicates internal resources are exhausted for checks.  The original issue is still our constant problem -- static ID values cause sessions to become invalidated by firewalls when there is a disruption on the firewall path.  A new session must be observed by the firewall to start letting traffic through again, which can only be done (currently) by restarting the collector since the code allocates those session ID values only one at startup. LM has been informed about this repeatedly, is aware of it and does nothing to fix it.  The only thing I’ve seen is a doc note blaming particular firewalls, but it impacts pretty much any stateful inspection firewall. More and more folks use firewalls for internal segmentation and there are many cases where a remote collector is needed due to lack of resources to deploy a local collector.

For us it was a careful dance of watching the graphs and the debug commands. I can’t tell you what to adjust yours to as each collector is unique.