Cisco Switch Backups Missing Lines of Configuration (Common ConfigSources)
Good Afternoon,
Has anyone encountered backup configurations of their switches in LogicMonitor missing lines of configuration? I've noticed this with several models of Cisco Catalyst switches. As a MSP, we monitor resources for several organizations, and thus far, I've only seen this behavior for a single organization, but the common config DataGroup is globally applied, and the Dynamic (SSH Interactive) and Standard (Interactive) DataSources have not been customized at any level. If I compare the show running configuration output when directly SSH'd into a switch, vs the running configuration output from a backup config file pulled from LM, the LM config is missing multiple lines of configuration. It appears that it's always the same lines of missing configuration when i download the backup config files. They don't seem to share any commonality (missing vlan on one line, a missing name for another vlan on another line, a missing port interface, but the dependent port configuration is present. The line position is present but it's always a blank space. Hope this makes sense, and any suggestions or assistance is greatly appreciated. Thanks!
Ernest
ErnestG72
OP9 months agoFinally discovered the root cause of this issue. We noticed that only one Managed Services customer was having this issue, so we started focusing in on what was different with their devices. They had recently migrated to ISE and 802.1x authentication and had created a new user account for the Collector. Apparently, the User did not have the necessary privileges, and a couple of those unauthorized commands were related to pagination and output, which affected the layout of the running-configuration in our backup file. Once this was edited and our LM Collector User in ISE was updated, the issue was resolved.
Dave Lee
·10 months agoSome lines do get purposefully excluded. You will see which in the auto.config.filter property on the device. This is populated by a Property Source that is specific to the device type. It is then picked up by the Common Config Source which excludes any lines that match the regex in auto.config.filter. I think it actually does a find/replace (putting a blank in place of anything that matches) which might explain your blank spaces/lines.
Usually it's for removing things that the CLI might output that you wouldn't want to store, but perhaps there is something that's excluding lines that you do want to store?
for example, this is what is in auto.config.filter for one of the cisco devices in my LM instance.
?mi)^\s*(\^|\^C|DONE|ERROR:.*|WARNING:.*|CMD EXEC ERROR.*|NO SUCH COMMAND.*|BAD PARAMETER STARTING.*|PRESS ENTER TO CONTINUE.*|BUILDING CONFIGURATION.*|<?\s*-+\s*MORE\s*(-+|or \(?q\)?uit)\s*>?.*|CONFIG GENERATION MAY TAKE SOME TIME.*|STARTUP CONFIGURATION IS CURRENTLY BEING ACCESSED BY ANOTHER SESSION.*|SYNTAX ERROR WHILE PARSING.*|THE FLASH DEVICE IS IN USE BY ANOTHER TASK.*|LINE HAS INVALID AUTOCOMMAND.*|LOGGING YOU IN AUTOMATICALLY.*|RETRIEVING USER PRIVILEGES.*|SYSTEM MAY NOT BE READY.*|INCORRECT (USAGE|COMMAND).*|INVALID (COMMAND|INPUT.*)|% INCOMPLETE COMMAND.*|% INVALID (COMMAND|INPUT DETECTED) AT '\^' MARKER.*|% UNKNOWN COMMAND OR COMPUTER NAME, OR UNABLE TO FIND COMPUTER ADDRESS.*|% WRONG NUMBER OF PARAMETERS OR INVALID RANGE.*|% PERMISSION DENIED FOR THE ROLE|% UNRECOGNIZED COMMAND|% ERROR IN AUTHENTICATION|% ACCESS DENIED|%+ NON-VOLATILE CONFIGURATION MEMORY IS BEING ACCESSED, TRY AGAIN LATER|PASSWORD:|PRESS RETURN TO GET STARTED.*|.*TTY\d IS NOW AVAILABLE|ENABLE 'DEBUG DISK FILESYSTEM' FOR DETAILS\.|THIS COMMAND IS ONLY APPLICABLE IN DUAL STAGE MODE|THIS COMMAND IS BEING DEPRECATED.*|UNKNOWN SCOPE.*|NO CONFIGURATION|ATTEMPTING TO AUTHORIZE.*|/BIN/BASH:.*COMMAND NOT FOUND|! Last configuration change at.*|! NVRAM config last updated at.*)\s*$
Joe Williams
·11 months agoHave you verified it isn't something like you need to do "show system:running-config" versus "show running-config" ?
Mike Moniz
·11 months agoThat sounds odd. If it's always the same lines I wonder if there might be a specific character (may not be visible) that the groovy script is failing on, or if the lines are too long or something like that.
The Common ConfigSource page has a troubleshoot section that might be worth reviewing. Especially items about paging (perhaps it's getting lost in the page boundaries?) and there is a section specifically for a type of Cisco device I'm not familiar with.