Palo Alto application data missing from Netflow
We havebeen able to get Netflow data working for a Palo Alto PA-820 firewall, but we are not seeing the application data show up. Does anyone have any suggestions on next steps we could take? Here is what has been done so far: Netflow profile has been configured on the Palo Alto side and assigned to the interface, including selecting the PAN-OS Field Types to get the App-ID and User-ID (https://docs.paloaltonetworks.com/pan-os/11-0/pan-os-admin/monitoring/netflow-monitoring/configure-netflow-exports) nbarhas been enabled on the collector: # enable netflow support for NBAR, IPV6 and Multicast fields netflow.nbar.enabled=true # enable netflow support for IPV6 fields netflow.ipv6.enabled=true Collector version is 34.003 We’re seeing everything we expect except the app & systemsdata on the Traffic tab for the device: Any thoughts on what we might be missing? Thank you. :-)
Netflow Alerting Rules
Not only restricting you to visualising the Netflow data on LM Platform. Interestingly, the most recent improvement to LogicMonitor Netflow is the Traffic Alerting Rules. It is possible to set up traffic alert rules for the NetFlow resources to get alerts when a resource's traffic hits a specific threshold, drops off for a specified length of time, etc. Traffic Alerting Rules feature are available and you can create rules at: Traffic Alert Rule at Group Level Traffic Alert Rule at Resource Level Don't missout on the advantages of this feature and refer the below link for more details. https://www.logicmonitor.com/support/traffic-alert-rule
PSA: LM wipes good known properties when unknown results occur
I have recently found that due to the excellent programming skills in the dev team that properties that have previously been autodiscovered can be wiped out when ephemeral issues produce unknown (no data) results. A good example is system.ips -- if the data has been scanned properly in the past and a blip occurs with no data, the previous values get overwritten with just the configured IP of the device. That leads to various fun side effects like NetFlow data not being matched to the device. To make things worse, the “no data” result does not set an internal flag to run a new AD scan earlier and you have to wait up to 24 hours for a regularly scheduled scan. I created a bug ticket requesting they set that flag and run a new scan as soon as possible, but was basically told to pound sand. My workaround was to use an undocumented API endpoint to trigger on specified devices so I stop losing NetFlow data and I scheduled it hourly. The “solution” I was given was to add a netflow property to hardcode the needed IP address for each device -- works, but it is a brittle fix and leads to undesirable manual property management. Beyond that, this issue affects more than NetFlow, that was just the problem that lead me to realize what was happening. Other properties routinely get messed up that could affect processing. This class of problem (replacing good data with unknown data) frequently occurs in modules as well -- for example, a lot of the Powershell configsource modules lack sufficient error checking and unknown results replace previously known good results, leading to change thrashing. Or they often forget to sort/normalize output leading to similar effects. The good news on those is they usually (eventually) listen to me. Anyone who wants to use my workaround can use this script (or at least the central logic if you prefer something other than Perl). I still lose data, but the window is smaller. https://github.com/willingminds/lmapi-scripts/blob/master/lm-actionNet flow / Please increase the filter capabilities
The current net flow reporting capabilities are very limited, even if the required net flow data will be delivered by the end devices.I.e.I'm trying to report all traffic for udp port 53 pointing to a specificdevice; or I'd like to combinesearch criteria, likea specific source and destination IP addresses using aspecific port. All this information will be exported by the end devices to the net flow collector, but it's not possible to report or filter on this. Why not? This option would be very useful foranalysingspecific traffic between two locations, systems,etc. Today, wejust get the top ten talkers and the top 20 flows... I.e. if roughly 60% of the traffic is outlined as "others" you'd like to get a chance to dive into this part of the overall traffic, correct?
Export Netflow from Linux to LogicMonitor
Exporting Netflow from Linux with softflowd NetFlow is an industry standard network protocol for monitoring traffic flows across a network interface. It is used most commonly by devices like firewalls, routers, and switches, but some software packages make it possible to export Netflow data from a server operating system - in this caseLinux (withsoftflowd) - toa Netflow collector (LogicMonitor)for traffic analysis. Ubuntu Documentation here:http://manpages.ubuntu.com/manpages/xenial/man8/softflowd.8.html The following assumes you have an Ubuntu device in your portal which you can access with sudoer permissions. It also assumes Netflow has been enabled for the device and the collector in question. Install softflowd: sudo apt-get install softflowd Open /etc/default/softflowd for editing: sudo nano /etc/default/softflowd Set the value for INTERFACE and add the destination ip:port (<collectorIP>:2055) under OPTIONS. Other options are available, check the link above for full documentation. # # configuration for softflowd # # note: softflowd will not start without an interface configured. # The interface softflowd listens on. You may also use "any" to listen # on all interfaces. INTERFACE= "eth0" # Further options for softflowd, see "man softflowd" for details. # You should at least define a host and a port where the accounting # datagrams should be sent to, e.g. # OPTIONS="-n 127.0.0.1:9995" OPTIONS= "-n 192.168.170.130:2055" Save your changes by pressing Ctrl-O , then exit nano by pressing Ctrl-X . Restart softflowd. sudo service softflowd restart Add a rule to the firewall to allow traffic on 2055. sudo ufw allow 2055 CentOs This is a bit more work since you can't just install a package; you'll need to download the source and compile. Most of the information here comes fromhttps://www.scribd.com/doc/199440303/Cacti-Netflow-Collector-Flowview-and-Softflowd More good info:https://thwack.solarwinds.com/thread/59620 Check to see if you have the compiler installed. which gcc If you don't get /usr/bin/gcc as the response, you'll need to install it. sudo yum install gcc Install libpcap-devel (you'll need this to compile softflowd). sudo yum install libpcap-devel Download the softflowd source. wget https: //storage .googleapis.com /google-code-archive-downloads/v2/code .google.com /softflowd/softflowd-0 .9.9. tar .gz Make sure you're in the directory where you saved the download, then untar the dowloaded source files. tar -xzvf softflowd-0.9.9. tar .gz Switch to the softflowd directory, then run the commands to compile and install it. cd softflowd-0.9.9 . /configure make make install Now we want to have softflowd start when the system boots. We'll need to add a line to the end of /etc/rc.d/rc.local . Use your device's interface after -i and your collector's IP address after -n . sudo nano /etc/rc .d /rc . local <add the following line to the end of the file > /usr/local/sbin/softflowd -i eth0 -n 10.13.37.111:2055 Save your changes with Ctrl-O , exit nano with Ctrl-X . Make sure /etc/rc.d/rc.local is executable. sudo chmod +x /etc/rc .d /rc . local Open port 2055 in the firewall so the collector can receive the data. sudo firewalld-cmd --zone=public --add-port=2055 /tcp --permanent Reboot the machine for all changes to take effect. *Original guide courtesy of@Kurt Huffmanat LogicMonitor