Documentation

NetFlow: Pre-requisites and configuration

On the page

Need some help?

This service template has been designed to meet specific needs. It requires various fields to be completed to target bandwidth consumption

Ideally, each deployed service should meet a particular need, such as measuring throughput generated by the e-mail service. In this case, the user will complete the various fields required for this measurement (destination IP of the mail server, SMTP port 25...).

Aim

Netflow service models allow you to monitor the average flow generated over a defined period of time by an application, source IP or destination IP and generate alerts if defined thresholds are exceeded. They also report data and performance graphs in the same way as other service models.

Netflow is a network protocol used to assess IP network traffic. It was developed by Cisco Systems. Today, NetFlow has become an industry standard supported by many devices. There are several versions of the protocol, but the most common versions are versions 5 and 9.

Network flows

NetFlow uses the concept of a flow to capture data about network behavior, such as the source and destination of network traffic, the applications using the network, and the amount of bandwidth allocated to those applications.

A stream is a unidirectional sequence of packets between a given source and a given destination, defined by a 7-tuple key comprising the following fields:

  • Source IP address
  • Destination IP address
  • Source Port
  • Destination port
  • IP Protocol
  • Input interface
  • Type of IP service

NetFlow records

The NetFlow information collected by a Flow Publisher is managed by creating records for each flow. Each record is managed in the NetFlow cache. When packets are captured, the statistics for active flows are updated. Once a stream has been created and placed in the NetFlow cache, it remains active until it expires. Once the flow expires, the flow record is added to a NetFlow export datagram for transmission to the NetFlow collector.

NetFlow support

In addition to Cisco, many other network equipment manufacturers offer NetFlow support on their devices. The list includes Juniper, Alcatel-Lucent and Nortel, among others. With respect to software platforms, there is support on VMware and Linux servers .

Some manufacturers use another names for this technology:

  • Jflow or cflowd at Juniper Networks
  • NetStream at 3Com/HP
  • NetStream at Huawei Technologies
  • Cflowd at Alcatel-Lucent
  • Rflow at Ericsson
  • AppFlow at Citrix

Architecture

Network elements (switches and routers) compile statistics on the network flow data they export to collectors. These detailed statistics can include the number of packets and bytes, application ports, IP addresses, QoS fields, the interfaces through which they pass, etc.

The architecture for collecting IP network traffic information is as follows:

Netflow - architecture

  • NetFlow exporter: Observes data in packets, creates monitored network traffic records and transmits this data to the NetFlow Collector.
  • NetFlow collector: Collects the records sent by the exporter and stores them in a local database.
  • ServiceNav Box: Retrieves information collected by the NetFlow Collector
  • SNP (Monitoring Platform) allows you to configure the NetFlow template to use the data reported by the ServiceNav Box

Configuration of network equipment

The network equipment must be configured to export the flows to the Netflow Collector Storage.

Our procedure Configuring NetFlow devices provides you with a large number of Netflow activation procedures depending on your device.

Setting up NetFlow Collector Storage

A collector can act as a central collector for both NetFlow and sFlow exports.

sFlow procedure: https://servicenav.coservit.com/en/documentations/how-to-use-our-service-model-networkanalysis-sflow/ 

Sizing NetFlow Collector Storage

How much disk space should an average NetFlow deployment consume? One of the biggest considerations is the impact exporting NetFlow data will have on available bandwidth, CPU overhead on devices and the hard drives that store it..

It is important to note that a network flow data export can contain records containing up to 30 conversations or streams . This is important because the average volume of NetFlow is directly proportional to the number of unique TCP/UDP sockets created by clients and servers on the network.

This aggregated nature of NetFlow, and the fact that NetFlow packets are composed solely of IP header information (i.e., not the packet payload itself), explains why the export consumes only 1-2% of the interface rate. Since 2004, Cisco's NetFlow experts have maintained a rule of thumb that NetFlow will create only 1-1.5% of throughput on the interface to which it is exported.

What is the typical stream volume per PC? The answer is: it depends, the trend seems to be about 100 flows / minute per computer, with a peak of 350

For example, a company has 1000 nodes and that each node generates 200 feeds per minute. This is equivalent to about 200,000 feeds in one minute, which is about 3300 flows per second.  Why so much flow?

Applications generate a lot of unique feeds, especially web browsers and most applications. Here are some typically very talkative applications:

  • Java, Adobe, Anti-virus, web browsers
  • Skype is very talkative and generates traffic to DNS
  • Web page feeds generating images, ads, etc.
  • Email constantly checking inbox
  • NetBios

A flow stored on the NetFlow Colletor Storage occupies 150 bytes of disk space, it is therefore recommended to provision 2 GB per day, per 100 nodes:

 Cpu(s)  4 vCPU
 RAM  8 GB
 Disk Space  20 Gb + 2 Gb per day and per 100 nodes
 Network interface  1 gbps

NetFlow Collector Storage Deployment

The NetFlow Collector Storage will be created from a ServiceNav Box master image.

It is therefore necessary to follow fully documentation on implementing a ServiceNav Box.

Configuring Collector Storage

Connecting to Collector Storage

Once the Collector Storage operating system has booted it asks for login information, this information is the following:

Login : Ask support
Password: Ask support

warning

 Caution: Never update your monitoring box. Updates are sent from the central platform.

Downloading the installation script

You need to download the installation script, to do so execute the following commands:

sudo su -

cd /home/coadmin

ftp -p software.servicenav.io

# Enter login & password

 Login : Ask support

Password: Ask support

 cd TOOLS

get xflow_installation.tar

exit

tar xvf xflow_installation.tar

Launching the installer

To launch the installer, execute these three commands:

sudo su -

cd /home/coadmin/xflow_installation

./xflow_installation.pl

Then follow the instructions on the screen:

###### xFLOW COLLECTOR INSTALLATION ##########
Copyright CoServIT 2012-2013. All rights reserved.
## Configure your flow Collector
## - Configure the rules
## - Configure the directory
## - Start and save process n(s)fcapd
## Data retention configuration
## Delete a configuration 
## Note: You can exit this program with Ctrl+C 
Press any key to continue

Press any key

##### xFLOW COLLECTOR CONFIGURATION ############

Do you want to configure the Flow Collector (y/n) [default: n] :

Type "y" and then press Enter to validate your choice

NetFlow (n) or sFlow (s) [default = n] :

We configure Netflow, type "n" and then press Enter to validate your choice

Listening Port : 9995

Fill in the listening port that you have defined and press Enter to validate your choice:

Directory : WAN

Fill in the name of the export destination directory you have defined and press Enter to validate your choice 

Do you want to save this configuration ?  (y/n) [default: n] :

Type "y" and then press Enter to validate your choice 

netflow config ok

 

Do you want to configure the data retention (y/n) [default : n] :

To set up data retention on the Collector, type "y" and then press Enter to validate your choice 

NetFlow (n) or sFlow (s) [default = n] :

We configure Netflow, type "n" and then press Enter to validate your choice

Data retention in days (example 2 for two days): 2

Fill in the number of days of data retention and press Enter to validate your choice 

Do you want to save this configuration ?  (y/n) [default: n] :

Type "y" and then press Enter to validate your choice

netflow retention ok

 

Configuring the NetworkAnalysis-NetFlow Service Template

Once the prerequisites have been configured via the installer, you can use the service template NetworkAnalysis-NetFlow by referring to the following procedure How to use our NetworkAnalysis-NetFlow service template

If an error occurs, contact support or refer to the following topics: "Checking" "Debug"

Checking Netflow Collector Storage

After launching the installer, wait about twenty minutes and check that the exported files arrive in the destination directory by executing the following command:

ll /home/coadmin/network_analysis/

netflow directory ok
Files in nfcapd format must be present and contain data, file size must be greater than 276 bytesIf you have a problem, contact support or refer to the " Debug" section. Debug 

Monitoring of NetFlow Collector Storage

In addition to the Netflow monitoring undertaken following installation of the Netflow Collector Storage, add self-monitoring services based on the following service template:

  • LIN-DirectorySize to monitor the size of your destination directories.
  • Lin-ProcessName  to monitor the proper execution of nfcapd processes

Debug

Tasks performed by the script :

  • Creation of an ACL authorizing listening on the specified port
  • Creation of the destination directory
  • Launching the NetFlow process
  • Creating a cron task to monitor the netflow process
  • Creation of a startup file to restart process in case of Collector reboot
  • Creating a Data Deletion and Retention Task

Creation of an ACL authorizing listening on the specified port

Check that the rule has been created by executing the following command:

Expected result: Presence of the rule by light authorizing the port filled in during installation.

iptables -L 

netflow acl ok

If the rule is not present, execute the following commands:

sudo su -

iptables -A INPUT -p udp -dport -j ACCEPT

/etc/init.d/iptables.sh restart

Check again if there is a problem, contact support.

Creation of the destination directory

By default the installer creates the destination directories under /home/coadmin/network_analysis . Check that the directory has been created by executing the following command: "Expected result: Presence of the directory specified during installation.

sudo su -

ll /home/coadmin/network_analysis/

netflow directory debug

If the directory does not exist, execute the following commands:

sudo su -

mkdir//home/coadmin/network_analysis/

chmod -R 777 /home/coadmin/network_analysis/

chown coadmin:coadmin /home/coadmin/network_analysis/

Check again, if there is a problem, contact support.

Launching the NetFlow process

The installer launches the netflow process based on the listening port and destination directory entered.

Check that the process is running by executing the following command:

ps -aux | grep nfcapd

and also check that the listening port and destination directory match those entered during installation.

netflow process debug
If the process is not running, execute the following command:

sudo su -

nfcapd -w -D -l /home/coadmin/network_analysis/ -p

Check again, if there is a problem, contact support.

 

Creation of a netflow process startup file for when the Collector is rebooted

(with no impact on the initialization and operation of the Collector)

The installer creates a netflow process reset file when the Collector is restarted.

Check the correct configuration by executing the following command:

sudo su -

ll /etc/rc0.d/ 

netflow debug file

In case of problems, contact support.

Creation of a cron task to monitor the netflow process (without impacting the running of the Collector)

The installer creates a cron task that runs a command every minute to check the process and restart it if necessary.

Check the creation of the task by executing the following command:

crontab -l

netflow crontab debug

In case of problems, contact support.

Creating a Data Deletion Task

If your data is not deleted or the data retention does not match what was entered during installation, execute the following command:

more /usr/local/nagios/libexec/nfcapd_deleteCache.sh

netflow purge debug

If the information returned is not correct or the number of retention days does not match what was set during installation, contact support.

Checking exports.

As a reminder, the files are generated by and are fed continuously by your NetFlow exporterThis means that the presence of these files does not necessarily mean that the exports have been successfully consumed by the Collector.

To check, execute the following command:

Sudo su -

ll /home/coadmin/network_analysis/

netflow exort debug

 

If there are no files, repeat the verification steps or contact support.

If the configurations are correct and the files are 276 bytesin size: which means that the files do not contain data, there are two possible causes:

  • Exports do not reach the Collector Storage
  • Exports look good in the Collector but do not contain any data.

In this case, contact support or run the following check commands:

tcpdump -i src

Ex:

Sudo su -
tcpdump -i ens160 src 192.168.238.156

netflow export2 debug

If exports occur, you should have connection information from your NetFlow Exporter to the Collector on the defined port. (for info, the above example is related to sFlow exports)

In case of problem check the configuration on your NetFlow Exporter or contact support.

This may also be of interest to you

Capture

sFlow: Prerequisites and configuration

top list

Networking 'Top' lists

Capture

How to use our NetworkAnalysis-NetFlow service template

en_US

Welcome to ServiceNav!

Need help? More information about our products? Write to us!
You have taken note of our privacy policy.

[COVID - 19 ] - TELEWORKING, TARGET AVAILABILITY 100% !

While the epidemic lasts, ensure the availability and performance of your IT services for teleworking, with ServiceNav!

Following the government's call to mobilize to help businesses overcome the current health and economic context, we help you, free of charge, to ensure the complete monitoring of your teleworking environments: VPN, VDI, Teams, Skype Enterprise, Citrix... Objectives: collection, availability and usage indicators, dashboards to support your communication.
We use cookies to ensure that you have the best possible experience on our site, and if you continue to use this site, we will assume that you are satisfied with it.

Reserve your place

You have taken note of our privacy policy.