Documentations

NetFlow: Prerequisites and configuration

On the page

Do you need help?

The service model has been specified to meet specific needs. It presents different fields to be filled in to target the bandwidth consumption

Ideally, each instantiated service should meet a need, for example, measuring the throughput generated by the mail service. In this case, the user will fill in the various fields necessary for this measurement (destination ip of the mail server, SMTP port 25....).

Introduction

Netflow service models allow you to monitor the average throughput generated over a defined period by an application, source ip or destination ip and generate alerts when defined thresholds are exceeded. They also provide performance data and graphs in the same way as other service models.

Netflow is a network protocol used to account for IP network traffic. It was developed by Cisco Systems. Nowadays, NetFlow has become an industry standard supported by many devices. There are several versions of the protocol, but the most common versions are 5 and 9.

Network flows

NetFlow uses the concept of a flow to capture data about network behavior, such as the source and destination of network traffic, the applications using the network, and the amount of bandwidth allocated to those applications.

A flow is a unidirectional sequence of packets between a source and a given destination, defined by a 7-tuple key comprising the following fields:

  • Source IP address
  • Destination IP address
  • Source Port
  • The port of destination
  • IP Protocol
  • Input interface
  • Type of IP service

NetFlow records

The NetFlow information collected by Flow Publisher is managed by creating records for each flow. Each record is managed in the NetFlow cache. As packets are captured, statistics for active flows are updated. Once a flow has been created and placed in the NetFlow cache, it remains active until it expires. Once the flow expires, the flow record is added to a NetFlow export datagram for transmission to the NetFlow collector.

NetFlow support

In addition to Cisco, many network equipment manufacturers offer NetFlow support on their boxes. The list includes Juniper, Alcatel-Lucent and Nortel, among others. On the software platform side, there is support on servers VMWare and Linux.

Some manufacturers use another name for this technology:

  • Jflow or cflowd at Juniper Networks
  • NetStream at 3Com/HP
  • NetStream at Huawei Technologies
  • Cflowd at Alcatel-Lucent
  • Rflow at Ericsson
  • AppFlow at Citrix

Architecture

Network elements (switches and routers) compile statistics on the network flow data they export to collectors. These detailed statistics can include packet and byte counts, application ports, IP addresses, QoS fields, interfaces through which they pass, etc.

The architecture for collecting IP network traffic information is as follows:

Netflow - architecture

  • NetFlow exporter: Observes packet data, creates records of monitored network traffic and transmits this data to the NetFlow collector.
  • NetFlow collector: Collects records sent by the exporter, stores them in a local database.
  • ServiceNav Box: Retrieves the information collected by the NetFlow collector
  • SNP (Supervisory Platform) allows you to configure the NetFlow is to use the data reported by the ServiceNav Box

Network equipment configuration

The network devices must be configured to export the flows to the Netflow Collector Storage.

Our procedure NetFlow, Device Configuration provides you with a large number of Netflow activation procedures depending on your device.

Implementation of NetFlow Collector Storage

The proposed master can be shared and collect NetFlow and sFlow exports.

Procedure sFlow: https://coservit.com/servicenav/fr/documentation/sflow-prerequis-et-configuration/ 

Sizing NetFlow Collector Storage

How much disk space should an average NetFlow deployment consume? One of the biggest concerns is that exporting NetFlow will impact the available bandwidth, processor overhead on the devices or hard drives that store it.

It is important to note that a network flow data export can contain recordings containing up to 30 conversations or streams . This is important because the average volume of NetFlow is directly proportional to the number of unique TCP/UDP sockets created by clients and servers on the network.

This aggregated nature of NetFlow, and the fact that NetFlow packets are composed solely of IP header information (i.e., not the packet payload itself), is why exporting consumes only 1-2% of interface throughput. Since 2004, Cisco's NetFlow experts have maintained a rule of thumb that NetFlow will only create 1 to 1.5% of throughput on the interface on which it is exported.

What is the typical flow volume per PC? The answer is: it depends, the trend appears to be approximately 100 streams/minute per computer, with a peak of 350

For example, a company has 1000 knots and each node generates 200 streams per minute. This is equivalent to approximately 200,000 streams in one minutewhich is approximately 3300 streams per second. Why so much flow?

Applications generate a lot of unique streams, especially web browsers and most applications. Here are some typical chatty applications:

  • Java, Adobe, Anti-virus, web browsers
  • Skype is very chatty and causes traffic to the DNS
  • Stream of web pages generating images, ads, etc.
  • Email constantly checking the inbox
  • NetBios

A flow stored on the NetFlow Colletor Storage takes up 150 bytes of disk space, so we recommend provisioning 2 GB per day for every 100 nodes:

 Cpu(s)  4 vCPU
 RAM  8 GB
 Disk space  20 GB + 2 GB per day per 100 nodes
 Network interface  1 gbps

Deploying NetFlow Collector Storage

The NetFlow Collector Storage will be created from a ServiceNav Box master.

It is therefore appropriate to apply fully documentation of the implementation of a ServiceNav Box.

Configuring Collector Storage

Connecting to Collector Storage

Once the Collector Storage has started on the operating system it asks for login information, this information is the following:

Login : To be requested from support
Mdp: To be requested from support

warning

 Warning: Never update your supervision box. The updates are controlled from the central platform.

Download the installation script

You need to download the installation script, to do this run the following commands:

sudo su -

cd /home/coadmin

ftp -p software.servicenav.io

# Enter login & password

 Login : To be requested from support

Mdp: To be requested from support

 cd TOOLS

get xflow_installation.tar

exit

tar xvf xflow_installation.tar

Launching the installer

To launch the installer, execute these three commands:

sudo su -

cd /home/coadmin/xflow_installation

./xflow_installation.pl

Then follow the instructions on the screen:

###### xFLOW COLLECTOR INSTALLATION #####
Copyright CoServIT 2012-2013. All rights reserved.
## Configure your flow Collector
## - Configure the rules
## - Configure the directory
## - Start and save process n(s)fcapd
## Data retention configuration
## Delete a configuration 
## Note: You can exit this program with Ctrl+C 
Press any key to continue

Press any key

##### xFLOW COLLECTOR CONFIGURATION ######

Do you want to configure the Flow Collector (y/n) [default: n] :

Type "y" then press the Input to validate your choice

NetFlow (n) or sFlow (s) [default = n] :

We configure Netflow, type "n" then press Input to validate your choice

Listening Port : 9995

Enter the listening port you have defined and press the Input to validate your choice:

Directory: WAN

Enter the name of the export destination directory you have defined and press the Input to validate your choice 

Do you want to save this configuration ?  (y/n) [default: n] :

Type "y" then press the Input to validate your choice 

netflow config ok

 

Do you want to configure the data retention (y/n) [default: n] :

To set up data retention on the Collector, type "y" and press Input to validate your choice 

NetFlow (n) or sFlow (s) [default = n] :

We configure Netflow, type "n" then press Input to validate your choice

Data retention in days (example 2 for two days): 2

Enter the number of days of data retention and press the Input to validate your choice 

Do you want to save this configuration ?  (y/n) [default: n] :

Type "y" then press the Input to validate your choice

netflow retention ok

 

Configuring the NetworkAnalysis-NetFlow service model

Once the prerequisites are configured via the installer, you can use the NetworkAnalysis-NetFlow by referring to the following procedure How to use our NetworkAnalysis-NetFlow service model

In case of malfunction, contact support or refer to the following topics: "Verification" and "Debug"

Checking Netflow Collector Storage

After launching the installer, wait about 20 minutes and check that the exported files arrive in the destination directory by running the following command:

ll /home/coadmin/network_analysis/

netflow directory ok
Files in nfcapd format must be present and contain data, file size must be greater than 276 bytesIf you have a problem, please contact support or refer to the " Debug "

Supervision of NetFlow Collector Storage

In addition to the monitoring done during the installation of the Netflow Collector Storage, add on the autosupervision the services based on the following service models:

  • LIN-DirectorySize to monitor the size of your destination directories.
  • Lin-ProcessName  to monitor the proper execution of processes nfcapd

Debug

Tasks performed by the script :

  • Create an ACL allowing listening on the specified port
  • Creation of the destination directory
  • Launching the NetFlow process
  • Creating a cron job to monitor the netflow process
  • Creation of an initialization file to restart the process when the Collector is restarted
  • Creating a Delete and Retain Task

Create an ACL allowing listening on the specified port

Check that the rule has been created by running the following command:

Expected result: Presence of the firewall rule authorizing the port specified during installation.

iptables -L 

netflow acl ok

If the rule is not present, run the following commands:

sudo su -

iptables -A INPUT -p udp -dport -j ACCEPT

/etc/init.d/iptables.sh restart

Check again if there is a problem, contact support.

Creation of the destination directory

By default the installer creates the destination directories under " /home/coadmin/network_analysis ". Check that the directory has been created by running the following command: "Expected result: Presence of the directory specified during installation.

sudo su -

ll /home/coadmin/network_analysis/

netflow directory debug

If the directory does not exist, run the following commands:

sudo su -

mkdir//home/coadmin/network_analysis/

chmod -R 777 /home/coadmin/network_analysis/

chown coadmin:coadmin /home/coadmin/network_analysis/

Check again, if you have a problem, contact support.

Launching the NetFlow process

The installer starts the netflow process based on the specified listening port and destination directory.

Check that the process is running by running the following command:

ps -aux | grep nfcapd

and also check that the listening port and destination directory match those specified during installation.

netflow process debug
If the process is not running, run the following command:

sudo su -

nfcapd -w -D -l /home/coadmin/network_analysis/ -p

Check again, if you have a problem, contact support.

 

Creation of a netflow process reset file in case of Collector restart

(without impact on the initialization and operation of the Collector)

The installer creates a reset file for the netflow process when the Collector is restarted.

Check the correct configuration by running the following command:

sudo su -

ll /etc/rc0.d/ 

netflow debug file

If you have any problems, contact support.

Creation of a cron task to monitor the netflow process (without impact on the Collector initialization)

The installer creates a cron job that runs a command every minute to check the process and restart it if necessary.

Verify the creation of the task by running the following command:

crontab -l

netflow crontab debug

If you have any problems, contact support.

Creating a Data Deletion Task

If your data is not deleted or the data retention does not match what was set during installation, run the following command:

more /usr/local/nagios/libexec/nfcapd_deleteCache.sh

netflow purge debug

If the order return does not match or the number of retention days does not match what was set during installation, contact support.

Verification of exports.

As a reminder, the nfcapd are generated by the nfcapd and are continuously fed by your export NetFlowThis means that the presence of these files does not necessarily indicate that the exports are present in the Collector.

To verify, run the following command:

Sudo su -

ll /home/coadmin/network_analysis/

netflow exort debug

 

If there are no files, repeat the verification steps or contact support.

If the configurations are correct and the files are equal in size to 276 bytesThis means that the files do not contain any data, two possible causes:

  • Exports do not show up at Collector Storage
  • Exports look good to the Collector but do not contain any data

In this case, contact support or run the following check commands:

tcpdump -i src

Ex:

Sudo su -
tcpdump -i ens160 src 192.168.238.156

netflow export2 debug

If the exports show up, you should have connection information from your NetFlow Exporter to the Collector on the defined port (for information, the above example is related to sFlow exports)

In case of problem, check the configuration on your NetFlow Exporter or contact the support.

You may also be interested in

Capture

sFlow: Prerequisites and configuration

top list

Top Lists

Capture

How to use our NetworkAnalysis-NetFlow service model

en_GB

Welcome to ServiceNav!

Do you need some help? More information about our products? Write to us!
You have taken note of our privacy policy.
We use cookies to ensure the best experience on our site. If you continue to use this site, we will assume that you are satisfied with it.

Reserve your place

You have taken note of our privacy policy.