The service model has been specified to meet specific needs. It presents different fields to be filled in to target the bandwidth consumption
Ideally, each instantiated service should meet a need, for example, measuring the throughput generated by the mail service. In this case, the user will fill in the various fields necessary for this measurement (destination ip of the mail server, SMTP port 25....).
Introduction
Netflow service models allow you to monitor the average throughput generated over a defined period by an application, source ip or destination ip and generate alerts when defined thresholds are exceeded. They also provide performance data and graphs in the same way as other service models.
Netflow is a network protocol used to account for IP network traffic. It was developed by Cisco Systems. Nowadays, NetFlow has become an industry standard supported by many devices. There are several versions of the protocol, but the most common versions are 5 and 9.
Network flows
NetFlow uses the concept of a flow to capture data about network behavior, such as the source and destination of network traffic, the applications using the network, and the amount of bandwidth allocated to those applications.
A flow is a unidirectional sequence of packets between a source and a given destination, defined by a 7-tuple key comprising the following fields:
- Source IP address
- Destination IP address
- Source Port
- The port of destination
- IP Protocol
- Input interface
- Type of IP service
NetFlow records
The NetFlow information collected by Flow Publisher is managed by creating records for each flow. Each record is managed in the NetFlow cache. As packets are captured, statistics for active flows are updated. Once a flow has been created and placed in the NetFlow cache, it remains active until it expires. Once the flow expires, the flow record is added to a NetFlow export datagram for transmission to the NetFlow collector.
NetFlow support
In addition to Cisco, many network equipment manufacturers offer NetFlow support on their boxes. The list includes Juniper, Alcatel-Lucent and Nortel, among others. On the software platform side, there is support on servers VMWare and Linux.
Some manufacturers use another name for this technology:
- Jflow or cflowd at Juniper Networks
- NetStream at 3Com/HP
- NetStream at Huawei Technologies
- Cflowd at Alcatel-Lucent
- Rflow at Ericsson
- AppFlow at Citrix
Architecture
Network elements (switches and routers) compile statistics on the network flow data they export to collectors. These detailed statistics can include packet and byte counts, application ports, IP addresses, QoS fields, interfaces through which they pass, etc.
The architecture for collecting IP network traffic information is as follows:
- NetFlow exporter: Observes packet data, creates records of monitored network traffic and transmits this data to the NetFlow collector.
- NetFlow collector: Collects records sent by the exporter, stores them in a local database.
- ServiceNav Box: Retrieves the information collected by the NetFlow collector
- SNP (Supervisory Platform) allows you to configure the NetFlow is to use the data reported by the ServiceNav Box
Network equipment configuration
The network devices must be configured to export the flows to the Netflow Collector Storage.
Our procedure NetFlow, Device Configuration provides you with a large number of Netflow activation procedures depending on your device.
Implementation of NetFlow Collector Storage
The proposed master can be shared and collect NetFlow and sFlow exports.
Procedure sFlow: https://coservit.com/servicenav/fr/documentation/sflow-prerequis-et-configuration/
Sizing NetFlow Collector Storage
How much disk space should an average NetFlow deployment consume? One of the biggest concerns is that exporting NetFlow will impact the available bandwidth, processor overhead on the devices or hard drives that store it.
It is important to note that a network flow data export can contain recordings containing up to 30 conversations or streams . This is important because the average volume of NetFlow is directly proportional to the number of unique TCP/UDP sockets created by clients and servers on the network.
This aggregated nature of NetFlow, and the fact that NetFlow packets are composed solely of IP header information (i.e., not the packet payload itself), is why exporting consumes only 1-2% of interface throughput. Since 2004, Cisco's NetFlow experts have maintained a rule of thumb that NetFlow will only create 1 to 1.5% of throughput on the interface on which it is exported.
What is the typical flow volume per PC? The answer is: it depends, the trend appears to be approximately 100 streams/minute per computer, with a peak of 350
For example, a company has 1000 knots and each node generates 200 streams per minute. This is equivalent to approximately 200,000 streams in one minutewhich is approximately 3300 streams per second. Why so much flow?
Applications generate a lot of unique streams, especially web browsers and most applications. Here are some typical chatty applications:
- Java, Adobe, Anti-virus, web browsers
- Skype is very chatty and causes traffic to the DNS
- Stream of web pages generating images, ads, etc.
- Email constantly checking the inbox
- NetBios
A flow stored on the NetFlow Colletor Storage takes up 150 bytes of disk space, so we recommend provisioning 2 GB per day for every 100 nodes:
| Cpu(s) | 4 vCPU |
| RAM | 8 GB |
| Disk space | 20 GB + 2 GB per day per 100 nodes |
| Network interface | 1 gbps |
Deploying NetFlow Collector Storage
The NetFlow Collector Storage will be created from a ServiceNav Box master.
It is therefore appropriate to apply fully documentation of the implementation of a ServiceNav Box.
Configuring Collector Storage
Connecting to Collector Storage
Once the Collector Storage has started on the operating system it asks for login information, this information is the following:
Login : To be requested from support
Mdp: To be requested from support
Warning: Never update your supervision box. The updates are controlled from the central platform.
Downloading the installation script (not necessary from version 5.0 on)
You need to download the installation script, to do this run the following commands:
sudo su -
cd /root/
ftp -p software.servicenav.io
# Enter login & password
Login : To be requested from support
Mdp: To be requested from support
cd TOOLS
get xflow_installation.tar
exit
tar xvf xflow_installation.tar
Launching the installer
To launch the installer, execute these three commands:
Up to and including version 4.19
sudo su -
cd /root/xflow_installation
./xflow_installation.pl
From version 5.0
sudo su -
cd /root/vsb_installation
./xflow_installation.pl
Then follow the instructions on the screen:
###### xFLOW COLLECTOR INSTALLATION #####
Copyright CoServIT 2012-2013. All rights reserved.
## Configure your flow Collector
## - Configure the rules
## - Configure the directory
## - Start and save process n(s)fcapd
## Data retention configuration
## Delete a configuration
## Note: You can exit this program with Ctrl+C
Press any key to continue
Press any key
##### xFLOW COLLECTOR CONFIGURATION ######
Do you want to configure the Flow Collector (y/n) [default: n] :
Type "y" then press the Input to validate your choice
NetFlow (n) or sFlow (s) [default = n] :
We configure Netflow, type "n" then press Input to validate your choice
Listening Port : 9995
Enter the listening port you have defined and press the Input to validate your choice:
Directory: WAN
Enter the name of the export destination directory you have defined and press the Input to validate your choice
Do you want to save this configuration ? (y/n) [default: n] :
Type "y" then press the Input to validate your choice
Do you want to configure the data retention (y/n) [default: n] :
To set up data retention on the Collector, type "y" and press Input to validate your choice
NetFlow (n) or sFlow (s) [default = n] :
Before version 5.0 only We configure Netflow, type "n" and press the key Input to validate your choice
Data retention in days (example 2 for two days): 2
Enter the number of days of data retention and press the Input to validate your choice
Do you want to save this configuration ? (y/n) [default: n] :
Type "y" then press the Input to validate your choice
Configuring the NetworkAnalysis-NetFlow service model
Once the prerequisites are configured via the installer, you can use the NetworkAnalysis-NetFlow by referring to the following procedure How to use our NetworkAnalysis-NetFlow service model
In case of malfunction, contact support or refer to the following topics: "Verification" and "Debug"
Checking Netflow Collector Storage
After launching the installer, wait about 20 minutes and check that the exported files arrive in the destination directory by running the following command:
ll /home/coadmin/network_analysis/
Supervision of NetFlow Collector Storage
In addition to the monitoring done during the installation of the Netflow Collector Storage, add on the autosupervision the services based on the following service models:
- LIN-DirectorySize to monitor the size of your destination directories.
- Lin-ProcessName to monitor the proper execution of processes nfcapd
Debug
Tasks performed by the script :
- Create an ACL allowing listening on the specified port
- Creation of the destination directory
- Launching the NetFlow process
- Creating a cron job to monitor the netflow process
- Creation of an initialization file to restart the process when the Collector is restarted
- Creating a Delete and Retain Task
Create an ACL allowing listening on the specified port
Check that the rule has been created by running the following command:
Expected result: Presence of the firewall rule authorizing the port specified during installation.
iptables -L
If the rule is not present, run the following commands:
sudo su -
iptables -A INPUT -p udp -dport -j ACCEPT
/etc/init.d/iptables.sh restart
Check again if there is a problem, contact support.
Creation of the destination directory
By default the installer creates the destination directories under " /home/coadmin/network_analysis ". Check that the directory has been created by running the following command: "Expected result: Presence of the directory specified during installation.
sudo su -
ll /home/coadmin/network_analysis/
If the directory does not exist, run the following commands:
sudo su -
mkdir//home/coadmin/network_analysis/
chmod -R 777 /home/coadmin/network_analysis/
chown coadmin:coadmin /home/coadmin/network_analysis/
Check again, if you have a problem, contact support.
Launching the NetFlow process
The installer starts the netflow process based on the specified listening port and destination directory.
Check that the process is running by running the following command:
ps -aux | grep nfcapd
and also check that the listening port and destination directory match those specified during installation.
sudo su -
nfcapd -w -D -l /home/coadmin/network_analysis/ -p
Check again, if you have a problem, contact support.
Creation of a netflow process reset file in case of Collector restart
(without impact on the initialization and operation of the Collector)
The installer creates a reset file for the netflow process when the Collector is restarted.
Check the correct configuration by running the following command:
sudo su -
ll /etc/rc0.d/
If you have any problems, contact support.
Creation of a cron task to monitor the netflow process (without impact on the Collector initialization)
The installer creates a cron job that runs a command every minute to check the process and restart it if necessary.
Verify the creation of the task by running the following command:
crontab -l
If you have any problems, contact support.
Creating a Data Deletion Task
If your data is not deleted or the data retention does not match what was set during installation, run the following command:
more /usr/local/nagios/libexec/nfcapd_deleteCache.sh
If the order return does not match or the number of retention days does not match what was set during installation, contact support.
Verification of exports.
As a reminder, the nfcapd are generated by the nfcapd and are continuously fed by your export NetFlowThis means that the presence of these files does not necessarily indicate that the exports are present in the Collector.
To verify, run the following command:
sudo su -
ll /home/coadmin/network_analysis/
If there are no files, repeat the verification steps or contact support.
If the configurations are correct and the files are equal in size to 276 bytesThis means that the files do not contain any data, two possible causes:
- Exports do not show up at Collector Storage
- Exports look good to the Collector but do not contain any data
In this case, contact support or run the following check commands:
tcpdump -i src
Ex:
Sudo su -
tcpdump -i ens160 src 192.168.238.156
If the exports show up, you should have connection information from your NetFlow Exporter to the Collector on the defined port (for information, the above example is related to sFlow exports)
In case of problem, check the configuration on your NetFlow Exporter or contact the support.
