Network Monitoring
For OpMon Traffic Analyzer to work properly, you need to have a device with Netflow protocol support in your network. Netflow protocol collects information about network traffic and sends it so that OpMon Traffic Analyzer can generate the information in a consolidated manner.
Faced with the need for protocols that offer traffic information from a large network with low computational cost, IETF has proposed IPFIX standard (IP Flow Information Export), whose purpose was to establish an architecture for traffic analysis. Subsequently to this work, several protocols were proposed, among them Netflow (RFC 3954), created by Cisco Systems. The working group has chosen Netflow protocol to be developed and implemented.
The main advantages of Netflow utilization are:
- Functioning as a cache to speed up lookups in routing tables;
- Bypasses verification of access-list tables (input only) every time a package arrives, so the routing process is more efficient;
- Allows you to export stream information used by Netflow cache, facilitating data collection for future analyses without the need to add an analyzer to each link.
Cisco defines a stream as a unidirectional package sequence between source and destination machines, and you could say that Netflow provides the summarization of information about the traffic of a router or switch. Network streams are highly granular and identified by both IP addresses and by the number of transport layer ports. To uniquely identify a stream, Netflow also uses the fields “Protocol type” and “Type of Service” (ToS) of the IP and the input logical interface of the router or switch.
The flows kept in the router/switch cache are exported to a collector (OpMon Traffic Analyzer) in the following situations: the element is idle for more than 15 seconds, its duration exceeds 30 minutes, a TCP connection is shut down with “FIN” or “RST” flag, the flow table is full, or the user resets the stream settings. The maximum time that a stream remains on the device before being exported is 30 minutes.
For Netflow to run on a router or switch it is necessary that your operating system version has this protocol support. We recommend that you consult your network provider on which equipments have this support.
In Cisco routers, for example, there is no general command that enables Netflow for all interfaces; to enable it, it is necessary to individually enable it via interface, through the route-cache flow command. The Netflow process works only for incoming packages, so the packages exported by Netflow refer only to the traffic that comes into the enabled interface.
For Netflow, the stream is defined as a set of five variables, aside from “Protocol Type”: Source IP, destination IP, source port and destination port. In addition to these variables, the Netflow tables keep the source and target interface, relating to the IP package traffic.
For each incoming IP package on the interface, Netflow identifies their flow and checks whether there is already an input of this flow in the cache table. If it exists, it switches directly to the specified destination interface and, if not, it performs a lookup on the routing tables and access-list tables. If this package has a restriction in the access-list tables or if its destination IP is not found on the routing tables, the package will then be sent to the “NULL” interface (a package with the destination to this interface identifies it was discarded). Netflow also creates an entry in its cache table for the “NULL” destination.
Another important process on Netflow is data export, known as flow-export. Flow-export is done by sending encapsulated data in UDP packages. Their destination is the OpMon Traffic Analyzer IP that was previously on the router, and the UDP package content depends on Netflow version. Currently the routers can export flows created by Netflow versions from 1 to 8. The moment when the router begins to export the stream data depends on the configuration.
In practice, Netflow is a software developed to characterize the network operation, which is critical to map the network behavior, including: application and use, efficiency and use of resources, impact of changes and amendments, abnormalities and vulnerabilities. Netflow creates an environment with tools to understand “who”, “what”, “when” and “how” the traffic flows over the network, enabling process improvements and adaptations of the environment to business needs.
Traditionally, the market relies on SNMP-based bandwidth monitoring which, despite facilitating capacity planning, does little to characterize traffic patterns and applications, which are essential to understand “how much” and “how” the network can support the deal. A more detailed understanding of how the band is being used is extremely important in IP networks; package and interface byte counters are useful, but understanding which IP addresses are traffic source and destination and which applications are generating it is priceless.
The ability to characterize IP traffic and understand “where” and “how” it flows is critical to network performance, availability, and troubleshooting. Monitoring IP traffic flows facilitates capacity planning and ensures that resources are used appropriately and in support to organizational goals. It helps determine where to apply QoS, to optimize the use of resources and plays a vital role in network security by detecting denial of service (DoS) attacks, propagation of worms and other undesirable events on the network.
Netflow solves many common problems found by IT and telecommunications professionals, such as:
- Reviewing new applications and their network impact – identifying loads of new network applications, such as SAP or adding new remote sites;
- Reduction of WAN traffic peak – assessing the impact on the network with the implementation of new policies, identifying “who” is using the network and the “top talkers”;
- Troubleshooting and identification of bottlenecks and critical network points – diagnosing poor network performance (slowness);
- Detection of unauthorized traffic – avoids expensive upgrades by identifying applications that are causing congestion;
- Safety and anomaly detection;
- Validation of QoS parameters – adequate bandwidth allocation for each service class (CoS).
Each package transmitted over a router or switch is examined by a set of attributes of the IP package, which are the IP package identity, or the package thumbprint, and determine whether they are unique or similar to other ones.
All packages with the same source/destination IP, source/destination ports, protocol interface and service class are grouped into a stream of packages and bytes, then counted and condensed into a database called sw cache Netflow. The following figure illustrates this process.
Flow information is extremely useful for understanding the network behavior:
- ESource address: reports on the traffic source;
- Destination address: reports on who’s getting the traffic;
- Doors: characterize the application;
- Service class: examines the traffic priority;
- Interface: reports on how traffic is being used by the network device.
Netflow must be “exported” to a report server, also known as “Netflow collector”, in our case, OpMon Traffic Analyzer. OpMon Traffic Analyzer organizes, interprets the flows exported and combines them to generate useful reports for traffic and security analysis. Netflow, unlike the SNMP polling, exports the information periodically to the collector.
A stream is ready for export when inactive for some time (no new packets received), or if the stream is of long term (active), and has a duration longer than the active timer (FTP of large files). In addition, the stream is ready for export when a TCP flag indicates that the stream ended. There are timers to determine whether a stream is inactive or a long term stream. All timers for export are configurable, but the standards are used in most cases. The collector can combine flows and aggregate traffic. For example, an FTP that lasts more than the active timer can be divided into multiple streams and the collector can combine them showing the total FTP traffic to a server at a particular time of the day.
