NetStream Technology White Paper

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Contents

Overview·· 2

Technical background· 2

Application scenarios· 2

NetStream implementation· 2

NetStream architecture· 2

NetStream mechanism·· 3

NetStream sampling· 4

NetStream flow creation· 4

NetStream flow aging· 5

Periodical aging· 5

Forced aging· 5

TCP FIN- and RST-triggered aging· 5

NetStream data export 6

Traditional data export 6

Aggregation data export 6

NetStream export formats· 8

NetStream data export formats· 8

Version 5 format 8

Version 8 format 9

Version 9 format 10

Version 10 format 12

Export format comparison· 13

NetStream mirroring· 14

NetStream filtering· 14

Typical network applications· 14

Configuring NetStream for traffic accounting· 14

Configuring NetStream for network planning· 15

 

Overview

Technical background

The fast development of the Internet increases the usage of network bandwidth and network support for more services and applications. Featuring inflexibility or a high cost of investments on specific servers, the traditional traffic statistics collection methods (such SNMP and port mirroring) cannot manage networks in a fine-grained manner. As a result, a new technology is urgently expected.

NetStream is developed to address these issues. NetStream is an accounting technology that provides statistics on a per-flow basis and defines the method for devices to output network traffic statistics. Devices collect and analyze data passing through them and report it to the network traffic analyzer. After consolidation, the data is stored in a database for further analysis and processing.

NetStream provides detailed data flow information with the least impact on NetStream performance. You can use NetStream on the access layer, aggregation layer, and core layer of a network. It helps you learn about network operating state, identify and resolve performance issues and network errors, and obtain guidelines for network optimization, device investments, and bandwidth optimization.

Application scenarios

NetStream has the following application scenarios:

·     Accounting—Provides fine-grained traffic data for accounting based on various metrics, such as link usage, bandwidth usage, and time ranges. ISPs can use the data to implement flexible accounting polices based on time, bandwidth, application, and service quality. Enterprises can use the data to calculate department cost and allocation cost for more efficient resource usage.

·     Network planning—Provides key information for network management tools, such as information about inter-AS traffic. This optimizes network design and planning, achieving optimal network performance and reliability with minimal network operation costs.

·     Network monitoring—Monitors real-time traffic on the Internet-facing interface and analyzes bandwidth usage for various services by being deployed at the network egress. Administrators can use the information to determine the network operating status, identify inappropriate network structures or performance bottlenecks on networks, and plan and allocate network resources.

·     User monitoring and analysis—Allows network administrators to obtain network and application resource usage of users so that they can efficiently plan and allocate network resources for ensuring network security.

NetStream implementation

NetStream collects and analyzes network service traffic on a per-flow basis. It classifies packets that have identical attributes into the same flow, and then collects, records, and output statistics for each flow. NetStream can also classify flows that have identical attributes into the same aggregate flow, and then collect and output statistics for the aggregate flow.

NetStream architecture

A typical NetStream system includes the following elements:

·     NetStream data exporter—A device configured with NetStream. The NDE provides the following functions:

¡     Classifies traffic flows.

¡     Collects data from the classified flows.

¡     Aggregates and exports the data to the NSC.

·     NetStream collector—A program running on an operating system. The NSC parses the packets received from the NDEs, and saves the data to its database.

·     NetStream data analyzer—A network traffic analyzing tool. Based on the data in NSC, the NDA generates reports for traffic billing, network planning, and attack detection and monitoring. The NDA can collect data from multiple NSCs. Typically, the NDA features a Web-based system for easy operation.

NSC and NDA are typically integrated into a NetStream server.

Figure 1 NetStream system

 

NetStream mechanism

NetStream functions by the following flows:

1.     The NDE reports the detailed collected flow statistics to the NSC periodically.

2.     The NSC stores the statistics to the database and sends the statistics to the NDA.

3.     The NDA analyzes the statistics for accounting and network planning.

As shown in Figure 2, the device, serving as an NDE, works as follows:

1.     The device performs NetStream sampling on service traffic as configured.

2.     The device creates NetStream flows based on key attributes.

3.     The device performs NetStream flow aging as configured.

4.     The device exports NetStream flows as configured.

Figure 2 NetStream processing flow for an NDE

 

NetStream sampling

You can use NetStream together with a sampler to collect and analyze statistics of sampled packets by setting an appropriate sampling interval. This reduces the impact of NetStream on device performance. The collected statistics can basically reflect the network flow status.

NetStream supports the following sampling modes:

·     Random—Samples a random packet out of a specified number of packets. For example, if the sampling interval is 100, the device selects 1 packet randomly from every 100 packets.

·     Fixed—Samples one packet out of a fixed number of packets. For example, if the sampling interval is 100, the device selects a packet after every 100 packets are transmitted. If the device selects the third packet, the device will continue to select the 103rd packet, 203rd packet, and so on.

NetStream flow creation

NetStream classifies the sampled packets that have the same key attributes into the same flow. For example, if the specified key attributes are the destination MAC address and the source MAC address, packets that have the same source and destination MAC addresses will be classified into the same flow.

Table 1 shows the key attributes for different types of packets.

Table 1 Key attributes

Packet type	Key attributes
Layer 2 packet	Destination MAC address, source MAC address, destination VLAN ID, source VLAN ID, protocol type, input interface, and output interface.
Layer 3 packet	Destination IP address, source IP address, destination port number, source port number, protocol type, ToS, IPv6 flow label, VPN, application ID, VXLAN ID, input interface, and output interface.
MPLS packet	MPLS label, destination IP address, source IP address, destination port number, source port number, protocol type, ToS, IPv6 flow label, VPN, application ID, input interface, and output interface.

 

NetStream flow aging

NetStream uses flow aging to enable the NDE to export NetStream data to NetStream servers. NetStream creates a NetStream entry for each flow for storing the flow statistics in the cache.

When a flow is aged out, the NDE performs the following operations:

·     Exports the summarized data to NetStream servers in a specific format.

·     Clears NetStream entry information in the cache.

NetStream supports the following flow aging methods:

·     Periodical aging.

·     Forced aging.

·     TCP FIN- and RST-triggered aging.

Periodical aging

Periodical aging uses the following methods:

·     Inactive flow aging—A flow is inactive if no packet arrives for the NetStream entry within the inactive flow aging timer. When the timer expires, the following events occur:

¡     The inactive flow entry is aged out.

¡     The statistics of the flow are sent to NetStream servers and are cleared in the cache. The statistics can no longer be displayed by using the display ip netstream cache command.

This method ensures that inactive flow entries are cleared from the cache in a timely manner so new entries can be cached.

·     Active flow aging—A flow is active if packets arrive for the NetStream entry within the active flow aging timer. When the timer expires, the statistics of the active flow are exported to NetStream servers. The device continues to collect active flow statistics.

This method periodically exports the statistics of active flows to NetStream servers.

Forced aging

To implement forced aging, use one of the following methods:

·     Clear the NetStream cache immediately. All entries in the cache are aged out and exported to NetStream servers.

·     Specify the upper limit for cached entries and configure the system to take either of the following actions when the limit is reached:

¡     Age out the oldest entries.

¡     Disable creation of a new entry in the cache.

TCP FIN- and RST-triggered aging

TCP FIN- and RST-triggered aging is automatically performed when a TCP connection is terminated.

A TCP connection is terminated when a packet with a FIN or RST flag is received.

When a packet with a FIN or RST flag is recorded for a flow with an existing NetStream entry, the entry is immediately aged out, exported, and cleared. However, when the first packet of a flow has a FIN or RST flag, a new NetStream entry is created instead of being aged out.

NetStream data export

Traditional data export

Traditional NetStream collects the statistics of each flow and exports the statistics to NetStream servers.

This method consumes more bandwidth and CPU than the aggregation method, and it requires a large cache size

Aggregation data export

NetStream aggregation merges the flow statistics according to the aggregation criteria of an aggregation mode, and it sends the summarized data to NetStream servers. The NetStream aggregation data export uses less bandwidth than the traditional data export.

Table 2 lists the available aggregation modes. In each mode, the system merges statistics for multiple flows into statistics for one aggregate flow if each aggregation criterion is of the same value. The system records the statistics for the aggregate flow. These aggregation modes work independently and can take effect concurrently.

For example, when the aggregation mode configured on the NDE is protocol-port, NetStream aggregates the statistics of flow entries by protocol number, source port, and destination port. Four NetStream entries record four TCP flows with the same destination address, source port, and destination port, but with different source addresses. In the aggregation mode, only one NetStream aggregation entry is created and sent to NetStream servers.

Table 2 NetStream aggregation modes

Aggregation mode	Aggregation criteria
AS aggregation	·     Source AS number ·     Destination AS number ·     Inbound interface index ·     Outbound interface index
BGP community aggregation	·     Inbound interface index ·     Outbound interface index ·     BGP community attribute
Protocol-port aggregation	·     Protocol number ·     Source port ·     Destination port
Source-prefix aggregation	·     Source AS number ·     Source address mask length ·     Source prefix (source network address) ·     Inbound interface index
Destination-prefix aggregation	·     Destination AS number ·     Destination address mask length ·     Destination prefix (destination network address) ·     Outbound interface index
Prefix aggregation	·     Source AS number ·     Destination AS number ·     Source address mask length ·     Destination address mask length ·     Source prefix ·     Destination prefix ·     Inbound interface index ·     Outbound interface index
Prefix-port aggregation	·     Source prefix ·     Destination prefix ·     Source address mask length ·     Destination address mask length ·     ToS ·     Protocol number ·     Source port ·     Destination port ·     Inbound interface index ·     Outbound interface index
ToS-AS aggregation	·     ToS ·     Source AS number ·     Destination AS number ·     Inbound interface index ·     Outbound interface index
ToS-source-prefix aggregation	·     ToS ·     Source AS number ·     Source prefix ·     Source address mask length ·     Inbound interface index
ToS-destination-prefix aggregation	·     ToS ·     Destination AS number ·     Destination address mask length ·     Destination prefix ·     Outbound interface index
ToS-prefix aggregation	·     ToS ·     Source AS number ·     Source prefix ·     Source address mask length ·     Destination AS number ·     Destination address mask length ·     Destination prefix ·     Inbound interface index ·     Outbound interface index
ToS-protocol-port aggregation	·     ToS ·     Protocol type ·     Source port ·     Destination port ·     Inbound interface index ·     Outbound interface index
ToS-BGP-nexthop	·     ToS ·     BGP next hop ·     Outbound interface index

 

If packets are not forwarded according to the BGP routing table, the AS number or BGP next hop cannot be obtained.

NetStream export formats

NetStream exports data in UDP datagrams in one of the following formats:

·     Version 5—Exports original statistics collected based on the 7-tuple elements and does not support the NetStream aggregation data export. The packet format is fixed and cannot be extended.

·     Version 8—Supports the NetStream aggregation data export. The packet format is fixed and cannot be extended.

·     Version 9—Based on a template that can be configured according to the template formats defined in RFCs. Version 9 supports exporting the NetStream aggregation data and collecting statistics about BGP next hop and MPLS packets.

·     Version 10—Similar to version 9. The difference between version 9 and version 10 is that version 10 export format is compliant with the IPFIX standard

NetStream data export formats

The NDE encapsulates the collected flow data in UDP packets and sends the packets to the NetStream server. A UDP packet can carry multiple collected statistic entries of which the format is decided by the NDE. The NetStream server analyzes the received data based on the data export formats. As shown in Figure 3, the NetStream export format is distinguished by the Version field in the packet header.

Figure 3 NetStream packet format

 

The IPv4 NetStream data export format supports version 5, version 8, version 9, and version 10. The IPv6 NetStream data export format supports version 9 and version 10.

Version 5 format

Figure 4 shows the packet header for the version 5 format, where:

·     Version: Indicates the data export version. The value is 5 for version 5.

·     Count: Indicates the number of flows recorded in the packet.

·     System Up Time: Indicates the packet generation time in milliseconds.

·     Unix Secs: Indicates the system up time in seconds.

·     Unix Nsecs: Indicates the system up time in nanoseconds.

·     Flow Sequence: Indicates the flow sequence number.

·     Engine Type: Indicates the engine type for flow exchange. The value is the device type.

·     Engine ID: Indicates the engine slot number for flow exchange. The value is the slot number of the NetStream processing module.

·     Reserved: Indicates the field reserved for future use. The value is 0.

Figure 4 Version 5 header

 

Version 8 format

Figure 4 shows the packet header for the version 8 format, where:

·     Version: Indicates the data export version. The value is 8 for version 8.

·     Count: Indicates the number of flows recorded in the packet.

·     System Up Time: Indicates the packet generation time in milliseconds.

·     Unix Secs: Indicates the system up time in seconds.

·     Unix Nsecs: Indicates the system up time in nanoseconds.

·     Flow Sequence: Indicates the flow sequence number.

·     Engine Type: Indicates the engine type for flow exchange. The value is the device type.

·     Engine ID: Indicates the engine slot number for flow exchange. The value is the slot number of the NetStream processing module.

·     Aggregation: Indicates the aggregation type.

·     Aggregation Version: Indicated the aggregation version number. The value is 0x02.

·     Sampling Interval: Indicated the sampling interval. The value is 0.

·     Reserved: Indicates the field reserved for future use. The value is 0.

Figure 5 Version 8 header

 

Version 9 format

Packet header format

Figure 4 shows the packet header for the version 9 format, where:

·     Version: Indicates the data export version. The value is 9 for version 9.

·     Count: Indicates the number of FlowSet records (including template record and data record) in the packet.

·     System Up Time: Indicates the packet generation time in milliseconds.

·     Unix Secs: Indicates the system up time in seconds.

·     Package Sequence: Indicates the sequence number for the export packet. The value increases by 1 each time a packet is sent. The value can be used for NSC to verify whether an export packet is missing.

·     Source ID: Indicates the unique identifier for the data export device. The value length is four bytes.

¡     The first two bytes are reserved for future use. The value is 0.

¡     The third byte is Engine Type, which indicates the engine type for flow exchange. The value is the device type.

¡     The fourth byte is Engine ID, which indicates the engine slot number for flow exchange. The value is the slot number of the NetStream processing card.

Figure 6 Version 9 header

 

Packet format

The format is fixed for version 5 and version 8 and cannot be changed. For version 9, you can customize different templates as needed.

A packet exported by using the version 9 format consists of a template packet and data packet. The template packet is used as a reference to explain flow information in the data packet. The NSC parses received data based on the templates.

Figure 7 Template packet

 

Figure 8 Data packet

 

1.     Template packet.

A template packet consists of the packet header and one or more template flow sets. Figure 9 shows the format of a template flow set, where:

¡     FlowSet ID: Indicates a template flow set of which the value is fixed at 0.

¡     Length: Indicates the total length of fields in the template flow set in bytes.

¡     Template ID: Identifies different templates.

¡     Field Count: Indicates the number of fields recorded by the template. A template flow set can contain multiple template records and this field displays the number of fields in certain template for locating the next template record.

¡     Field Type: Defines the type of flow information in the data record that uses the template. You can customize the value like Source AS and Destination AS.

¡     Field Length: Indicates the total length of fields in the data record that uses the template in bytes. For example the field length is four bytes for the Source AS and Destination AS fields.

Figure 9 Template flow set packet

 

2.     Data packet.

A data packet consists of the packet header and one or more data flow sets. Figure 10 shows the format of a data flow set, where:

¡     FlowSet ID: Indicates the template ID. The NSC associates data with the template through a template ID for data analysis.

¡     Length: Indicates the total length of the data flow set in bytes.

¡     Record-Filed Value: Indicates the data flow information. The type and length of these fields are defined in the Field Type and Field Length fields of the template flow set.

Figure 10 Data flow set packet

 

3.     Relationship between the template flow set and data flow set.

As shown in Figure 11, the template flow set for a data flow set is identified through the Template ID field. If the template ID is the same for the two packets, the two packets can be associated successfully and you can identify the corresponding type and length for each data flow. For example:

¡     In the template flow set, the Field 1 Type is Source IPv4 address and the Field 1 Length is 4.

¡     In the data flow set, the Record-Field 1 Value is 137.66.8.1.

¡     In the data flow set, 137.66.8.1 is parsed as a source IP address of which the length is four bytes.

Figure 11 Relationship between the template flow set and data flow set

 

Version 10 format

The version 10 format is similar to the version 9 format. The main difference is that fields supported by the packet header and the template are different. Version 10 supports more fields.

Figure 4 shows the packet header for the version 10 format, where:

·     Version: Indicates the data export version. The value is 10 for version 10.

·     Length: Indicates the length of the exported packet in bytes, which includes the packet header length.

·     Export Time: Indicates the time at which the flows are exported from the NDE.

·     Sequence Number: Indicates the serial number of the exported packet. The sequence number increases 1 each time a packet is exported. The NSC can use this field to identify whether an exported packet is missing.

·     Observation Domain ID: The observation domain ID is the same for flows exported from the same device.

Figure 12 Version 10 header

 

Export format comparison

Table 3 Advantages and disadvantages of different export versions

Version	Supported flows	Benefits	Disadvantages
V5	Traditional data	·     Exports detailed flow information to the NSC with abundant and comprehensive fields. ·     NDE has light loads.	·     Packet format is fixed and is hard to be extended. ·     The data scale is large and the NSC cannot save the data for a long time. ·     The NSC and NDA have heavy loads for parsing and analysis. ·     The NSC/NDA software upgrade is required to adapt to NDE-exported packet changes when new export information is extended.
V8	Aggregate data	·     Reduces network bandwidth usage. ·     Supports configuration of different aggregation methods based on statistics collection objects. ·     Supports expansion of new aggregation method.	·     The packet format is fixed and is hard to be extended. ·     NDE. ·     The NSC/NDA software upgrade is required to adapt to NDE-exported packet changes when new export information is extended.
V9	Traditional data and aggregate data	·     Uses templates to realize flexible and extensible packet formats. ·     The packet format is extensible and you can add new statistics collection objects without changing the export packet format. ·     Software upgrade is not required for the NSC/NDA to adapt to NDE-exported packet changes.	N/A
V10	Traditional data and aggregate data	·     Uses templates to realize flexible and extensible packet formats. ·     The packet format is extensible and you can add new statistics collection objects without changing the export packet format. ·     Software upgrade is not required for the NSC/NDA to adapt to NDE-exported packet changes. ·     Supports more fields in the template than version 9.	N/A

 

NetStream mirroring

NetStream mirroring copies packets that pass through the device to a NetStream module for collecting traffic statistics. The forwarding performance of the device is not affected.

NetStream mirroring has the following types:

·     NetStream flow mirroring—Copies the packets that meet specific QoS match criteria to a NetStream module.

·     NetStream port mirroring—Copies the packets passing through an interface to a NetStream module

NetStream filtering

NetStream filtering uses an ACL to identify packets. Whether NetStream collects data for identified packets depends on the action in the matching rule.

·     NetStream collects data for packets that match permit rules in the ACL.

·     NetStream does not collect data for packets that match deny rules in the ACL.

Typical network applications

Configuring NetStream for traffic accounting

The marketing department and technology department of a company are connected to the Internet through Device.

Configure NetStream on the interface of Device that connects to the Internet to collect incoming and outgoing traffic statistics. Device will report the statistics to the NetStream server for further analysis. This helps the company staff to monitor the network status of the two departments and provide basis for accounting by department.

Figure 13 Network diagram

 

Configuring NetStream for network planning

A campus is divided into three teaching areas and assigned with the same network bandwidth. However, new services such as VoIP, P2P, and IPTV make the original bandwidth hard to meet the daily requirements.

Configure NetStream on the interface that connects Gateway to the Internet to collect incoming and outgoing traffic statistics. Gateway will report the statistics to the NetStream server for further analysis. This helps obtain the bandwidth usage details, plan and assign network resources, and monitor network traffic.

Figure 14 Network diagram