Overview

This page contains software able to generate synthetic traffic, or to dump and reply (even full-payload) packet traces at 10Gbps+ speed on off-the-shelf hardware. The 10Gbps+ capture software is based on the DPDK stack, while the traffic generator (sythetic vs reply) software is based on the PacketShader stack.

This effort stem from our work on high performance traffic classification [IMC-12], which we would ask you to kindly cite shall you use our tools. While our main goal in [IMC-12] and [TMA-14b] was to design a high performance traffic classification and flow management engines respectively, however to stress test the system we needed an efficient packet injection tool, able to operate at 10Gbps and beyond speed, on common hardware.

We have so far brewed two tools that are based on PacketShader and on DPDK respectively: since they can be useful beyond the focus of our previous work (on traffic classification [IMC-12], management [TMA-14b] and analysis (stay tuned)), we release them as open source software to the community in this page.

References

  1. [COMMAG-17] Trevisan, Martino , Finamore, Alessandro, Mellia, Marco, Munafo, Maurizio and Rossi, Dario, Traffic Analysis with Off-the-Shelf Hardware: Challenges and Lessons Learned . IEEE Communication Magazine, 2017.
  2. [DPDKStat-16] Trevisan, Martino , Finamore, Alessandro, Mellia, Marco, Munafo, Maurizio and Rossi, Dario, DPDKStat: 40Gbps Statistical Traffic Analysis with Off-the-Shelf Hardware . In Tech. Rep., 2016.
  3. [TMA-14b] Georges Nassopulos, Dario Rossi, Francesco Gringoli, Lorenzo Nava, Maurizio Dusi and Pedro Maria Santiago del Rio, Flow management at multi-Gbps: tradeoffs and lessons learned . In Traffic Measurement and Analysis (TMA), pages 1-14 , Apr 2014.
  4. [IMC-12] P.M. Santiago del Río, D. Rossi, F. Gringoli, L. Nava, L. Salgarelli and J. Aracil , Wire-Speed Statistical Classification of Network Traffic on Commodity Hardware . In ACM SIGCOMM Internet Measurement Conference (IMC), pages 65-72, November 2012.

DPDK tools

This part relates to the DPDK-based tools. Due to our focus on full-packet payload DPI, and flow-level reconstruction for traffic analysis and monitoring, we needed to process, store and reply packets at 10Gbps+.

While in our studies we reply traffic at 10Gbps (which requires to read from multiple SSD disks in parallel) and 40Gbps (requires to send multiple copies with same TCP payload but hacked IP headers), the whole system is not simply exportable a single ``click-and-run'' software package. Plans are to release software and system description at later time, and for the time being we start releasing the packet capture and replay software at GitHub

Download

You can find the most up-to-date release of our software, as well as instructions on their usage, on the respective GitHub page. This page reports a short but comprehensive guide on all tools that we released.

How to install

The DPDK-Dump tool uses DPDK v1.8.0. The tool requires a Linux kernel version >=2.6.3, and a number of other dependencies (that are described on GitHub for a Debian based distribution. A number of necessary configuration steps, whose fine-tuning may depend on your hardware are then necessary and are described in details at the respective GitHub pages for dump and reply tools:

  • Reserve a large number of hugepages to DPDK
  • Set CPU governors to performance mode
  • Bind the interfaces you want to use with DPDK drivers

How to capture and replay traffic, at a glance.

Once the tools are compiled and the system configured, you can start DPDK-Dump with the executable ./build/dpdk-dump and DPDK-Reply with ./build/dpdk-replay (root priviledges are needed in both cases). Unless otherwise specified, DPDK-Dump uses all the ports bound to the DPDK drivers; similarly, unless otherwise specified DPDK-reply uses all the port bound to DPDK drivers and will send the same traffic on them. More details about both tools are reported below or at their GitHub pages dump and replay.

How to capture traffic

Once the tools are compiled and the system configured, you can start DPDK-Dump with the executable ./build/dpdk-dump (root priviledges are needed). Unless otherwise specified, DPDK-Dump uses all the ports bound to the DPDK drivers. The tool offer the following parameters:

    ./build/dpdk-dump 
       -c COREMASK -n NUM [-w PCI_ADDR] --
       -w file [-c num_pkt] [-C max_size] 
       [-G rotate_seconds] [-W num_rotations] 
       [-B buffer_size]

Options have the following meaning (note that parameters before -- are DPDK-related: see DPDK knowledgbase for further explanation):

  • COREMASK: The cores where to bind the program. It needs 2 cores
  • NUM: Number of memory channels to use (2 or 4).
  • PCI_ADDR: The port(s) where to capture. If not present, it captures from every port.
  • file: output file in pcap format
  • num_pkt: quit after saving num_pkt packets
  • max_size: quit after saving an amount of packets equal to max_size (in KB).
  • rotate_seconds: rotate dump files each rotate_seconds. Progressive numbers edded to file name.
  • num_rotations: maximum number of rotations to do. If 0 it quits after rotate_seconds.
  • buffer_size: Internal buffer size. Default is 1e6 packets.

Here some example of command lines:

To start a capture unbounded in time to the file capture.pcap:

    sudo ./build/dpdk-dump -c 0x03 -n 4 -- -w capture.pcap

To start a capture, rotating capture file every 60 seconds and stopping after half an hour (30 files):

    sudo ./build/dpdk-dump -c 0x03 -n 4 -- -w capture.pcap -G 60 -w 30

To start a capture just on the device with the specified PCI address (the first -w is a parameter of DPDK enviroment, the second of DPDK-Dump).

    sudo ./build/dpdk-dump -c 0x03 -n 4 -w 01:00.0 -- -w capture.pcap

The system approximately each one seconds prints statistics about capturing performances (one line per port).

    PORT:  0 Rx: 102669 Drp: 0 Tot: 102669 Perc: 0.000%
    PORT:  1 Rx: 88060  Drp: 0 Tot: 88060  Perc: 0.000%
    -------------------------------------------------
    TOT:     Rx: 190729 Drp: 0 Tot: 190729 Perc: 0.000%

Have fun!

How to reply traffic

To start DPDK-Replay just execute ./build/dpdk-replay. If not specified DPDK-replay uses all the port bound to DPDK drivers and will send the same traffic on them. Root priviledges are needed. The are few parameters:

     ./build/dpdk-dump 
          -c COREMASK -n NUM [-w PCI_ADDR] -- 
          -f file [-s sum] [-R rate] [-B buffer_size]
          [-C max_pkt] [-t times] [-T timeout]

The parameters have this meaning:

  • COREMASK: The core where to bind the program. It needs 2 cores
  • NUM: Number of memory channels to use. It's 2 or 4.
  • PCI_ADDR: The port(s) where to send. If not present, it sends the same traffic to every port.
  • file: input file in pcap format
  • sum: when using multiple interfaces, sum this number to IP addresses each time a packet is sent on the interface.
  • rate: rate to send on the wire. If not specified it sends at wire speed.
  • buffer_size: Internal buffer size. Default is 1 Milion packets.
  • max_pkt: quit after sending max_pkt packets on every interface.
  • times: how many times to send a packet on each interface. Default is 1.
  • timeout: timeout of the replay in seconds. Quit after it is reached.

The parameters before -- are DPDK enviroment related. See its guide for further explaination.

Here some example of command lines:

To start replaying capture.pcap on all DPDK interfaces at wire speed:

    sudo ./build/dpdk-replay -c 0X02 -n 4 -- -f /mnt/traces/capture.T

To start replaying capture.pcap on the two specified interfaces at 8 Gbps on each link:

    sudo ./build/dpdk-replay -c 0X02 -n 4 -w 2:00.0 -w 2:00.1 -- -f /mnt/traces/capture.pcap -r 8

To start replaying capture.pcap on the two specified interfaces at wire speed. On the first interface, 1 is added to source and destination IP address, on the second one 2 is added:

    sudo ./build/dpdk-replay -c 0X02 -n 4 -w 2:00.0 -w 2:00.1 -- -f /mnt/traces/capture.pcap -s 1

The system approximately each one seconds prints statistics about its performances.

    Rate:   10.000Gbps     1.319Mpps [Average rate:    10.000Gbps     1.319Mpps], Buffer:   89.998% Packets read speed:    0.333us

PacketShader tools

This parts relate to the the High-speed Pcap (HPCAP) tools based on PacketShader (note: the original page is no longer actively maintained). If you're looking for our traffic classification and management emulator instead, please have a look at this page).

HPCAP is able to send both synthetic and real pcap traces up to 14.2 Mpps and 10 Gbps. Note that, due to our focus on ``early classification'' in [IMC-12], no full-packet payload replay has been tested. Note that CPU and memory affinity is crucial (to avoid or reduce cache misses) in order to get line-rate. Please refer to our papers for more technical details.

Download

hpcap-0.1.tar.gz (148 downloads so far)

The HPCAP tools use PacketShader I/O packet engine version 0.1. HPCAP can be also thought as an example of use of such high speed driver.

To use HPCAP, the first step is to download PacketShader. Notice that PacketShader (and therefore our tools) is not supported on kernel versions older than 2.4.

How to install

In this example, we will install a PacketShader driver with 4 RSS queues. First, remove the default ixgbe driver and replace it with PacketShader driver:

 rmmod ixgbe
 cd io_engine/driver
 make clean
 make
 ./install.py 4 4

Now, two interfaces per 82599 NIC appears, named xge0 and xge1 (check with ifconfig). As a last step, compile the PacketShader library.

 cd io_engine/lib
 make clean
 make

Finally, copy hptrac to samples folder in PacketShader io_engine. Congratulations, you're done!

How to generate synthetic traffic

In this example, we generate a stream of TCP segments, with incremental IP addresses and TCP ports (in interface xge0, using 4 queues, 5 packets per flow, 64B per packet):

 cd io_engine/samples/hpcap_gen
 make
 sudo ./hpcap_gen -i xge0 -q 4 -c 4096 -p 64 -f 5

How to replay a real pcap trace

In this example, we reply a pcap trace over interface xge0, using 4 RSS queues. To stress test our classification system, we capping the flow length to 5 packets and also cap the packet length to 64B

 cd io_engine/samples/hpcap_replay
 make
 sudo ./hpcap_replay -i xge0 -q 4 -c 1024 -t trace.pcap -p 64

Publications

  1. [COMMAG-17] Trevisan, Martino , Finamore, Alessandro, Mellia, Marco, Munafo, Maurizio and Rossi, Dario, Traffic Analysis with Off-the-Shelf Hardware: Challenges and Lessons Learned . IEEE Communication Magazine, 2017.
  2. [DPDKStat-16] Trevisan, Martino , Finamore, Alessandro, Mellia, Marco, Munafo, Maurizio and Rossi, Dario, DPDKStat: 40Gbps Statistical Traffic Analysis with Off-the-Shelf Hardware . In Tech. Rep., 2016.
  3. [TMA-14b] Georges Nassopulos, Dario Rossi, Francesco Gringoli, Lorenzo Nava, Maurizio Dusi and Pedro Maria Santiago del Rio, Flow management at multi-Gbps: tradeoffs and lessons learned . In Traffic Measurement and Analysis (TMA), pages 1-14 , Apr 2014.
  4. [IMC-12] P.M. Santiago del Río, D. Rossi, F. Gringoli, L. Nava, L. Salgarelli and J. Aracil , Wire-Speed Statistical Classification of Network Traffic on Commodity Hardware . In ACM SIGCOMM Internet Measurement Conference (IMC), pages 65-72, November 2012.