Telemetry is the Art Behind Capturing Network Packet Data

In this context telemetry is remotely collecting data on a given network packet. That data can be where a given packet was at a specific point in time along with the abstracted metadata that represents the network flow the data belongs to. Knowing everything about your network data can enable you to tune application performance while also securing your network. Solarflare provides several different methods for capturing network packet telemetry directly from the card itself, through a performance SolarCapture (SCP) driver or a complete SolarCapture Appliance.

Solarflare NIC ASIC

 
Solarflare NIC ASIC

XtremePacket Engine

Rules based action on incoming and outgoing packets can be time stamped, counted, cloned, dropped, filtered and translated

L2 Switch

Incoming and outgoing packets can switched and Intelligently steered to a collection of cores on the host

 

 

 

SolarCapture Appliance 

The SolarCapture Appliance builds on SCP by creating a highly-tuned lossless collection platform to record and index network traffic. Network packets are indexed on capture, and then the index and raw data are written to disk in an extremely efficient manner to ensure lossless packet capture to disk at both 10GbE and 40GbE speeds. Additionally, read requests of the packets stored on disk are interleaved in such a way so that they will never impact the appliance’s lossless capture capability. From the graphical user interface (GUI) an administrator can see a variety of capture statistics, as well as request capture files that represent the output of user driven queries. These libpcap files can then be passed to other end user applications running on the server for more detailed analysis.

SolarCapture (SCP) Libpcap Driver

SCP is a kernel bypass performance tuned replacement for Libpcap. It is designed to provide a high rate of packet capture to memory, in the 10s of millions of packets per second. SCP supports three operating modes: sniff, steal or both. In sniff mode packets are delivered normally to applications or the network, but a clone of each packet, both transmitted and received packets, is sent to SCP for processing. In steal mode SCP receives all the packets coming into an interface. Finally, both modes can be engaged so that two parallel applications can analyze the same packet streams. To facilitate this SCP has a feature called clustering, which enables the creation of clusters of cores that can then be attached to capture instances for processing. SCP supports Berkley Packet Filter (BPF) which can be applied to capture so that SCP only receives the packets requested while all others follow their normal path.