Getting Started
Netify DPI - How It Works
The open-source Netify DPI Agent does one thing and one thing very well: network analysis using deep packet inspection (DPI). The DPI Agent passively captures network traffic and produces both network metadata streams and high-speed Linux datasets. This solution can be used to:
- Prioritize Zoom traffic
- Put BitTorrent traffic into a lower-priority QoS bucket
- Block devices from using weak security ciphers on the network
- Detect devices on the network
- Measure application bandwidth and health by IP/subscriber
- Provide zero-rating on Netflix traffic
- Send network metadata to an in-house message queue system
In the Netify DPI Agent developer documentation, we go through the basic setup for delivering the above solutions. But first, let's take a look at the big-picture problem.
The Problem
Problem: we need to get Zoom and the all-important Fortnite network traffic through with low latency and then shuffle less time-sensitive traffic like BitTorrent and Netflix to the "I don't care about latency" part of the network.
In the modern network, these application/protocol features require tapping into Layer 7 to get the necessary information. At Layer 2/3 (MAC addresses, IPs, UDP/TCP port numbers), Fortnite and Zoom can look like plain old HTTPS traffic on a common cloud platform like Amazon AWS. Differentiating this traffic using IPs and ports is complex and often impossible. Tricky protocols like BitTorrent, Wireguard, and others are impossible to manage at Layer 3.
The Linux kernel, traffic control tools (tc), iptables, nftables, and other engines do not speak native Layer 7, and nor should they. But that leaves us with a crucial problem:
Solution: Hello Netify DPI!
Netify DPI enters the chat. The DPI engine provides several ways to interact with Layer 7 data using Layer 3 tools and formats. Before diving into the details below, stepping back and understanding the Netify DPI workflow is essential. There are three stages:
- Inputs - the source of the network data
- Processors - processor plugins convert Layer 7 traffic into Layer 3 formats
- Outputs - output plugins can export the data to a log, socket, or message queue
Let's look at each step of the workflow in more detail.
Netify DPI Workflow
Inputs
The agent ingests network traffic from what we call an Input. The input is either:
- Network interfaces, or
- Packet capture files
Most Netify Agent deployments are configured to capture live traffic from network interfaces, but capture files can be handy for testing, development, and offline processing.
Netify DPI ingests the network traffic from the inputs, examines the first 1 to 32 packets of a connection, and performs deep packet analysis. Subsequent packets are tracked for bandwidth statistics and various key performance metrics.
Inputs
Traffic is captured off live network interfaces via a number of different drivers: pcap, tpacket v3, and NFQ.
Processors
With the inputs configured, the agent uses processor plugins to transform the analysis into various metadata and datasets. The metadata can be used for bandwidth statistics, cybersecurity analysis, and more. The datasets, on the other hand, can be used by standard Layer 3 tools to implement:
- QoS and QoE
- Firewalls
- Routing
There are three standard processor plugins available in Netify DPI - these are described below.
Flow Actions ProcessorThe Flow Actions Plugin provides a high-speed interface into firewall, QoS, and routing systems in Linux. In particular, the plugin produces three types of datasets:
Core ProcessorThe Core Plugin provides network metadata information in JSON format. Metadata includes:
- Application
- Protocol
- Extracted hostnames
- TLS ciphers
- Bandwidth statistics
- Performance indicators
- and many more
Third-party tools typically ingest this flow metadata to provide enhanced network intelligence solutions.
Aggregator ProcessorFlow data from Netify DPI generates a very large amount of data. However, the Aggregator Plugin provides a way to summarize this data more compactly. For example, the plugin can be configured to track total bandwidth usage by a particular application - an efficient way to offer zero-rating.
Processors
Processors plugins transforms the network traffic into metadata streams and actions.
Outputs
The final part of the Netify DPI agent workflow is outputs. The metadata from one or more of the Netify processors can be connected to any of the following outputs:
- Socket
- Message Queue
- Log
And yes, you can mix and match these plugins. For example, the flow metadata from the core processor can be sent to the socket for analysis while simultaneously sending aggregator processor bandwidth data to a message queue server.
Outputs
The data from the processors can be output to a number of different outputs:
Next Steps
If you want to start diving into a Netify DPI agent implementation, you can jump into the Installing Netifyd documentation. Alternatively, feel free to explore the various features implemented in Netify's plugin system.
Evaluate Netify DPI
Do you want to get started with evaluating Netify DPI? Request the Integrators Kit today.
Integrators Kit