Network Policies

Netify DPI - Network Policy Datasets

This document provides details on how the Netify DPI agent converts Layer 7 application and protocol data into something a native Layer 3 network stack can understand. This makes it possible to implement datasets for next-generation network policy engines, for example:

  • Blocking evasive VPN protocols
  • Shaping the BitTorrent protocol with a QoS engine
  • Implementing a MultiWAN routing policy based on application latency requirements

Netify DPI also provides Live Data Streams to interact with the available network metadata. Data streams make it possible to, for example, detect and log weak encryption and ciphers used in a TLS/SSL connection.

The Problem

Problem: we need to get Zoom and the all-important Fortnite network traffic through with low latency, and then shuffle less time-sensitive traffic like BitTorrent and Netflix to the "I don't care about latency" part of the network.

In the modern network, these application/protocol features require tapping into Layer 7 to get the information we need. At Layer 2/3 (MAC addresses, IPs, UDP/TCP port numbers), Fortnite and Zoom can look like plain old HTTPS traffic on a common cloud platform like Amazon AWS. It’s hard and often impossible to differentiate this traffic at Layer 3. And tricky protocols like BitTorrent, Wireguard, and others are essentially impossible to manage at Layer 3.

The Linux kernel, traffic control tools (tc), iptables, nftables, and other engines do not speak native Layer 7, and nor should they. But that leaves us with a crucial problem:

How do we get Layer 7 information into a format that standard Layer 3 tools can understand?
OS Model

Hello Netify DPI!

Let’s go through an example where a network administrator would like to put Zoom traffic into a high-priority QoS bucket. The first step is handled by the Netify DPI Agent - netifyd. The agent ingests network traffic on one end and publishes different data sets which can be enabled/disabled to meet your needs:

  • Data streams
  • Statistics
  • IP sets
  • nftables sets
  • Connection tracking labels

The agent does the analysis in the first few packets of a connection and makes a determination. The Data Stream provides a large set of metadata, but we're going to focus on a Layer 3 Dataset required for the Zoom application classification. The IP sets, nftables, and connection tracking features in the Flow Actions Plugin can be used for the job, but we are going to focus on the IP Sets solution in this example. This plugin will continuously update the client IP, server IP, and server port involved in every Zoom conversation. Where is this information stored? Read on. 

Netify continuously updates the Zoom lookup table:

192.168.4.189   52.216.110.177   8801 
192.168.4.189   52.216.110.178   8801
192.168.4.189   193.122.215.48   443 
...

Excerpt from Netify DPI data stream:

"flow": {
  "detected_application_name": "10228.netify.zoom",
  "local_ip": "192.168.4.189",
  "local_port": 59209,
  "other_ip": "52.216.110.177",
  "other_mac": "00:90:fb:29:ca:ba",
  "other_port": 8801,
  ... snip ...

IP Sets

There are many ways to create a shared IPs/port lookup table, but the Linux kernel-level IP Set module fits the bill. If you are already familiar with IP Sets, please feel free to jump to the next section. If you are relatively new to IP Sets, here's a very quick introduction.

On most flavors of Linux, you can use the ipset command to play around with IP sets from the command line. You may need to install an additional Linux package as it's not always available on a default Linux installation. Once installed, you can create an IP set on a Linux system like so: 

# sudo ipset create netify.zoom "hash:ip,port,ip" timeout 600

We’re using the "ip,port,ip" IP set list type - that’s what Netify DPI needs for our Zoom lookup table. For added zing, we have added a 600-second timeout. In other words, an entry in the list will self-destruct after 600 seconds (10 minutes... I did the math). Now let's add our network record that would mimic the Zoom flow entry -- a client connection (192.168.4.189) going to what was determined to be a Zoom server IP: 52.216.110.177. Oh, and Zoom uses non-standard HTTPS ports - it was port 8801 in the JSON metadata (see above) coming out of the Netify DPI Agent. 

# sudo ipset add netify.zoom 52.216.110.177,8801,192.168.4.18
# sudo ipset list netify.zoom
... snip ...
Members:
52.216.110.177,tcp:8801,192.168.4.18 timeout 593

Of course, it’s usually Netify's IP Sets engine that is maintaining this IP Set lookup table. You can also accomplish similar behavior using connection tracking labels and nftables. Regardless, now you know what's going on behind the scenes.

Layer 3 Tools

The last step is to integrate Layer 3 tools with our IP Set information. Some examples:

  • tc - traffic control and QoS
  • ip route/rule - multiWAN policies via marks
  • iptables/nftables - firewall management

IP sets are accessible in both userland and inside the kernel, so there is a lot of speed and flexibility. Going back to our Zoom scenario, an example firewall rule might look like this: 

sudo iptables -t mangle -A PREROUTING -m set --match-set netify.zoom dst,dst,src -j MARK --set-mark 0x85

Don't get too hung up on the details of the firewall rule. It's just an example of how it's possible to use native iptables tools to classify Layer 7 traffic. The iptables system will mark Zoom traffic with mark 0x85. This can then be used in standard Linux traffic control rules to manage bandwidth/QoS, send the traffic out a specific WAN interface, or do other Linux things.

A Linux-based Layer 3 system now has access to Layer 7 detections. Ta-da!

Further Reading

Related Information

Plugins and Addons

Netify Agent



Getting Started



Plugins and Addons



Netify Tools



Open Source DPI

Evaluate Netify DPI

Do you want to get started with evaluating Netify DPI? Request the Integrators Kit today.

Integrators Kit