Netify DPI - Network Policies

This document provides details on how the Netify DPI agent converts Layer 7 application/protocol data into something a native Layer 3 network stack can understand. This makes it possible to implement next generation Network Policies, for example:

  • Blocking evasive VPN protocols
  • Logging weak encryption and ciphers
  • Shaping the BitTorrent protocol with a QoS engine
  • Implementing a MultiWAN routing policy based on application latency requirements

The Problem

Problem: we need to get Zoom and the all-important Fortnite network traffic through with low latency and shuffle less time-sensitive traffic like BitTorrent and Netflix to the "I don't care about latency" part of the network.

In the modern network, these application/protocol features require tapping into Layer 7 to get the information we need. At Layer 2/3 (MAC addresses, IPs, UDP/TCP port numbers), Fortnite and Zoom can look like plain old HTTPS traffic on a common cloud platform like Amazon AWS. It’s hard and often impossible to consistently differentiate this kind of traffic at Layer 3. And tricky protocols like BitTorrent, Wireguard, and others are all but impossible to manage at Layer 3.

The Linux kernel, traffic control tools (tc), iptables, and other engines do not speak native Layer 7, and nor should they. But that leaves us with a key problem:

How do we get Layer 7 information into a format that standard Layer 3 engines can understand?
OS Model

Hello Netify DPI!

Let’s go through an example where a network administrator would like to put Zoom traffic into a high priority QoS bucket. The first step is handled by the Netify DPI Agent - netifyd. The agent ingests network traffic on one end and publishes two different data sets:

  • Network Policies
  • Data Streams

The agent does this analysis in the first few packets of a connection and makes a determination. There are all sorts of metadata available in the Data Stream, but we're going to focus on the Network Policy required for the Zoom application classification.

Netify's Network Policy Engine will continuously update the client IP, server IP, and server port involved in every Zoom conversation. Where is this information stored? Read on.

Netify continuously updates Zoom lookup table:

192.168.4.189   52.216.110.177   8801 
192.168.4.189   52.216.110.178   8801
192.168.4.189   193.122.215.48   443 
...
Excerpt from Netify DPI data stream:

"flow": {
  "detected_application_name": "10228.netify.zoom",
  "local_ip": "192.168.4.189",
  "local_port": 59209,
  "other_ip": "52.216.110.177",
  "other_mac": "00:90:fb:29:ca:ba",
  "other_port": 8801,
  ... snip ...

IP Sets

There are many ways to create a shared IPs/port lookup table, but the Linux kernel-level IP Set module fits the bill. If you are already familiar with IP Sets, please feel free to jump to the next section. If you are relatively new to IP Sets, here's a very quick introduction.

On most flavors of Linux, you can use the ipset command to play around with IP sets from the command line. You may need to install an additional Linux package as it's not always available on a default Linux installation. Once installed, you can create an IP set on a Linux system like so:

# sudo ipset create NETIFY_ZOOM "hash:ip,port,ip" timeout 600

We’re using the "ip,port,ip" IP set list type - that’s what Netify DPI needs for our Zoom lookup table. For added zing, we have added a 600-second timeout. In other words, an entry in the list will self-destruct after 600 seconds (10 minutes... I did the math). Now let's add our network record that would mimic the Zoom flow entry -- a client connection (192.168.4.189) going to what was determined to be a Zoom server IP: 52.216.110.177. Oh, and Zoom uses non-standard HTTPS ports - it was port 8801 in the JSON metadata (see above) coming out of the Netify DPI Agent.

# sudo ipset add NETIFY_ZOOM 52.216.110.177,8801,192.168.4.18
# sudo ipset list NETIFY_ZOOM
... snip ...
Members:
52.216.110.177,tcp:8801,192.168.4.18 timeout 593

Of course, it’s usually the Netify DPI Agent or Netify Firewall Agent that is maintaining this IP Set lookup table. Regardless, now you know what's going on behind the scenes.

Layer 3 Tools

The last step is to integrate Layer 3 tools with our IP Set information. Some examples:

  • tc - traffic control and QoS
  • ip route/rule - multiWAN policies via marks
  • iptables/nftables - firewall management via marking

IP sets are accessible in both userland and inside the kernel, so there is a lot of flexibility. Going back to our Zoom scenario, an example firewall rule might look like this:

sudo iptables -t mangle -A PREROUTING -m set --match-set NETIFY_ZOOM dst,dst,src -j MARK --set-mark 0x85

Don't get too hung up on the details of the firewall rule. It's just an example of how it's possible to use native iptables tools to classify Layer 7 traffic. The iptables system will mark Zoom traffic with mark 0x85. This can then be used in standard Linux traffic control rules to manage bandwidth/QoS, send the traffic out a specific WAN interface, or do other Linux things.

A Linux-based Layer 3 firewall now has access to Layer 7 detections. Ta-da!

Further Reading

Integration and Custom Solutions

Do you have any questions about integration, APIs or custom development?

Contact Us