IP Sets

Mark BitTorrent Traffic - IP Sets

Introduction

This example demonstrates how to use Netify's IP Sets engine to mark BitTorrent, YouTube, and Social Media applications using iptables. This solution uses three tools to implement traffic prioritization:

  • The Netify IP Sets engine in the Flow Actions Plugin
  • The Linux kernel IP sets feature
  • iptables to mark traffic

Getting Started

The IP Sets Integration is a part of the Netify Flow Actions Plugin - a premium add-on to the Netify Agent.

IP Sets can be used to implement bandwidth, firewall, and QoS solutions on routers, firewalls, and gateways. The Netify DPI IP Sets integration provides a way to create high-speed IP sets based on any Netify deep packet inspection flow criteria. These sets can then be used to block, shape, mark, or perform other actions to network traffic.

You can also review alternative options for taking actions on DPI flow data - see the nftables and connection tracking (CT) labels integrations for details.

Prerequisites
- Installing Netifyd
- Network Interface Configuration
- Flow Actions Plugin

Netify Network Policy Engine

IP Sets - Primer

If you are already familiar with IP Sets, please feel free to jump to the next section. If you are relatively new to IP Sets, here's a very quick introduction.

There are many ways to create a shared IPs/port lookup table, but the Linux kernel-level IP Set module fits the bill. On most flavors of Linux, you can use the ipset command to play around with IP sets from the command line. You may need to install an additional Linux package as it's not always available on a default Linux installation. Once installed, you can create an IP set on a Linux system like so: 

# sudo ipset create netify.zoom "hash:ip,port,ip" timeout 600

We’re using the "ip,port,ip" IP set list type - that’s what Netify DPI needs for our Zoom lookup table. For added zing, we have added a 600-second timeout. In other words, an entry in the list will self-destruct after 600 seconds (10 minutes... I did the math). Now let's add our network record that would mimic the Zoom flow entry -- a client connection (192.168.4.189) going to what was determined to be a Zoom server IP: 52.216.110.177. Oh, and Zoom uses non-standard HTTPS ports - it was port 8801 in the JSON metadata (see above) coming out of the Netify DPI Agent. 

# sudo ipset add netify.zoom 52.216.110.177,8801,192.168.4.18
# sudo ipset list netify.zoom
... snip ...
Members:
52.216.110.177,tcp:8801,192.168.4.18 timeout 593

Of course, it’s usually Netify's IP Sets engine that is maintaining this IP Set lookup table. You can also accomplish similar application groups using connection tracking labels and nftables. Regardless, now you know what's going on behind the scenes.

Example Configuration

Introduction

The netify-proc-flow-actions.json JSON configuration file provides a starting point for implementing our solution. See sidebar for the example configuration.

For this example, we use basic configurations to demonstrate a core feature. The Netify Flow Actions documentation provides more detailed information on the various parameters. Notably, the Flow Actions Plugin provides a very powerful expression language for pinpointing network traffic.

Actions

The actions block defines the traffic that needs to be identified. In our example, we have created three different actions:

  • BitTorrent protocol traffic
  • YouTube application traffic
  • All Social Media applications, except Twitter

Target

The targets block defines which engine to use for the flow action. In our example, we are creating IP Sets.

Target Defaults

The target_defaults section in the configuration provides baseline settings for the IP Sets target. This parameter can be used to fine-tune various policies for the IP Sets engine. 

{
  "version": 1,
  "actions": {
    "bittorrent": {
      "criteria": "protocol == 'bittorrent';",
      "targets": [
        "ipset.bittorrent"
      ]
    },
    "youtube": {
      "criteria": "app == 'youtube';",
      "targets": [
        "ipset.youtube"
      ]
    },
    "social": {
      "criteria": "category == 'social-media';",
      "targets": [
        "ipset.social"
      ],
      "exemptions": [
        "app == 'netify.twitter';"
      ]
    }
  },
  "targets": {
    "ipset.bittorrent": {
      "target_type": "ipset",
      "set_name": "nfa.bittorrent"
    },
    "ipset.youtube": {
      "target_type": "ipset",
      "set_name": "nfa.youtube"
    },
    "ipset.social": {
      "target_type": "ipset",
      "set_name": "nfa.social"
    }
  },
  "target_defaults": {
    "ipset": {
      "interface": "*",
      "type": "hash:ip,port,ip",
      "ttl": 120,
      "managed": true,
      "flush_on_create": true,
      "flush_on_destroy": true
    }
  },
  "exemptions": [
    "192.168.1.1",
    "protocol == 'dns';"
  ]
}

Example In Action

The ipset command can be used to view a list of active entries. In the adjacent sidebar, attempted BitTorrent connections are shown. 

# ipset list
Name: nfa.bittorrent.v4
Type: hash:ip,port,ip
Revision: 5
Header: family inet hashsize 1024 maxelem 65536 timeout 120 skbinfo
Size in memory: 2696
References: 0
Number of entries: 13
Members:
101.100.139.138,udp:51413,192.168.55.140 timeout 117
60.115.123.115,udp:55753,192.168.55.140 timeout 115
62.210.124.230,udp:51413,192.168.55.140 timeout 119
71.150.226.54,udp:51413,192.168.55.140 timeout 115
37.221.197.43,udp:51413,192.168.55.140 timeout 115
45.76.32.64,tcp:51413,192.168.55.140 timeout 115

Layer 3 Tools

The last step is to integrate Layer 3 tools with our IP Set information. Some examples:

  • tc - traffic control and QoS
  • ip route/rule - multiWAN policies via marks
  • iptables/nftables - firewall management

IP sets are accessible in both userland and inside the kernel, so there is a lot of speed and flexibility. Going back to our BitTorrent scenario, an example firewall rule might look like this: 

sudo iptables -t mangle -A PREROUTING -m set --match-set nfa.bittorrent.v4 dst,dst,src -j MARK --set-mark 0x85

Don't get too hung up on the details of the firewall rule. It's just an example of how it's possible to use native iptables tools to classify Layer 7 traffic. The iptables system will mark BitTorrent traffic with mark 0x85. This can then be used in standard Linux traffic control rules to manage bandwidth/QoS, send the traffic out a specific WAN interface, or do other Linux things.

A Linux-based Layer 3 system now has access to Layer 7 detections. Ta-da!

Documentation


Netify Agent Plugins


Evaluate Netify DPI

Do you want to get started with evaluating Netify DPI? Request the Integrators Kit today.

Integrators Kit