Flow Actions Processor
Netify Flow Actions Processor Plugin
- Introduction
- License
- Installation
- Configuration
- Flow Criteria / Expression Engine
- Integration
- Examples
- Next Steps
Introduction
The Flow Actions Plugin for the Netify DPI Agent applies user-defined criteria to new flow detections in near real-time. Matching flows are passed to one or more Action Targets for further processing.

The main purpose of the Flow Actions plugin is to provide a high-speed interface to other Operating System components that can empower policy enforcement, traffic control, and/or network auditing/logging applications.
The plugin provides a modular design where target types can be disabled at compile time to provide the smallest footprint possible. The Actions, Targets, and global settings are configured by a JSON file which can be reloaded at any time by sending a reload (HUP) signal to the main Netify Agent process. Action criteria are configured using either a simple or advanced syntax. Simple syntax selects flows by application, protocol, or category IDs or tags only. Advanced syntax executes user-defined Flow Criteria expressions. These expressions can operate over all available flow metadata to precisely control how individual flows are evaluated.
The diagram illustrates the flow execution path and the relationships between Agent, Plugin (Action Criteria/Targets), and Operating System components.
License
Netify Flow Actions Plugin is a proprietary plugin requiring a license. Please contact us for details.
Installation
Netify plugins are packaged in the same workflow as the agent and can usually be installed using a similar syntax that was implemented during the installation of the Netify agent. Exceptions to this rule occur when the plugin requires a Software License Agreement or if an upgrade or downgrade to a different version of the plugin is required.
Alma Linux
Alma Linux 9
The Netify Flow Actions plugin is a licensed (proprietary) plugin. Instructions to access this plugin will have been provided to you by a Netify representative. Typically, the package will be available via a web link with Basic authentication protection. Example:
cd /tmp/
wget https://client:secret@download.netify.ai/5/almalinux/9/stable/x86_64/netify-proc-flow-actions_1.0.66-1.os9.x86_64.rpm
apt install ./netify-proc-flow-actions_1.0.66-1.os9.x86_64.rpm
Alma Linux 8
The Netify Flow Actions plugin is a licensed (proprietary) plugin. Instructions to access this plugin will have been provided to you by a Netify representative. Typically, the package will be available via a web link with Basic authentication protection. Example:
cd /tmp/
wget https://client:secret@download.netify.ai/5/almalinux/8/stable/x86_64/netify-proc-flow-actions_1.0.66-1.os8.x86_64.rpm
yum install ./netify-proc-flow-actions_1.0.66-1.os8.x86_64.rpm
Debian
Debian 13 (Bookworm)
The Netify Flow Actions plugin is a licensed (proprietary) plugin. Instructions to access this plugin will have been provided to you by a Netify representative. Typically, the package will be available via a web link with Basic authentication protection. Example:
cd /tmp/
wget https://client:secret@download.netify.ai/5/debian/10/amd64/netify-proc-flow-actions_1.0.66-1_amd64.deb
apt install ./netify-proc-flow-actions_1.0.66-1_amd64.deb
Debian 11 (Bullseye)
The Netify Flow Actions plugin is a licensed (proprietary) plugin. Instructions to access this plugin will have been provided to you by a Netify representative. Typically, the package will be available via a web link with Basic authentication protection. Example:
cd /tmp/
wget https://client:secret@download.netify.ai/5/debian/10/amd64/netify-proc-flow-actions_1.0.66-1_amd64.deb
apt install ./netify-proc-flow-actions_1.0.66-1_amd64.deb
Debian 10 (Buster)
The Netify Flow Actions plugin is a licensed (proprietary) plugin. Instructions to access this plugin will have been provided to you by a Netify representative. Typically, the package will be available via a web link with Basic authentication protection. Example:
cd /tmp/
wget https://client:secret@download.netify.ai/5/debian/10/amd64/netify-proc-flow-actions_1.0.66-1_amd64.deb
apt install ./netify-proc-flow-actions_1.0.66-1_amd64.deb
OpenWRT
OpenWRT 23.05
The Netify Flow Actions plugin is a licensed (proprietary) plugin. Instructions to access this plugin will have been provided to you by a Netify representative. Typically, the package will be available via a web link with Basic authentication protection. Example:
pkg add https://client:secret@download.netify.ai/5/openwrt/23.05/amd64/netify-proc-flow-actions_1.0.66-1_x86_64.ipk
OpenWRT 22.03
The Netify Flow Actions plugin is a licensed (proprietary) plugin. Instructions to access this plugin will have been provided to you by a Netify representative. Typically, the package will be available via a web link with Basic authentication protection. Example:
pkg add https://client:secret@download.netify.ai/5/openwrt/22.03/amd64/netify-proc-flow-actions_1.0.66-1_x86_64.ipk
OpenWRT 21.02
The Netify Flow Actions plugin is a licensed (proprietary) plugin. Instructions to access this plugin will have been provided to you by a Netify representative. Typically, the package will be available via a web link with Basic authentication protection. Example:
pkg add https://client:secret@download.netify.ai/5/openwrt/21.02/amd64/netify-proc-flow-actions_1.0.66-1_x86_64.ipk
OpenWRT 19.07
The Netify Flow Actions plugin is a licensed (proprietary) plugin. Instructions to access this plugin will have been provided to you by a Netify representative. Typically, the package will be available via a web link with Basic authentication protection. Example:
pkg add https://client:secret@download.netify.ai/5/openwrt/19.07/amd64/netify-proc-flow-actions_1.0.66-1_x86_64.ipk
OPNsense
OPNsense 24.1 - Savvy Shark
The Netify Flow Actions plugin is a licensed (proprietary) plugin. Instructions to access this plugin will have been provided to you by a Netify representative. Typically, the package will be available via a web link with Basic authentication protection. Example:
pkg add https://client:secret@download.netify.ai/5/freebsd/13.2/amd64/netify-proc-flow-actions_1.0.66,1.pkg
OPNsense 23.7 - Restless Roadrunner
The Netify Flow Actions plugin is a licensed (proprietary) plugin. Instructions to access this plugin will have been provided to you by a Netify representative. Typically, the package will be available via a web link with Basic authentication protection. Example:
pkg add https://client:secret@download.netify.ai/5/freebsd/13.2/amd64/netify-proc-flow-actions_1.0.66,1.pkg
pfSense
pfSense Plus 23.x
The Netify Flow Actions plugin is a licensed (proprietary) plugin. Instructions to access this plugin will have been provided to you by a Netify representative. Typically, the package will be available via a web link with Basic authentication protection. Example:
pkg add https://client:secret@download.netify.ai/5/freebsd/14.0/amd64/netify-proc-flow-actions_1.0.66,1.pkg
pfSense CE 2.7.x
The Netify Flow Actions plugin is a licensed (proprietary) plugin. Instructions to access this plugin will have been provided to you by a Netify representative. Typically, the package will be available via a web link with Basic authentication protection. Example:
pkg add https://client:secret@download.netify.ai/5/freebsd/13.2/amd64/netify-proc-flow-actions_1.0.66,1.pkg
Rocky Linux
Rocky Linux 9
The Netify Flow Actions plugin is a licensed (proprietary) plugin. Instructions to access this plugin will have been provided to you by a Netify representative. Typically, the package will be available via a web link with Basic authentication protection. Example:
cd /tmp/
wget https://client:secret@download.netify.ai/5/rockylinux/9/stable/x86_64/netify-proc-flow-actions_1.0.66-1.os9.x86_64.rpm
apt install ./netify-proc-flow-actions_1.0.66-1.os9.x86_64.rpm
Rocky Linux 8
The Netify Flow Actions plugin is a licensed (proprietary) plugin. Instructions to access this plugin will have been provided to you by a Netify representative. Typically, the package will be available via a web link with Basic authentication protection. Example:
cd /tmp/
wget https://client:secret@download.netify.ai/5/rockylinux/8/stable/x86_64/netify-proc-flow-actions_1.0.66-1.os8.x86_64.rpm
yum install ./netify-proc-flow-actions_1.0.66-1.os8.x86_64.rpm
openSUSE Linux
openSUSE Leap 15.5
The Netify Flow Actions plugin is a licensed (proprietary) plugin. Instructions to access this plugin will have been provided to you by a Netify representative. Typically, the package will be available via a web link with Basic authentication protection. Example:
cd /tmp/
wget https://client:secret@download.netify.ai/5/opensuse/15.5/stable/x86_64/netify-proc-flow-actions_1.0.66-1.os15.5.x86_64.rpm
zypper install ./netify-proc-flow-actions_1.0.66-1.os15.5.x86_64.rpm
Ubuntu
Ubuntu 24.04 - Noble
The Netify Flow Actions plugin is a licensed (proprietary) plugin. Instructions to access this plugin will have been provided to you by a Netify representative. Typically, the package will be available via a web link with Basic authentication protection. Example:
cd /tmp/
wget https://client:secret@download.netify.ai/5/ubuntu/noble/binary-amd64/netify-proc-flow-actions_1.0.66-1_amd64.deb
apt install ./netify-proc-flow-actions_1.0.66-1_amd64.deb
Ubuntu 22.04 - Jammy
The Netify Flow Actions plugin is a licensed (proprietary) plugin. Instructions to access this plugin will have been provided to you by a Netify representative. Typically, the package will be available via a web link with Basic authentication protection. Example:
cd /tmp/
wget https://client:secret@download.netify.ai/5/ubuntu/jammy/binary-amd64/netify-proc-flow-actions_1.0.66-1_amd64.deb
apt install ./netify-proc-flow-actions_1.0.66-1_amd64.deb
Ubuntu 20.04 - Focal
The Netify Flow Actions plugin is a licensed (proprietary) plugin. Instructions to access this plugin will have been provided to you by a Netify representative. Typically, the package will be available via a web link with Basic authentication protection. Example:
cd /tmp/
wget https://client:secret@download.netify.ai/5/ubuntu/focal/binary-amd64/netify-proc-flow-actions_1.0.66-1_amd64.deb
apt install ./netify-proc-flow-actions_1.0.66-1_amd64.deb
Configuration
Plugin Loader Configuration
All plugins are disabled by default, and the Netify Flow Actions plugin is no different. To enable:
netifyd --enable-plugin proc-nfa
Alternatively, you can edit /etc/netifyd/plugins.d/10-netify-proc-flow-actions.conf and set enable to yes.
# Netify Flow Actions Processor Plugin Loader
# Copyright (C) 2023 eGloo Incorporated
#
##############################################################################
[proc-nfa]
enable = yes
plugin_library = /usr/lib64/libnetify-proc-flow-actions.so.0.0.0
conf_filename = ${path_state_persistent}/netify-proc-flow-actions.json
# vim: set ft=dosini :
As can be seen from the file layout, it's possible to run the plugin with as many instances as needed, all using different configuration files to modify the behaviour. A scenario where this fidelity might be desirable is if a network admin wanted to treat two (or more) groups of devices very differently in terms of traffic flow policy enforcement.
# Netify Flow Actions Processor Plugin Loader
# Copyright (C) 2023 eGloo Incorporated
#
##############################################################################
[proc-nfa-device-class-ot]
enable = yes
plugin_library = /usr/lib64/libnetify-proc-flow-actions.so.0.0.0
conf_filename = ${path_state_persistent}/netify-proc-flow-actions-ot.json
[proc-nfa-device-class-it]
enable = yes
plugin_library = /usr/lib64/libnetify-proc-flow-actions.so.0.0.0
conf_filename = ${path_state_persistent}/netify-proc-flow-actions-it.json
# vim: set ft=dosini :
In the example above, we have created two distinct instances of the Flow Actions plugin, one for a group of devices in "IT" and the other specifically for "OT" devices. As can be seen from the configuration file sample, the IT and OT will use completely separate files to manage the configuration, presumably with expressions matching src IP's that group devices on the network into the correct IT or OT class.
Plugin Configuration
Once the plugin has been enabled, it can be configured using the JSON configuration file specified in the plugin loader configuration. Let's look at a configuration sample to review the syntax and parts of the file.
{
"version": 1,
"targets": {
"log.games": {
"target_type": "log",
"target_enabled": false,
"prefix": "netify-games",
"interval": 60
},
"log.external": {
"target_type": "log",
"target_enabled": false,
"prefix": "netify-external",
"interval": 60
},
"sink.games": {
"target_type": "sink",
"target_enabled": false,
"sink": "sink-socket",
"channel": "default",
"flow_events": "new | updated | expired",
"format": "json",
"compressor": "none"
},
"ipset.games": {
"target_type": "ipset",
"target_enabled": false,
"set_name": "nfa.games",
"set_family": "ipv4",
"type": "hash:ip,port,ip",
"ttl": 60,
"flush_on_destroy": true
},
"ctlabel.auto": {
"target_type": "ctlabel",
"target_enabled": false,
"auto_label": true
},
"ctlabel.games": {
"target_type": "ctlabel",
"target_enabled": false,
"auto_label": false,
"labels": [
"netify-games"
]
},
"nftset.games": {
"target_type": "nftset",
"target_enabled": false,
"set_name": "nfa.games",
"set_family": "ipv4",
"type": [
"local_addr",
"ip_proto",
"other_port",
"other_addr"
],
"ttl": 60,
"flush_on_create": true
},
"pftable.games": {
"target_type": "pftable",
"target_enabled": false,
"table_name": "nfa.games",
"address_type": "other",
"ttl": 60,
"managed": true
}
},
"target_defaults": {
"log": {
"path": "/tmp",
"prefix": "netify-flow-actions",
"suffix": ".json",
"interval": 300
},
"ipset": {
"set_family": "*",
"ttl": 600
},
"nftset": {
"table_name": "netify",
"table_family": "inet",
"size": 65535,
"ttl": 300,
"managed": true,
"flush_on_destroy": true
},
"pftable": {
"target_type": "pftable",
"anchor_name": "*",
"address_type": "other"
},
"sink": {
"flow_events": "new",
"format": "json",
"compressor": "none"
}
},
"target_globals": {
"ctlabel": {
"max_bits": 127,
"connlabel_conf": "/etc/xtables/connlabel.conf"
},
"pftable": {
"device": "/dev/pf"
}
},
"actions": {
"games": {
"enabled": false,
"interface": "*",
"criteria": [
"category == 'games';",
"${path_state_persistent}/nfa-games.d"
],
"targets": [
"log.games",
"ipset.games",
"ctlabel.games",
"nftset.games"
],
"exemptions": [
"127.0.0.1",
"app == 'netify.playstation';"
],
"halt_on_match": true
},
"external": {
"enabled": false,
"interface": "*",
"criteria": [
"${path_state_persistent}/nfa-external.conf"
],
"targets": [
"log.external"
]
},
"auto_label": {
"enabled": true,
"interface": "lan",
"criteria": "*",
"targets": [
"ctlabel.auto"
]
}
},
"exemptions": [
"192.168.1.1",
"protocol == 'dns';"
]
}
Actions
Actions set the criteria that will be evaluated against each flow detection. The evaluation happens after the DPI processing of the flow has been completed, and subsequently again if the flow's metadata has been updated. If the action's criteria match, the specified action targets are called.
Each action must reference at least one target. There is no limit to the number of targets an action can reference.
Action Properties
Property | enabled |
---|---|
Description | Actions can be individually enabled or disabled. |
Type | boolean |
Default | true |
Property | interface |
---|---|
Description | Filter flows by interface or interface role. Specify an interface name, such as: eth0, or the interface's role, which can be either lan or wan. If an asterisk (*) is used, the Flow actions will be performed across all listening interfaces. |
Type | string |
Default | * |
Property | criteria |
---|---|
Description | The criteria property is mandatory and defines either the application, protocol, or category ID/tag to filter on, or a full expression which can evaluate a flow using any of the available metadata. |
Type | string |
Property | exemptions |
---|---|
Description | Exemptions are optional and can be thought of as the inverse of criteria. Usually they are employed to filter out certain hosts by address, however they can also be configured with an expression for more advanced exemption filtering. |
Type | string |
Property | targets |
---|---|
Description | An array of one or more target names to call when a flow matches the action criteria. |
Type | array |
Property | halt_on_match |
---|---|
Description | If set to true, matching on this action will cause the plugin to stop evaluating any further actions listed after this instance. |
Type | boolean |
Default |
Targets
Targets are called by actions when a given flow matches the specified criteria. Any number of actions can reference any other number of targets. Targets perform the tasks that their type implements, such as adding an address to an IP set or adding matching flow metadata to a log file.
The following action target types are supported:
Name | Platforms | Description |
---|---|---|
log | All | The log target will write periodic JSON-formatted log files. |
ipset | Linux | The ipset target adds to or updates user-defined ipsets that can then be referenced by various third-party firewall, QoS/QoE, and routing applications. |
nftset | Linux | The nftset target adds to or updates user-defined nftsets that can then be referenced by various third-party firewall, QoS/QoE, and routing applications. |
ctlabel | Linux | The ctlabel action target can be configured to apply user-defined Netfilter conntrack labels to match flows. |
pftable | FreeBSD | The pftable action target can be used to manage pf tables. The pf device is made available by ensuring the kernel module has been loaded (kldload pf ) |
sink | All | Flow events that match an action's criteria can be dispatched to a sink plugin for further processing. For example, all new flow detections of a given category, such as games, could be sent to a local socket or message queue for another application to process. |
Common Target Properties
Property | ttl |
---|---|
Description | The Plugin will attempt to remove the matching element after not seeing matching traffic after this TTL. Elements can be set to expire after a specified number of seconds by configuring the ttl property. Generally it is recommended to always set a ttl, to a fairly high value, such as 900 (15 minutes), to ensure the list of elements doesn't grow too large. Set the ttl to 0 to disable timed expiration. |
Type | unsigned |
Default | 60 |
Targets | ipset, nftset, pftable |
Property | auto_expire |
---|---|
Description | When enabled, the Agent will automatically notify the Plugin when a flow expires and an attempt will be made to remove the element from the set/table. |
Type | boolean |
Default | false |
Targets | ipset, nftset, pftable |
Property | managed |
---|---|
Description | When the managed mode is enabled, the plugin will attempt to create the set/table if it doesn't exist. |
Type | boolean |
Default | false |
Targets | ipset, nftset, pftable |
Property | flush_on_create |
---|---|
Description | When the managed mode is enabled and flush_on_create is enabled, the set/table will be flushed (cleared), on start-up if it exists. |
Type | boolean |
Default | false |
Targets | ipset, nftset, pftable |
Property | flush_on_destroy |
---|---|
Description | When enabled, flush_on_destroy will flush (clear) all elements on shutdown. This option does not depend on managed mode being enabled. |
Type | boolean |
Default | false |
Targets | ipset, nftset, pftable |
Log Target Properties
Property | interval |
---|---|
Description | The interval of time, in seconds, between log writes. A new file is created for each interval only if new flow data is collected (empty log files will not be created). |
Type | unsigned |
Default | 60 |
Property | path |
---|---|
Description | The full path of the desired log directory. New log files will be created here. |
Type | string |
Default | /tmp |
Property | prefix |
---|---|
Description | The log filename prefix. New log files will begin with this identifier. |
Type | string |
Default | netify-flow-actions |
Property | suffix |
---|---|
Description | The log filename suffix and/or extension. New log files will end with this identifier. |
Type | string |
Default | .json |
ipset Target Properties
IP Sets are high-speed Linux kernel tables that are used to store lists of network addresses. More information about IP Sets can be found on the project website, or from the ipset man page.
Property | set_name |
---|---|
Description | The IP set name. |
Type | string |
Property | set_family |
---|---|
Description | The IP family of the set. Values: ipv4, ipv6, or * for both. An IP family suffix will automatically be appended to the set's name. For example: social-media.v4 |
Type | string |
Default | * |
Property | type |
---|---|
Description | The IP set type. IP set types are comprised of a storage method and a datatype. Valid storage methods are: bitmap, hash, and list . The currently supported list of types are: ip, ip,port, ip,port,ip, and mac. |
Type | string |
Default | hash:ip |
Property | max_elements |
---|---|
Description | The maximum number of elements that can be stored in the sets. |
Type | unsigned |
Default | 65536 |
Property | ttl |
---|---|
Description | The Agent will automatically notify the Plugin when a flow expires. The Plugin will then attempt to remove the matching element. However, elements can be set to auto-expire after a specified number of seconds by configuring the ttl property. Generally, it is recommended to always set a ttl, to a fairly high value, such as 900 (15 minutes), to ensure the set doesn't grow too large. Setting the ttl to 0 will disable auto-expiry. |
Type | unsigned |
Default | 0 |
Property | skb_mark |
---|---|
Description | Firewall mark as a hexadecimal string. When any of the skb_* options are set, and an appropriate firewall rule is present (a SET netfilter target with the --map- set argument), packets that match this IP set will have their corresponding socket buffer options set. |
Type | string |
Property | skb_mask |
---|---|
Description | Corresponding mask to a specified skb_mark, as a hexadecimal string. If skb_mark is set, without a mask, the default value is: 0xffffffff |
Type | string |
Property | skb_prio |
---|---|
Description | Traffic control tc class. The option has the format: MAJOR:MINOR. |
Type | string |
Property | skb_queue |
---|---|
Description | Hardware queue ID, unsigned integer. |
Type | unsigned |
Property | managed |
---|---|
Description | When the managed mode is enabled, the plugin will attempt to create the IP set if it doesn't exist. NOTE: If you change the plugin's configuration, and existing sets are present, you must manually destroy them in order for any new settings to take effect. |
Type | boolean |
Default | false |
Property | flush_on_create |
---|---|
Description | When the managed mode is enabled and flush_on_create is enabled, the IP set will be flushed (cleared), on start-up if it exists. |
Type | boolean |
Default | false |
Property | flush_on_destroy |
---|---|
Description | When enabled, flush_on_destroy will flush (clear) the IP set on shutdown. This option does not depend on managed mode being enabled. |
Type | boolean |
Default | false |
nftset Target Properties
Property | table_name |
---|---|
Description | The table name that the NFT set is a member of. |
Type | string |
Default | nfa |
Property | table_family |
---|---|
Description | The family of the table. |
Type | string |
Default | inet |
Property | set_name |
---|---|
Description | The mandatory name of the NFT set. |
Type | string |
Property | set_family |
---|---|
Description | The IP family of the set. Valid values are ipv4, ipv6, or * for both. |
Type | string |
Default | * |
Property | type |
---|---|
Description | The set type is defined as an array of flow metadata fields. Rules that reference the NFT set must do so using the same field order that is defined here. The supported fields are:
|
Type | array |
Property | max_elements |
---|---|
Description | The maximum number of elements that can be stored in the NFT set. |
Type | unsigned |
Default | 65536 |
ctlabel Target Properties
The Netfilter Conntrack API provides an interface module for applying user-defined labels (bits) on a per-flow basis. Flow labels can be examined using the conntrack
command from the conntrack-tools package. For example: conntrack -L -o labels
Property | labels |
---|---|
Description | The labels array contains one or more label bits or label names. All labels will be applied to all flows that reach the target. |
Type | array |
Property | auto_label |
---|---|
Description | The target can parse prefixed label names from connlabels.conf which, when enabled, will automatically apply them to matching flows.The syntax for the label names is: The Google/Meet/Duo -> a-google-meet-duo |
Type | array |
Property | log_errors |
---|---|
Description | By default, errors are not logged to reduce processing overhead. When the Agent is in debug mode and log_errors is enabled, error messages will be displayed for flows that could not be labeled. |
Type | boolean |
Default | false |
Flow Criteria / Expression Engine
The flow criteria (or expression engine) within the Flow Actions Plugin is one of its most versatile features. It allows integrators and administrators to craft rules with varying complexity.
Syntax
Match All / Wildcard
The simplest form of expression is the 'match all flows', accomplished with the use of the asterisk (*) as follows:
{
...
"criteria": "*"
...
}
An interface filter and/or exemptions can still be used to filter out flows when using a wildcard for the flow criteria.
Application and Protocol Matching
Once a flow has been classified by the agent's DPI engine, matching by either application or protocol is a common requirement. Matching by ID supports a "Simple Syntax" as well as a method to create more complex expressions, termed "Advanced Syntax". Matcing by category also supports a three levels of specificity (ex. origin of category association):
Simple Syntax
Using the syntax ai
for Application matching and pi
for Protocol matching, simple rules can be created. If desired, a flow criteria can be
inverted through the use of the NOT operator: !
. Matches can be made by tags or ID:
- ai:facebook
- ai:netify.facebook
- ai:119
- pi:dns
- !pi:5
--dump-apps
and --dump-protos
arguments. For
more information, see the command line arguments documentation.{
...
"criteria": "ai:facebook"
"criteria": "pi:mysql"
...
}
Both applications and protocols are grouped into categories in Netify. Categories can also be used in the creation of expressions:
- ac:vpn-and-proxy (is VPN and Proxy Category)
- ac:31 (is VPN and Proxy Category)
- pc:games (is Games Category)
- !pc:5 (is NOT Games Category)
--dump-categories
argument. For
more information, see the command line arguments documentation.{
...
"criteria": "!pc:games"
...
}
Advanced Expressions
Expressions are syntactically similar to C-based programming languages. They operate on flow criteria, for example: application, local_ip, local_mac, remote_port, protocol,
application_category, etc. Expressions must be terminated by a semicolon and all string types must be enclosed within single quotes: '...'
.
Comparison Operators
- Equal to: ==
- Not equal to: !=
- Greater than: >
- Greater than or equal to: >=
- Less than: <
- Less than or equal to: <=
Boolean Operators
- NOT: !
- AND: &&, and
- OR: ||, or
Regular Expressions
rx:
prefix. For example, if you wanted to match on the detected_hostname property:
"criteria": "detected_hostname == 'rx:.*xxx';"
Precedence
Group expressions with round brackets to control evaluation precedence.
Examples
"criteria": [
"protocol_id == 20;",
"protocol == 'icmp';",
"category == 'adult';",
"application == 'netify.linkedin';"
],
Flow Criteria
Property | ip_version |
---|---|
Description | IP version. Valid values are 4 for IPv4, and 6 for IPv6. |
Type | unsigned |
Property | ip_protocol |
---|---|
Description | Internet protocol ID. Example, 6 for TCP. |
Type | unsigned |
Property | ip_nat |
---|---|
Description | When supported (Linux-only) and for external (WAN) interfaces only. The NAT flag is set if the flow is found in the connection tracking table. |
Type | boolean |
Property | vlan_id |
---|---|
Description | Virtual LAN (VLAN) ID. |
Type | unsigned |
Property | other_type |
---|---|
Description | The 'other' address constants are:
|
Type | constant |
Property | local_mac |
---|---|
Description | Local MAC address in the format: 01:02:03:ab:cd:ef. This attribute type is not a string and should not be enclosed in single quotes. |
Type | mac_address |
Property | other_mac |
---|---|
Description | Other MAC address in the format: 01:02:03:ab:cd:ef. This attribute type is not a string and should not be enclosed in single quotes. |
Type | mac_address |
Property | local_ip |
---|---|
Description | Local IP address or CIDR notation in either IPv4 or IPv6. This attribute type is not a string and should not be enclosed in single quotes. |
Type | ip_address |
Property | other_ip |
---|---|
Description | Other IP address or CIDR notation in either IPv4 or IPv6. This attribute type is not a string and should not be enclosed in single quotes. |
Type | ip_address |
Property | local_port |
---|---|
Description | Local port, when applicable, depending on ip_protocol |
Type | unsigned |
Property | other_port |
---|---|
Description | Other port, when applicable, depending on ip_protocol |
Type | unsigned |
Property | origin |
---|---|
Description | The origin end-point of the flow (which side, local or other, initiated the packet flow). Possible values:
|
Type | constant |
Property | tunnel_type |
---|---|
Description | If the flow contains encapsulated data that could be identified and extracted (not encrypted), such as GTP, the tunnel_type will be set according to the following values:
|
Type | string |
Property | fwmark |
---|---|
Description | When available and on supported platforms (Linux-only), the connection tracking mark value |
Type | unsigned |
Property | detection_guessed |
---|---|
Description | If the protocol could not be determined via packet analysis, an attempt to 'guess' the protocol is made using various methods such as the port address(es) |
Type | boolean |
Property | application |
---|---|
Description | The detected application tag |
Type | string |
Property | application_id |
---|---|
Description | The detected application ID. Also supports the short form, app_id. |
Type | unsigned |
Property | application_category |
---|---|
Description | The detected application category. Also supports the short form, app_category. |
Type | unsigned |
Property | protocol |
---|---|
Description | The detected protocol tag |
Type | string |
Property | protocol_id |
---|---|
Description | The detected protocol ID. Also supports the short form, proto_id. |
Type | unsigned |
Property | protocol_category |
---|---|
Description | The detected protocol category. Also supports the short form, proto_category. |
Type | unsigned |
Property | domain_category |
---|---|
Description | If one or more custom domain category lists have been configured, matches from them can be explicitly tested for using domain_category |
Type | string |
Property | category |
---|---|
Description | The category criteria is a hybrid evaluation where either application_category, protocol_category or the domain_category are tested, eliminating the need to write two or more statements |
Type | string |
Property | risks |
---|---|
Description | The detected risk |
Type | string |
Property | ndpi_risk_score |
---|---|
Description | The risk score as detected by the nDPI library |
Type | unsigned |
Property | ndpi_risk_client |
---|---|
Description | The individual nDPI risk score for the client-side |
Type | unsigned |
Property | ndpi_risk_server |
---|---|
Description | The individual nDPI risk score for the server-side |
Type | unsigned |
Property | detected_hostname |
---|---|
Description | The detected hostname, which can be extracted from a variety of protocols |
Type | string |
Property | ssl_version |
---|---|
Description | For detected TLS flows, the SSL/TLS version can be tested. The values are usually represented using hexadecimal values, for example, TLS v1.3 would be: 0x0303 |
Type | string |
Property | ssl_cipher |
---|---|
Description | For detected TLS flows, the SSL/TLS negotiated cipher can be tested. The values are usually represented using hexadecimal values, for example TLS v1.3 would be: 0x0303 |
Type | string |
Flow Criteria Examples
Note how the other_local
is not in single quotes due to it being matched against a constant.
"criteria": "other_type == other_local && (app_category == 'voip' || protocol_category == 'voip');"
Multiple expressions can be joined in an implicit OR
statement by setting the criteria to an array.
"criteria": [
"application == 'facebook';",
"application == 'youtube';",
"application == 'twitter';"
]
Specifying the criteria as a folder pathname causes individual 'configlets' within the folder to be parsed and used as the criteria.
"criteria": "${path_state_persistent}/nfa-games.d"
In this case, we could drop in the following configlets that define individual games we might want to group:
/etc/netifyd/nfa-games.d/10-rockstar-games.conf
app == 'netify.rockstar-games';
/etc/netifyd/nfa-games.d/10-epic-games.conf
app == 'netify.epic-games';
/etc/netifyd/nfa-games.d/10-ea-games.conf
app == 'netify.ea-games';
Integration
The Netify Flow Actions plugin provides the mechanism for translating from Layer 7 classifications to Layer 3 control. The plugin supports the following targets:
- IP sets
- nftables sets
- aliases (BSD)
- Connection Tracking (CT) Labels
- Logging
Examples
The examples below show what is possible with Netify agent and the Flow Actions processor plugin when the agent is running on a hardware or virtual instance processing realtime packets on the data plane.
Block Adult Content with iptables
The example for blocking adult content will demonstrate three core concepts around expressions that can be created with Netify Flow Actions plugin:
- Application matching
- Category matching along with management of a categories.d list
- IP based exemption
Enabling the Flow Actions Plugin
All plugins are disabled by default. Enable the Flow Actions plugin using either of the two methods below.
1. Use netifyd
.
netifyd --enable-plugin proc-nfa
2. Edit the plugin's configuration file.
sudo vi /etc/netifyd/plugins.d/10-netify-proc-flow-actions.conf
Set the enable
attribute to yes.
Configuring Flow Actions Plugin
With the plugin enabled, the next step is to modify the configuration file, /etc/netifyd/netify-proc-flow-actions.json
. For this specific
example, we'll drop in a full sample config and review each of the lines for their purpose.
{
"version": 1,
"target_defaults": {
"log": {
"path": "/tmp",
"prefix": "netify-flow-actions",
"suffix": ".json",
"interval": 300
},
"ipset": {
"set_family": "*",
"ttl": 600
}
},
"targets": {
"log.block": {
"target_type": "log",
"target_enabled": true,
"prefix": "netify-block",
"interval": 60
},
"ipset.block": {
"target_type": "ipset",
"set_name": "nfa.block",
"set_family": "ipv4",
"type": "hash:ip,port,ip",
"managed": true,
"ttl": 60,
"auto_expire": true,
"flush_on_destroy": true
}
},
"actions": {
"10-adult": {
"enabled": true,
"interface": "*",
"criteria": [
"category == 'adult';"
],
"targets": [
"log.block",
"ipset.block"
]
}
},
"exemptions": [
"protocol == 'dns';"
]
}
The version
is a way to maintain compatibility across upgrades. There has been no requirement to issue another version of the JSON
configuration format at this time, so it is safe to leave this at version 1.
The target_defaults
directive is a set of defaults that will apply to all target definitions found in the
configuration file. These attributes can be overridden in the individual sections.
The log
target type will have these attributes applied if an override is not set.
- path - the path the log file will be saved to
- prefix - the prefix to apply to the log file
- suffix - the suffix to apply to the log file
- interval - the time (in seconds) to aggregate data sent to the log file
/tmp/netify-flow-actions-log-20240317-092000.json
Where the numerical part of the filename is the YYYYMMDD-hhmmss, which, in the example above would include any events between 9:15 and 9:20 AM on March 27,
2024 (300 seconds or 5 minutes).
The ipset
target type will have these attributes applied if an override is not set.
- set_family - an asterisk (*) denotes all set families, but ipv4 and ipv6 can be defined explicitly if desired
- ttl - Time to Live (TTL) configures how long members added to the ipset exist before being flushed out automatically
The target
attribute is the start of defining one or more targets.
We define a log type target, with the key log.block
that will be used in our action. We override some defaults.
We define an ipset type target, with the key ipset.block
that will be used in our action. We override some defaults.
- target_type - the type of target, in this case, ipset
- set_family - the ipset family, either ipv4 or ipv6
- type - the ipset type - this must match the ipset created manually if managed is not set to true
- managed - if
true
, the ipset will be created automatically when the Netify daemon starts or is restarted - auto_expire - if set to true, a member will be automatically flushed when a flow purge event is triggered
- flush_on_destroy - removes all members from the ipset if Netify is restarted or stopped
The actions
attribute is the start of defining one or more actions.
We define an action that sets the criteria that will be evaluated against each flow detection. One or more actions can
be defined. In the example above, we create a single action, keyed on the name 10-adult
.
- enabled - by setting to true, we have enabled the action
- interface - by setting to *, we are matching on every interface Netify is listening on. Performance improvements can be gained if limiting the interface to only those required. Aside from all (*), an interface name, such as: eth0, or the interface's role, which can be either lan or wan, can be specified.
- criteria - any flow matching the adult category will be sent to the target for processing
- targets - on a match, what target(s) will be processed. The target names, in this case, log.block and log.ipset must be defined by a key in the targets section of the same JSON file.
Global exemptions are specified outside any action
block. The Global Exemptions
array is the ideal location to filter critical infrastructure and traffic from entering policy controls, like
matching a DROP target. In this example, we exempt any DNS protocol from matching a pattern as well as flows originating from the host iteself.
Flow Actions Plugin in Action
Since our configuration is using ipset, we need to make sure this new dependency is installed. In addition, we made changes to our configuration - we restart the Netify agent for the changes to be implemented.
apt update
apt install ipset
systemctl restart netifyd
Recall, our configuration has the ipset target set to managed. After restart, verify the ipset exists:
ipset -L
Name: nfa.block.v4
Type: hash:ip,port,ip
Revision: 5
Header: family inet hashsize 1024 maxelem 65536 timeout 60 skbinfo
Size in memory: 200
References: 0
Number of entries: 0
Members:
Notice there are currently no members. We can quickly change that by making a cURL
request to a popular adult website.
curl https://pornhub.com
ipset -L
Name: nfa.block.v4
Type: hash:ip,port,ip
Revision: 5
Header: family inet hashsize 1024 maxelem 65536 timeout 60 skbinfo
Size in memory: 392
References: 0
Number of entries: 1
Members:
66.254.114.41,tcp:443,192.168.1.1 timeout 54
Layer 7 to Layer 3 mapping is now complete, and with that, it is time to add the last element - an iptables firewall rule to perform our action on the matched set.
Normally, this rule would be added to the FORWARD
chain like this:
iptables -t filter -A FOWARD -m set --match-set nfa.block.v4 dst,dst,src -j DROP
OUTPUT
chain.
iptables -t filter -A OUTPUT -m set --match-set nfa.block.v4 dst,dst,src -j DROP
Throttling/QoS with Traffic Control (tc)
The example for applying a throttle will demonstrate the core concept around expressions that can be created with Netify Flow Actions plugin:
- Flow Actions "dot-d" folders
Enabling the Flow Actions Plugin
All plugins are disabled by default. Enable the Flow Actions plugin:
netifyd --enable-plugin proc-nfa
Configuring Flow Actions Plugin
With the plugin enabled, the next step is to modify the configuration file, /etc/netifyd/netify-proc-flow-actions.json
. For this specific
example, we'll drop in a full sample config and review each of the lines for their purpose.
{
"version": 1,
"targets": {
"ipset.throttle": {
"target_type": "ipset",
"set_name": "nfa.throttle",
"set_family": "ipv4",
"type": "hash:ip,port,ip",
"managed": true,
"ttl": 60,
"auto_expire": true,
"flush_on_destroy": true
}
},
"actions": {
"throttled": {
"enabled": true,
"interface": "*",
"criteria": [
"${path_state_persistent}/nfa-throttle.d"
],
"targets": [
"ipset.throttle"
]
}
}
}
In this example, notice how our criteria isn't specified by matching on applications categories or any other attribute of the flow. Instead, we have
defined a folder, in this case, /etc/netifyd/nfa-throttle.d
.
During startup or on receiving a HUP command, the agent will parse any configlets found in this folder. The convenience and power this model provides should not be underestimated, especially when making modifications programmatically. A UX developer can simply drop and remove (or disable) individual configlet files to tailor a solution. To see how that would work, run:
echo "app == 'netify.netflix';" > /etc/netifyd//nfa-throttle.d/10-netflix.conf
systemctl reload netifyd
What we've just done is echo out valid criteria to match any flows with application matching netify.netflix
to a configlet that we
can easily recognize the purpose of. By sending a HUP, the Netify Agent, and by proxy, the Flow Actions plugin has parsed the new configlet and
incorporated it into the processing engine. If we then ran:
echo "app == 'netify.youtube';" > /etc/netifyd/nfa-throttle.d/10-youtube.conf
systemctl reload netifyd
We would now be matching on both Netflix and YouTube traffic. To undo a change, we could do either of the following:
rm -rf /etc/netifyd/nfa-throttle/10-netflix.conf
systemctl reload netifyd
or
mv /etc/netifyd/nfa-throttle/10-netflix.conf /etc/netifyd/nfa-throttle/10-netflix.conf.disabled
systemctl reload netifyd
Until now, we would have an ipset
called nfa.throttle.v4, but we don't have any actions on the ipset. Let's change that now. Create a file
named netify.throttle and add the following:
#!/bin/bash
export LAN=eth1
tc qdisc add dev $LAN root handle 1: htb default 20
tc class add dev $LAN parent 1: classid 1:1 htb rate 500kbit burst 500k
tc filter add dev $LAN protocol ip parent 1: prio 1 handle 6 fw flowid 1:1
tc -p -s -d qdisc show dev $LAN
iptables -t filter -A FORWARD -m set --match-set nfa.throttle.v4 dst,dst,src -j MARK --set-mark 6
iptables -t filter -A OUTPUT -m set --match-set nfa.throttle.v4 dst,dst,src -j MARK --set-mark 6
Note, the OUTPUT rule above is only needed if you are 'kicking the tires' and evaluating this plugin on a non-gateway device. It will apply to packets originating from the local process / host.
Also note, that within the script is the interface name for the LAN interface. Set this according to your environment. For example, on OpenWRT standard
installs, it would be export LAN=br-lan
.
We have now implemented a throttling rule of 500kbps using Hierarchy Token Bucket (HTB) within a traffic control (tc) class. The iptables
rules in place are designed to mark packets which are then filtered and shaped accordingly. You can confirm that shaping is occurring by looking at the
tc
stats:
export LAN=eth1
tc -p -s -d qdisc show dev $LAN
and, of course, by using Netflix or YouTube...you will experience a sluggish load time and a downgrade in the video quality due to the throttling.