Expression Engine

The expression engine is a powerful matching and filtering system used throughout Netify Agent v5 to define flow policies, network intelligence rules, and classification criteria. It enables network administrators and integrators to craft sophisticated rules that match flows based on dozens of attributes including application, protocol, network location, traffic direction, intelligence risk scores, and custom metadata.

Whether you are enforcing network policies with the Flow Actions processor, or creating custom indicators with the Network Intelligence processor, the expression engine provides the flexibility to match precisely the flows you need.

Supported application and protocol tags and IDs can be listed using the --dump-apps and --dump-protos arguments. For more information, see the command line arguments documentation.

Two Supported Syntaxes

Two syntax modes are available for creating flow expressions: a compact syntax optimized for common use cases, and a full-featured advanced syntax for complex matching logic.

Compact Syntax: Streamlined syntax for matching applications, protocols, and categories using short tags and identifiers.
Advanced Syntax: Full-featured syntax with comparison operators, booleans, regular expressions, and dozens of flow attributes.

Compact Syntax

Introduced in Netify 5.2, the compact syntax rules can be created for flow expressions by using the ai syntax for application matching and pi syntax for protocol matching.

ai:facebook: Facebook application short tag
ai:netify.facebook: Facebook application full tag
ai:119: Facebook application ID
pi:5: DNS protocol ID

Both applications and protocols are grouped into categories in Netify. The ac syntax can be specified for application categories, and pc syntax for protocol categories.

ac:vpn-and-proxy: VPN and proxy application category
ac:31: VPN and proxy application category ID
pc:games: Game protocol category tag
pc:5: Game protocol category ID

If desired, a flow criterion can be inverted through the use of the not operator: ! .

Compact Syntax Examples

{
    ...
    "criteria": "ai:facebook"
    "criteria": "ai:netify.facebook"
    "criteria": "pi:mysql"
    ...
    // Not operator examples
    "criteria": "!pc:games"
    "criteria": "!pi:5"
    ...
}

Advanced Expressions

Advanced expressions use standard logic operators and can trigger on a full suite of flow criteria, for example: application, local_ip, local_mac, other_port, protocol, application_category, etc. Expressions must be terminated by a semicolon and all string types must be enclosed within single quotes.

Only strings are enclosed in single quotes. The Netify Agent will return an error when initializing the Flow Actions plugin on any deviation.

Comparison Operators

Equal to: ==
Not equal to: !=
Greater than: >
Greater than or equal to: >=
Less than: <
Less than or equal to: <=

Boolean Operators

NOT: !
AND: &&, and
OR: ||, or

Regular Expressions

For any property which is a string type, regular expressions are supported using rx: prefix. For example, if you wanted to match on the detected_hostname property:

"criteria": "detected_hostname == 'rx:.*xxx';"

Regular expressions were introduced in Netify version 5.2.

Precedence

Group expressions with round brackets to control evaluation precedence.

Flow Criteria Examples

Operators and Precedence

Here is an example of matching the VoIP application category, or the VoIP protocol category, but only on the local network. Note how the other_local is not in single quotes due to it being matched against a constant.

"criteria": "other_type == other_local && (app_category == 'voip' || protocol_category == 'voip');"

Lists

Multiple expressions can be joined in an implicit OR statement by setting the criteria to an array.

"criteria": [
    "application == 'youtube';",
    "protocol_id == 20;",
    "protocol == 'icmp';",
    "category == 'adult';",
    "application == 'netify.linkedin';"
]

Configlets

Specifying the criteria as a folder pathname causes individual 'configlets' within the folder to be parsed and used as the criteria.

"criteria": "${path_state_persistent}/nfa-games.d"

In this case, we could drop in the following configlets that define individual games we might want to group:

/etc/netifyd/nfa-games.d/10-rockstar-games.conf

app == 'netify.rockstar-games';

/etc/netifyd/nfa-games.d/10-epic-games.conf

app == 'netify.epic-games';

/etc/netifyd/nfa-games.d/10-ea-games.conf

app == 'netify.ea-games';

With this type of configuration, any match on any of the configlet criteria rules would trigger the associated action.

Flow State

If you wanted to trigger an action every time a flow classification completed, you could use the following criteria expression in your configuration.

"criteria": "detection_complete;"

It is also possible to match on more than one state flag. For example:

"criteria": "detection_updated || detection_complete;"

Match All / Wildcard

The simplest form of expression is the 'match all flows', accomplished with the use of the asterisk (*). An interface filter and/or exemptions can still be used to filter out flows when using a wildcard for the flow criteria.

"criteria": "*"

Flow Criteria Reference

address_group

string

The address_group is a powerful way to match or exclude on bulk elements - either IP or MAC addresses. The syntax for matching on an address group is to put an ampersand in front of the address group name, e.g. "address_group = @printers;". The address group does not have to be defined beforehand to be used in the criteria. See documentation.

app

string

Alias of `application`.

app_id

integer

Alias of `application_id`.

application

string

The detected application tag.

application_category

integer

The detected application category ID. Also supports the short form, app_category.

application_id

integer

The detected application ID. Also supports the short form, app_id.

cat

string

Alias of `category`.

cat_id

integer

Alias of `category_id`.

category_id

integer

Numeric category identifier.

detected_hostname

string

The detected hostname, which can be extracted from a variety of protocols.

detection_complete

boolean

Flag to indicate classification has been completed, typically in fewer than 32 packets.

detection_guessed

boolean

If the protocol could not be determined via packet analysis, an attempt to 'guess' the protocol is made using various methods such as the port address(es).

detection_init

boolean

Flag to indicate initial application and/or protocol has been detected.

detection_updated

boolean

Flag to indicate flow metadata has been extracted (e.g., TLS certificate info) leading to the possible application being refined.

dhc_hit

boolean

Internal classification hit flag (DHC).

dns_hostname

string

Hostname extracted from DNS queries.

dom_category

string

Alias of `domain_category`.

domain_category

string

If one or more custom domain category lists have been configured, matches from them can be explicitly tested for using domain_category.

dst_ip

ip_address

Destination IP address.

dst_mac

mac_address

Destination MAC address.

dst_net_category

string

Alias of `dst_network_category`.

dst_net_category_id

integer

Alias of `dst_network_category_id`.

dst_network_category

string

Destination network category tag.

dst_network_category_id

integer

Destination network category numeric ID.

dst_port

integer

Destination port number.

expiring

boolean

Flag indicating the flow is nearing expiration.

expired

boolean

Flag indicating the flow has expired.

fhc_hit

boolean

Internal classification hit flag (FHC).

fwmark

integer

When available and on supported platforms (Linux-only), the connection tracking mark value.

iface

string

Network interface name (or index).

iface_nfq_dst

string

NFQUEUE destination interface name.

iface_nfq_src

string

NFQUEUE source interface name.

intel_*

string

Intel metadata fields with dynamic suffixes (matches tokens starting with 'intel_').

ip

ip_address

Any IP address (local or other). This attribute type is not a string and should not be enclosed in single quotes.

ip_dscp

integer

Differentiated Services Code Point (DSCP) value.

ip_nat

boolean

The NAT flag is set if the flow is found in the connection tracking table on a WAN. Linux only.

ip_protocol

integer

Internet protocol ID. Example, 6 for TCP.

ip_version

integer

IP version. Valid values are 4 for IPv4, and 6 for IPv6.

local_ip

ip_address

Local IP address or CIDR notation in either IPv4 or IPv6. This attribute type is not a string and should not be enclosed in single quotes.

local_mac

mac_address

Local MAC address in the format: 01:02:03:ab:cd:ef. This attribute type is not a string and should not be enclosed in single quotes.

local_net_category

string

Alias of `local_network_category`.

local_net_category_id

integer

Alias of `local_network_category_id`.

local_network_category

string

Local network category tag.

local_network_category_id

integer

Local network category numeric ID.

local_port

integer

Local port, when applicable, depending on ip_protocol.

mac

mac_address

Any MAC address (local or other). This attribute type is not a string and should not be enclosed in single quotes.

ndpi_risk_client

integer

Client-side nDPI risk score.

ndpi_risk_score

integer

Aggregated nDPI risk score.

ndpi_risk_server

integer

Server-side nDPI risk score.

net_category

string

Alias of `network_category`.

net_category_id

integer

Alias of `network_category_id`.

network_category

string

Network category tag.

network_category_id

integer

Network category numeric ID.

origin

constant

The origin end-point of the flow (which side, local or other, initiated the packet flow).

Options: origin_local, origin_other, origin_unknown

other_ip

ip_address

Other IP address or CIDR notation in either IPv4 or IPv6. This attribute type is not a string and should not be enclosed in single quotes.

other_mac

mac_address

Other MAC address in the format: 01:02:03:ab:cd:ef. This attribute type is not a string and should not be enclosed in single quotes.

other_net_category

string

Alias of `other_network_category`.

other_net_category_id

integer

Alias of `other_network_category_id`.

other_network_category

string

Other network category tag.

other_network_category_id

integer

Other network category numeric ID.

other_port

integer

Other port, when applicable, depending on ip_protocol.

other_type

constant

The flow other type - see telemetry conventions.

Options: other_local, other_multicast, other_remote, other_broadcast, other_unknown, other_unsupported, other_error

port

integer

Any port number.

protocol

string

The detected protocol tag - recommendation: use IDs

protocol_category

integer

The detected protocol category. Also supports the short form, proto_category.

protocol_id

integer

The detected protocol ID. Also supports the short form, proto_id.

risks

string

Risk tags associated with the flow.

soft_dissector

boolean

Flag indicating a soft dissector was used for classification.

src_ip

ip_address

Source IP address.

src_mac

mac_address

Source MAC address.

src_net_category

string

Alias of `src_network_category`.

src_net_category_id

integer

Alias of `src_network_category_id`.

src_network_category

string

Source network category tag.

src_network_category_id

integer

Source network category numeric ID.

src_port

integer

Source port number.

ssl_cipher

string

Alias of `tls_cipher`.

ssl_version

string

Alias of `tls_version`.

tag

string

Custom tag defined in the Overlay feature.

tag_category

string

Tag category name.

tag_category_id

integer

Numeric tag category identifier.

tls_cipher

string

For detected TLS flows, the SSL/TLS negotiated cipher can be tested. The values are usually represented using hexadecimal values, for example TLS v1.3 would be: 0x0303.

tls_ech

boolean

Flag indicating use of TLS ECH (Encrypted Client Hello).

tls_ja4

string

JA4 TLS fingerprint or identifier.

tls_version

string

For detected TLS flows, the SSL/TLS version can be tested. The values are usually represented using hexadecimal values, for example, TLS v1.3 would be: 0x0303.

tunnel_type

string

If the flow contains encapsulated data that could be identified and extracted (not encrypted), such as GTP, the tunnel_type will be set.

Options: tunnel_none, tunnel_gtp

vlan_id

integer

Virtual LAN (VLAN) ID.