Category Lists

• Overview
• Creating Category Lists
• Memory Considerations
• Next Steps

Overview

Netify's DPI Agent and application signature list actively track the top 200 applications in use on the Open Source signature. Netify Informatics subscribers and OEM/integration clients have a license to access and use the commercial signature list, having over 1600 application definitions.

In some cases, having very detailed metadata and classification for an application isn't necessary. One such case is adult content, by some estimates, having over 4M domains. Any user of the Netify agent, regardless of the application signatures in use, can BYOC (Bring Your Own Category) list. By creating and maintaining a category list, the Netify agent's classification will attempt to pattern match and tag in near real-time a category associated with unique flows on the network.

With the category ID of a flow populated, this additional information can then be used with Netify plugins.

Creating Category Lists

Categories cannot be defined - you must use one of the available category classifications known to Netify. To generate a list of new categories, run:

netifyd --dump-categories
     2: application: adult
     3: application: advertiser
     4: application: entertainment
     5: application: business
     6: application: education
     9: application: financial
    10: application: file-sharing
    11: application: gambling
    12: application: games
    13: application: government
    15: application: mail
    16: application: malware
    17: application: messaging
    18: application: news
    19: application: portal
    20: application: recreation
    21: application: reference
    23: application: shopping
    24: application: social-media
    26: application: sports
    27: application: technology
    28: application: vpn-and-proxy
    29: application: streaming-media
    30: application: cybersecurity
    31: application: os-software-updates
    32: application: voip
    33: application: device-iot
    34: application: remote-desktop
    35: application: cdn
    36: application: hosting
    37: application: telco
     2: protocol: database
     4: protocol: file-server
     5: protocol: file-sharing
     7: protocol: games
     9: protocol: infrastructure
    11: protocol: mail
    12: protocol: media
    13: protocol: media-provider
    14: protocol: networking
    16: protocol: proxy
    17: protocol: authentication
    18: protocol: remote-desktop
    20: protocol: voip
    21: protocol: vpn
    22: protocol: web
    24: protocol: messaging
    25: protocol: printing

Only application categories (those with the prefix "application: ") can be used. To create a new BYOC list, create a file in the /etc/netifyd/categories.d folder.

sudo touch /etc/netifyd/categories.d/10-adult.conf

• The filename must start with a number followed by a dash. This number determines the priority of the list. For example, if you obtained an adult content list from two different sources, the one with the lower priority would be matched first.
• After the dash, the category as listed must follow, in lowercase characters.
• The file name must end in '.conf'. Any filename in the categories.d directory that does not end in .conf will be ignored. This can be useful for enabling/disabling lists.

Once the file has been created, it is time to populate its contents. Each line in the categories file represents a unique matching pattern. There are three types, identified by the prefix used.

• dom A domain match. Any root or subdomain will result in a positive match
• rxp An extended POSIX-compliant regular expression match
• net An IP or CIDR match

An example 10-adult.conf file might start like this:

dom:adultfind.com
rxp:.*porn.*
net:66.254.96.0/19

The "dom" entry ensures this domain is classified as adult since there isn't an application signature for it - just because we know about a domain, doesn't mean it ends up in the application signatures list...again, the 'boil-the-ocean' mantra). The regular expression ("rxp") has a good chance of blocking adult content without any false positives. Finally, the "net" entry provides IP and network matching.

Categories show up in various JSON objects through the Netify solution. For example, in Netify Flow Actions, you may see in a blocked request a category attribute.

"category": {
  "protocol": 9,
  "application": 5,
  "domain": 0,
  "network": 0
}

In the example above:

• protocol The protocol category ID.
• application The application category ID.
• domain The application category ID if a match originates from a categories.d dom or rxpentry.
• network The application category ID if a match originates from a categories.d net entry.

sudo systemctl reload netifyd

Memory Considerations

The Netify agent is not packaged with any lists and is up to the user or integrator to obtain open source or legally acquired lists for each endpoint. The applicability for even a small list of 100,000 entries will require tens of MB of RAM. This may be perfectly acceptable on some hardware or virtual machines, but on many embedded devices, consuming this much memory would not be possible.

Next Steps

• Privacy Settings