Category Domain Lists

Category Lists

• Overview
• Creating Category Lists
• Memory Considerations
• Next Steps

Overview

Netify's DPI Agent and application signature list actively track the top 200 applications in use on the Open Source signature. Netify Informatics subscribers and OEM/integration clients have a license to access and use the commercial signature list, having over 1600 application definitions.

In some cases, having very detailed metadata and classification for an application isn't necessary. One such case is adult content, by some estimates, having over 4M domains. Any user of the Netify agent, regardless of the application signatures in use, can BYOC (Bring Your Own Category) list. By creating and maintaining a category list, the Netify agent's classification will attempt to pattern match and tag in near real-time a category associated with unique flows on the network.

With the category ID of a flow populated, this additional information can then be used with Netify plugins.

Creating Category Lists

Categories cannot be defined - you must use one of the available category classifications known to Netify. To generate a list of new categories, run:

netifyd --dump-categories
     1: application: adult
     2: application: advertiser
     3: application: business
     4: application: cdn
     5: application: cybersecurity
     6: application: device-iot
     7: application: education
     8: application: entertainment
     9: application: file-sharing
    10: application: financial
    11: application: games
    12: application: government
    13: application: hosting
    14: application: mail
    15: application: malware
    16: application: messaging
    17: application: news
    18: application: os-software-updates
    19: application: portal
    20: application: recreation
    21: application: reference
    22: application: remote-desktop
    23: application: shopping
    24: application: social-media
    25: application: sports
    26: application: streaming-media
    27: application: technology
    28: application: telco
    29: application: unclassified
    30: application: voip
    31: application: vpn-and-proxy
     1: protocol: authentication
     2: protocol: database
     3: protocol: file-server
     4: protocol: file-sharing
     5: protocol: games
     6: protocol: infrastructure
     7: protocol: mail
     8: protocol: media
     9: protocol: media-provider
    10: protocol: messaging
    11: protocol: networking
    12: protocol: printing
    13: protocol: proxy
    14: protocol: remote-desktop
    15: protocol: unclassified
    16: protocol: voip
    17: protocol: vpn
    18: protocol: web

Only application categories (those with the prefix "application: ") can be used. To create a new BYOC list, create a file in the /etc/netifyd/categories.d folder.

sudo touch /etc/netifyd/categories.d/10-adult.conf

• The filename must start with a number followed by a dash. This number determines the priority of the list. For example, if you obtained an adult content list from two different sources, the one with the lower priority would be matched first.
• After the dash, the category as listed must follow, in lowercase characters.
• The file name must end in '.conf'. Any filename in the categories.d directory that does not end in .conf will be ignored. This can be useful for enabling/disabling lists.

Once the file has been created, it is time to populate its contents. Each line in the categories file represents a unique matching pattern. There are three types, identified by the prefix used.

• dom A domain match. Any root or subdomain will result in a positive match
• rxp An extended POSIX-compliant regular expression match
• net An IP or CIDR match

An example 10-adult.conf file might start like this:

dom:adultfind.com
rxp:.*porn.*
net:66.254.96.0/19

The "dom" entry ensures this domain is classified as adult since there isn't an application signature for it - just because we know about a domain, doesn't mean it ends up in the application signatures list...again, the 'boil-the-ocean' mantra). The regular expression ("rxp") has a good chance of blocking adult content without any false positives. Finally, the "net" entry pulls from an ASN from Reflected Networks that hosts this type of content.

sudo systemctl reload netifyd

Memory Considerations

The Netify agent is not packaged with any lists and is up to the user or integrator to obtain open source or legally acquired lists for each endpoint. The applicability for even a small list of 100,000 entries will require tens of MB of RAM. This may be perfectly acceptable on some hardware or virtual machines, but on many embedded devices, consuming this much memory would not be possible.

Next Steps

• Privacy Settings