Category Domain Lists

Category Lists

• Overview
• Creating Domain Lists
• Memory Considerations

Overview

Netify's DPI Agent and application signature lists actively tracks the top 200 applications in use on the Open Source signature. Netify Informatics subscribers and OEM/integration clients have a license to access and use the commercial signature list, having over 1600 application definitions.

In some cases, having very detailed metadata and classification for an application isn't necessary. One such case is adult content, by some estimates, having over 4M domains. Any user of the Netify agent, regardless of the applications signatures in use, can BYOD (Bring Your Own Domain) list. By creating and maintaining a domain list, the Netify agent's classification will attempt to pattern match and tag in near real time to a category associated with unique flows on the network.

With the category ID of a flow populated, this additional information can then be used with the IPset plugin.

Creating Category Lists

Categories cannot be defined - you must use one of the available category classifications known to Netify. To generate a list of new categories, run:

netifyd --dump-categories
     1: application: adult
     2: application: advertiser
     3: application: business
     4: application: cdn
     5: application: cybersecurity
     6: application: device-iot
     7: application: education
     8: application: entertainment
     9: application: file-sharing
    10: application: financial
    11: application: games
    12: application: government
    13: application: hosting
    14: application: mail
    15: application: malware
    16: application: messaging
    17: application: news
    18: application: os-software-updates
    19: application: portal
    20: application: recreation
    21: application: reference
    22: application: remote-desktop
    23: application: shopping
    24: application: social-media
    25: application: sports
    26: application: streaming-media
    27: application: technology
    28: application: telco
    29: application: unclassified
    30: application: voip
    31: application: vpn-and-proxy
     1: protocol: authentication
     2: protocol: database
     3: protocol: file-server
     4: protocol: file-sharing
     5: protocol: games
     6: protocol: infrastructure
     7: protocol: mail
     8: protocol: media
     9: protocol: media-provider
    10: protocol: messaging
    11: protocol: networking
    12: protocol: printing
    13: protocol: proxy
    14: protocol: remote-desktop
    15: protocol: unclassified
    16: protocol: voip
    17: protocol: vpn
    18: protocol: web

Only application categories (those with the prefix "application: ") can be used. To create a new BYOC list, create a file in the /etc/netifyd/domains.d folder.

sudo touch /etc/netifyd/domains.d/10-adult.conf

• The filename must start with a number followed by a dash separated. This number determines the priority of the list. For example, if you obtained an adult content list from two different sources, the one with the lower priority would be matched first.
• After the dash, the category as listed must follow, in lowercase characters.
• The file name must end in '.conf'. Any filename in the domains.d directory that does not end in .conf will be ignored. This can be useful for enabling/disabling lists.

Once the file has been created, it is time to populate its contents. Each line in the categories file represents a unique domain. Domains are wildcarded by default (ex. abc.example.com would match an example.com entry).

An example 10-adult.conf file might start like this:

adultfind.com

sudo systemctl reload netifyd

Memory Considerations

The Netify agent is not packaged with any lists and is up to the user or integrator to obtain open source or legally acquired lists for each endpoint. The applicability for even a small lists of 100,000 entries will require tens of MB of RAM. This may be perfectly acceptible on some hardware or virtual machines, but on many embedded devices, consuming this much memory would not be possible.