Netify DPI - Flow Data Format

This document provides details on the network metadata information available through the Netify DPI Core Plugin.


Cybersecurity engines, QoE managers, performance analytics solutions, and other third-party applications can connect to the Netify DPI Agent to interact with the network flow metadata. The Netify Agent's socket interface (TCP/IP and UNIX sockets) provides near real-time agent status and flow/DPI detection data via JSON encoded payloads.

With the Netify DPI Data Stream, you can develop and create your own integrations: open source, proprietary, in-house... it's up to you.

Connection Tracking


Please see the Core Plugin documentation and Socket Plugin documentation for details on how to configure the Netify DPI agent.

Data Stream Types

The following format information applies to Netifyd version 5.x and later.

There are several different types of data available via the Netify Agent JSON encoded data streams:

  • Flow - Flow detection and risk metadata
  • Flow Purge - Flow purge and performance data
  • Flow Stats - Flow stats time-based data
  • Interfaces - Network interface information with associated MACs and IPs
  • Interface Stats - Network interface stats
  • Endpoints - MACs and IP addresses detected on the network
  • Agent Hello - Agent startup information
  • Agent Status - Agent information: memory, cache info, and summary stats

Information on each type of data is described below. The structures should be self-documenting, but if further explanation or details are required, please don't hesitate to contact us.

Type: Flow - Detection


Detection data is available as soon as the first 1 to 32 packets of a network conversation have been completed. This data provides a path to quickly react to network traffic patterns on the network. It also provides the network metadata extracted from the deep packet inspection engine.

Let's take a look at some sample data (see adjacent JSON data). This particular flow comes from a mobile phone that started up the WhatsApp app. You can see the usual network data in the JSON structure: IP version, MAC addresses, IPs, ports, network interfaces, and timestamps. In addition, you can see the additional DPI data. Read on.

Application: WhatsApp

The detected_application_name shows that it was a flow generated by WhatsApp. In this particular case, this detection was made using the TLS/SSL SNI hostname and certificate common name. Other heuristics are also used to detect applications.

Protocol: HTTPS

The DPI engine detected the HTTPS protocol (detected_protocol). This is more than just mapping port 443 to HTTPS, the engine did a full deep packet inspection and used the Application-Layer Protocol Negotiation (ALPN) data to fine-tune detections. In fact, Netify will detect SSL/SNI over any port.


The application, domain and protocol categories are given in the flow data. In this example, the application category is "Messaging" (#17) and the protocol category is "Web" (#22).


Any flows that trigger a risk event will have details provided in the risk section of the metadata.

Other/Local Nomenclature

The local designation indicates the endpoint's local side, and the other designation will be indicated by the other_type field. These designations do not indicate a flow's direction. To determine which side of a flow started the conversation, consult the local_origin field. When this field is true, it indicates that the local endpoint started transmitting first. When false, the opposite endpoint started the flow.

In the stats data (see below) and referencing the example to the right, we see:

"local_origin": true
"local_ip": "",
"local_bytes": 2434,
"other_type": "remote",
"other_ip": "",
"other_bytes": 6139

In this example, transmitted 2434 bytes and received 6139 bytes from The flow originated from

DPI: HTTPS Details

Detecting the protocol is just the start. The Netify Agent also breaks down the details of the SSL/TLS connection:

  • SNI hostname:
  • Certificate CN: *
  • TLS version: 0x0303 (TLS 1.2)
  • Encryption cipher: 0xc02b info
  • JA3/TLS fingerprint: details
  • Subject DN

Other protocols will have different metadata attached. For example, HTTP traffic also includes the URL and user agent.

  "type": "flow",
  "interface": "ens34",
  "internal": true,
  "established": false,
  "flow": {
    "category": {
      "application": 17,
      "domain": 0,
      "protocol": 22
    "risks": {
      "ndpi_risk_score": 10,
      "ndpi_risk_score_client": 5,
      "ndpi_risk_score_server": 5,
      "risks": [ 15 ]
    "digest": "09c9e2b73d68fef245c09141cb63dad8d9001a6c",
    "ip_nat": false,
    "ip_version": 4,
    "ip_protocol": 6,
    "vlan_id": 0,
    "other_type": "remote",
    "local_origin": true,
    "other_mac": "00:90:fb:29:ca:ba",
    "local_mac": "a0:c9:a0:e5:2c:eb",
    "other_ip": "",
    "local_ip": "",
    "other_port": 443,
    "local_port": 38972,
    "detected_protocol": 196,
    "detected_protocol_name": "HTTPS",
    "detected_application": 544,
    "detected_application_name": "142.netify.whatsapp",
    "detection_guessed": 0,
    "ssl": {
      "alpn": [
      "alpn_server": [],
      "version": "0x0303",
      "cipher_suite": "0xc02b",
      "client_sni": "",
      "server_cn": "*",
      "client_ja3": "d8c87b9bfde38897979e41242626c2f3",
      "server_ja3": "6e15a5bf660856fa03186247ca41d059",
      "issuer_dn": "C=US, O=DigiCert Inc, OU=www...",
      "subject_dn": "C=US, ST=California, L=Menlo..."
    "first_seen_at": 1574786068665,
    "first_update_at": 1574786068665,
    "last_seen_at": 1574786068715

Example Use Cases

As you can imagine, the Netify data stream can be used to interact with the underlying operating system. Some sample uses cases:

  • SD-WAN routing policies based on applications
  • Firewall and QoS policies based on DPI information
  • Wireless access point (WAP) optimization

Type: Flow Purge - Stats and Performance Data


When a network flow is either closed or becomes idle, the Netify Agent publishes a flow_purge record to the Data Stream. Today, this stream provides network statistics: byte counters, traffic counters, timestamps, etc. In the near future, expect to see performance information added to the payload, for example:

  • Statistics on DNS response times
  • HTTP response codes and times
  • Network performance statistics
  • and more
"type": "flow_purge"
"reason": "expired",
"flow": {
  "digest": "178bf5650a79d5e8ddc6a988d0c02b3d799180d0",
  "last_seen_at": 1606232131756,
  "local_bytes": 2434,
  "local_packets": 21,
  "other_bytes": 6139,
  "other_packets": 16,
  "total_bytes": 8573,
  "total_packets": 37,
  "detection_packets": 3,
  ... snip ...

Example Use Cases

  • Cybersecurity analysis
  • Policy and Charging Rules Function (PCRF)
  • Zero-rating policy engines

Type: Flow Stats - Stats Information


Some network flows are long-lived, for example, audio/video streams and VPN connections. The Netify Agent will periodically publish a flow_stats record to the Data Stream. The network status on active flows provides real-time insights into the network.

Example Use Cases

  • Live bandwidth statistics
  • QoE based on live data usage
"type": "flow_stats",
"flow": {
  "digest": "178bf5650a79d5e8ddc6a988d0c02b3d799180d0",
  "last_seen_at": 1606232131756,
  "local_bytes": 2434,
  "local_packets": 21,
  "other_bytes": 6139,
  "other_packets": 16,
  "total_bytes": 8573,
  "total_packets": 37

Type: Interfaces

The interfaces data type provides the MAC address and associated IPs for the given network interface.

"type": "interfaces",
"eth0": {
  "mac": "02:42:ac:11:00:02",
  "role": "LAN"
  "capture_type": "PCAP",
  "addr": [

Type: Interface Stats

The interface_stats data type provides various statistics on the performance of the underlying network interface.

"type": "interface_stats",
"eth0": {
  "capture_dropped": 0,
  "capture_filtered": 0,
  "discarded": 0,
  "discarded_bytes": 0,
  "ethernet": 7,
  "flow_dropped": 0,
  "fragmented": 0,
  "icmp": 0,
  "igmp": 0,
  "ip": 7,
  "ip_bytes": 918,
  "largest_bytes": 215,
  "mpls": 0,
  "pppoe": 0,
  "queue_dropped": 0,
  "raw": 7,
  "tcp": 0,
  "tcp_resets": 0,
  "tcp_seq_errors": 0,
  "udp": 7,
  "vlan": 0,
  "wire_bytes": 1086

Type: Endpoints

The endpoints data type provides the MAC address and associated IPs for detected systems on the network.

"type": "endpoints",
"02:42:3f:e6:a4:ee": [

Type: Agent Hello

The agent_hello data type provides versioning and feature information. This makes it possible for third-party applications to manage upgrades and changes to the Netify Agent data stream.

"type": "agent_hello",
"agent_version": 5.0.1,
"build_version": "Netify Agent/5.0.1 (debian; x86_64; ...)",
"json_version": 1.9

Type: Agent Status

The agent_status data type provides agent information, including memory usage, cache sizes and more.

"type": "agent_status",
"cpu_cores": 8,
"cpu_system": 0.157047,
"cpu_system_prev": 0.151123,
"cpu_user": 0.088339,
"cpu_user_prev": 0.086594,
"dhc_size": 0,
"dhc_status": true,
"flow_count": 7,
"flow_count_prev": 7,
"flows_active": 1,
"flows_expired": 0,
"flows_expiring": 0,
"flows_in_use": 1,
"flows_purged": 0,
"maxrss_kb": 28324,
"maxrss_kb_prev": 28324,
"tcm_kb": 9077,
"tcm_kb_prev": 9100,
"timestamp": 1697557964,
"update_imf": 1,
"update_interval": 15,
"uptime": 450

Evaluate Netify DPI

Do you want to get started with evaluating Netify DPI? Request the Integrators Kit today.

Integrators Kit