Netify Agent - Data Stream Types

This document provides details on the information available through the Netify Agent data streams. If you would like to interact with the data in a live environment, please start with the getting started introduction.

There are several different types of data available via the Netify Agent JSON encoded payloads:

  • Flows - Detection Data
  • Flows - Purge and Performance Data
  • Flows - Stats Data
  • Agent - Startup Information
  • Agent - Status Information

Information on each type of data is described below.

Connection Tracking

Flow Data

Detection Data - type: flow

Detection data is available as soon as the first 1-10 packets of a network conversation has been completed. This data provides a path to quickly react to network traffic patterns on the network. It also provides the network metadata extracted from the deep packet inspection engine.

Example use cases:

  • SD-WAN routing policies based on applications
  • Firewall and QoS policies based on DPI information
  • Wireless access point (WAP) optimization
  • Zero rating policy engines
"type": "flow"
"flow": {
  "detected_application_name": "10091.netify.amazon-aws",
  "detected_protocol": 196,
  "detected_protocol_name": "HTTPS",
  "digest": "178bf5650a79d5e8ddc6a988d0c02b3d799180d0",
  "first_seen_at": 1606232125526,
  "first_update_at": 1606232125526,
  "last_seen_at": 1606232125632,
  "local_ip": "192.168.4.189",
  "local_mac": "f8:e9:03:01:69:13",
  "local_origin": true,
  "local_port": 52974,
  "other_ip": "52.216.110.189",
  "other_mac": "00:90:fb:29:ca:ba",
  "other_port": 443,
  "other_type": "remote",
  "ssl": {
    "cipher_suite": "0xc02f",
    "client_ja3": "1fd36067223570569bbf156fece40978",
    "client_sni": "s3.amazonaws.com",
    "server_ja3": "704239182a9091e4453fdbfe0fd17586",
    "version": "0x0303"
  },
  ... snip ...

Stats and Performance Data - type: flow_purge

When a network flow is either closed or becomes idle, the Netify Agent publishes a flow_purge record. Today, this stream provides network statistics: byte counters, traffic counters, timestamps, etc. Later in 2021, we will be adding performance information to the payload, for example:

  • Statistics on DNS response times
  • HTTP response codes and times
  • Network performance statistics
  • Cybersecurity analysis

Requires version 3.06 or later.

"type": "flow_purge"
"reason": "terminate",
"flow": {
  "digest": "178bf5650a79d5e8ddc6a988d0c02b3d799180d0",
  "last_seen_at": 1606232131756,
  "local_bytes": 2434,
  "local_packets": 21,
  "other_bytes": 6139,
  "other_packets": 16,
  "total_bytes": 8573,
  "total_packets": 37
  ... snip ...

Stats Data - type: flow_stats

Some network flows are long-lived, for example audio/video streams and VPN connections. The Netify Agent will periodically publish network statistics on active flows in order to provide real-time insights on the network.

Example use cases:

  • Live bandwidth statistics
  • QoE based on live data usage
  • Policy and Charging Rules Function (PCRF)

Feature coming soon.

"type": "flow_stats",
"flow": {
  "digest": "178bf5650a79d5e8ddc6a988d0c02b3d799180d0",
  "last_seen_at": 1606232131756,
  "local_bytes": 2434,
  "local_packets": 21,
  "other_bytes": 6139,
  "other_packets": 16,
  "total_bytes": 8573,
  "total_packets": 37
},

Agent Data

Agent Info - type: agent_hello

The agent_hello data type provides versioning and feature information. This make it possible for third party applications to manage upgrades and changes to the Netify Agent data stream.

"type": "agent_hello",
"agent_version": 3.06,
"build_version": "Netify Agent/3.06 (debian; x86_64; netlink; dns-cache; plugins; tcmalloc; inotify; regex) nDPI/2.9.0 JSON/1.90",
"json_version": 1.9

Agent Status - type: agent_status

The agent_status data type provides agent and system status information. This make it possible to monitor and manage the performance of the underlying DPI engine.

"type": "agent_status",
"cpu_cores": 1,
"cpu_system": 0.157217,
"cpu_system_prev": 0.103613,
"cpu_user": 0.117912,
"cpu_user_prev": 0.103613,
"dhc_size": 0,
"dhc_status": true,
"flows": 25,
"flows_prev": 22,
"maxrss_kb": 42352,
"maxrss_kb_prev": 42292,
"sink_status": false,
"sink_uploads": false,
"tcm_kb": 22825,
"tcm_kb_prev": 22838,
"timestamp": 1606274389,
"update_imf": 1,
"update_interval": 15,
"uptime": 30

Integration and Custom Solutions

Do you have any questions about integration, APIs or custom development?

Contact Us