Compliance Telemetry Blueprint

In an era of increasing digital oversight, network operators are no longer responsible for connectivity alone; they are also custodians of critical forensic data. From the DPDP Act in India to NIS2 in Europe and Marco Civil in Brazil, regulations increasingly require service providers and private enterprises to maintain high-fidelity records of network activity.

This Compliance Telemetry Blueprint uses DPI and the telemetry captured by the Netify Agent to provide detailed, auditable records of network activity. By capturing 5-tuple and related metadata at the start of each network flow, this blueprint enables network appliances to deliver the forensic visibility and accountability traditionally associated with high-end and carrier-grade equipment.

Benefits

Eliminate the NAT Attribution Problem

In environments using Carrier-Grade NAT (CGNAT), a single public IP can hide thousands of users. Netify can be configured to capture the source port and internal private IP mapping before hardware offload occurs, allowing for 1:1 user identification during legal discovery.

Regulatory Safe Harbor Protection

Offering public Wi-Fi or managed internet exposes businesses to real legal risks. This blueprint equips businesses with reliable, timestamped activity logs that prove their network was just the pipeline. By handing authorities the exact data they need, these logs help shield businesses from liability.

Requirements

Install and configure the Netify Agent
Install and configure the Aggregator Plugin
Install and configure the MQTT Sink Plugin for the telemetry

If you require customized sink/export drivers or a telemetry format for your environment, please contact us. The Netify Agent plugin system provides a flexible, extensible framework for integrating with external systems, enabling you to adapt telemetry output, enrich data, and align with your existing pipelines, storage backends, and analytics workflows with minimal friction.

Configuration

In this blueprint example, our target enterprise already operates a Kafka deployment for IoT systems. To leverage that existing infrastructure, Netify compliance telemetry is published to Kafka Connect via MQTT. Implementing this setup requires two primary configuration steps.

Data Collection via Aggregator Plugin

The Aggregator Plugin is configured to produce compliance-focused telemetry using aggregator type 4, which generates detailed, audit-ready network records. This mode captures enriched flow data with the context required for traceability and reporting, making it well suited for regulatory requirements and post-incident analysis. A log_interval of 60 seconds provides a consistent cadence of aggregated output without overwhelming downstream systems.

To optimize transport and downstream ingestion, the configuration leverages both compression and MessagePack encoding. While the global defaults use JSON with gzip (gz) compression, the aggregate channel overrides the format to msgpack, producing a compact binary payload that reduces serialization overhead. When combined with compression, this significantly lowers bandwidth usage and improves throughput when forwarding data via MQTT into Kafka, particularly in high-volume compliance deployments.

Aggregator Configuration: netify-proc-aggregator.json

{
    "aggregator": 4,
    "log_interval": 60,
    "privacy_mode": false,
    "format": "json", 
    "compressor": "gz",
    "batched_rows": 0,
    "nested": false,
    "sinks": {
        "sink-mqtt": {
            "aggregate": {
                "format": "msgpack",
            }
        }
    }
}

Data Delivery via MQTT Plugin

The MQTT Source Connector on Kafka Connect securely ingests telemetry from an MQTT broker over TLS. Authentication is handled via a dedicated service account, ensuring that only authorized agents can publish data. This provides a reliable bridge between Netify telemetry streams and downstream Kafka infrastructure.

A key aspect of this configuration is the dynamic MQTT topic structure, which incorporates unique channel and agent identifiers into the path. When ingested by Kafka Connect, this topic path can be passed into the record key or headers - guaranteeing clean multi-tenant separation and maintaining device ordering across Kafka partitions in shared environments.

MQTT Configuration: netify-sink-mqtt.json

{
    "host": "kafka-connect.example.com",
    "port": 8883,
    "user": "agent",
    "pass": "secret",
    "topic": "netify/${channel}/${uuid_agent}",
    "message_qos": 1,
    "message_retain": false
}

Conclusion

Compliance is often seen as a burden, but with the right telemetry stack it becomes a competitive advantage. By implementing the Compliance Telemetry Blueprint, vendors can deliver hardware that is "audit-ready" out of the box. Whether meeting regulatory requirements or providing a forensic trail for an insurance claim, Netify DPI ensures the necessary data is captured, attributed, and delivered without compromising network performance.