Chapter 0: Welcome & How to Use This Guide

Welcome to the Data Plane Microservice onboarding guide. This document will take you from zero knowledge of this codebase to being a productive contributor.

What is the Data Plane?

The data-plane (also called eric-pc-up-data-plane) is the packet-forwarding engine inside Ericsson's Packet Core Gateway (PCG) product. It processes millions of packets per second — GTP tunneling, NAT, DPI, QoS enforcement, service chaining — all in a cloud-native Kubernetes pod using DPDK for high-speed I/O.

How to Navigate

• Use the sidebar on the left to jump between chapters
• The progress bar at the top shows how far you've read
• Quizzes (marked with 📝) appear every few chapters — test yourself!
• Code blocks show real patterns from the codebase
• Colored boxes: ℹ️ Info, ⚠️ Warning, 💡 Tip

Repository Structure at a Glance

data-plane/
├── main/           # Main application (dp.c, main.c, config parsers)
├── pktio/          # Packet I/O subsystem (DPDK, AF_XDP, TAP backends)
├── vrf/            # Virtual Routing & Forwarding, GRE, FIB
├── sc/             # Service Chaining engine
├── cgnat/          # Carrier-Grade NAT
├── ipfix/          # IPFIX flow export (NAT logging)
├── upf/            # UPF (User Plane Function) session handling
├── rib-client/     # RIB (Routing Information Base) client
├── punt/           # Punt path (slow-path packets to controller)
├── contest/        # Container integration tests
├── up-common/      # Shared libraries (EVL, mbox, RCU, timers, etc.)
├── cdpi-main/      # DPI (Deep Packet Inspection) integration
├── scripts/        # CI helper scripts
├── dpi-packages/   # DPI heuristics packages
├── Makefile        # Developer shortcuts
├── CMakeLists.txt  # Top-level CMake (in main/)
├── Jenkinsfile*    # CI/CD pipeline definitions
├── ruleset2.0.yaml # Bob build rules
└── common-properties.yaml  # Docker images, repos, versions

Chapter 1: 3GPP User Plane Fundamentals

The Split Architecture

In modern 3GPP networks (4G EPC and 5G Core), the control plane and user plane are separated (CUPS — Control and User Plane Separation):

Key Concepts You Already Know (from AMF)

• AMF handles NAS signaling, registration, mobility — you know this well
• SMF manages sessions and tells the UPF what to do via PFCP
• UPF (our data-plane) does the actual packet forwarding

What the UPF Does

4G vs 5G Terminology

Chapter 2: PFCP Protocol & Session Management

What is PFCP?

PFCP (Packet Forwarding Control Protocol, 3GPP TS 29.244) is the protocol between the SMF (control plane) and UPF (user plane). Think of it as "the boss telling the worker what to do with packets."

PFCP Session Lifecycle

The Rules Model (PDR → FAR/QER/URR)

Each PFCP session contains rules that tell the UPF how to handle packets:

• PDR (Packet Detection Rule) — "Match packets with these criteria" (source interface, GTP TEID, IP filter, etc.)
• FAR (Forwarding Action Rule) — "Do this with matched packets" (forward, drop, buffer, encapsulate in GTP)
• QER (QoS Enforcement Rule) — "Apply this rate limit / QoS marking"
• URR (Usage Reporting Rule) — "Count bytes/packets and report when threshold hit"

How Config Reaches data-plane

The data-plane does NOT speak PFCP directly in PCG mode. Instead:

1. SMF sends PFCP to the UP Control (UPC) microservice
2. UPC translates PFCP rules into a JSON configuration model
3. UPC pushes config to data-plane via CM Mediator (REST/JSON)
4. data-plane parses JSON and installs forwarding rules

// In main/dp_cm_mediator.c — handles config from CM Mediator
// In main/dp_config_json_parser.c — parses the JSON config model
// In main/dp_config_json_validator.c — validates before applying

Chapter 3: Where Data Plane Fits in Ericsson PCG

The PCG Product

PCG (Packet Core Gateway) is Ericsson's cloud-native 5G UPF product. It runs on Kubernetes and consists of multiple microservices:

Key Neighboring Microservices

EPG vs PCG vs VPN-GW

The same data-plane binary is used in multiple products:

• PCG — Cloud-native 5G UPF (Linux containers, DPDK)
• EPG — Evolved Packet Gateway on IPOS (bare-metal blades, WITH_IPOS_SDK)
• VPN-GW — VPN Gateway variant

Chapter 4: High-Level Architecture Overview

Single-Process, Multi-Threaded

The data-plane is a single process with many threads, each pinned to a specific CPU core. This is a classic DPDK pattern — avoid context switches, avoid locks, maximize cache locality.

Key Architectural Principles

• Run-to-completion: Each packet is fully processed on one CPU core (no hand-offs mid-pipeline)
• Lock-free data structures: RCU (Read-Copy-Update) for config updates while forwarding continues
• Shared-nothing where possible: Per-CPU data structures to avoid cache bouncing
• Mailbox for cross-CPU communication: When CPUs must communicate, they use lock-free mailboxes
• Event-driven controller: The controller thread uses EVL (event loop) for async I/O

The Global g_dp Structure

The entire data-plane state is rooted in a single global: dataplane_t* g_dp. This is the "god object" that holds references to all subsystems:

// From main/dp.h
typedef struct dataplane dataplane_t;
extern dataplane_t* g_dp;

dataplane_t*
dataplane_create(
    cb_timer_framework_cpu_t* cb_timer_fws,
    dp_options_t* options,
    random_t* random,
    evl_t* evl,           // Event loop for controller
    evl_t* evl_dbproxy,   // Event loop for DB proxy
    ...
    mbox_t* mbox,         // Mailbox for inter-CPU msgs
    rcu_t* rcu,           // Read-Copy-Update
    pktio_t* pktio,       // Packet I/O subsystem
    ...
);

Chapter 5: CPU Roles

CPU Role Assignment

Each CPU core in the data-plane pod is assigned a specific role. The assignment is dynamic based on available cores:

Controller Thread Details

The controller runs an EVL event loop and handles:

• REST API (health checks: /alive, /ready)
• Configuration reception and parsing (CM Mediator)
• Timer management (session timeouts, keepalives)
• OAM (Operations, Administration, Maintenance)
• Redis/DB operations (session state persistence)
• Kafka producer (IPFIX events, logs)
• PFCP relay (in some modes)

Worker Thread Details

Workers run a tight poll loop (no sleeping!) that:

1. Polls the NIC RX queue (via DPDK/eventdev)
2. Classifies the packet (GTP? IP? ARP?)
3. Looks up the session/PDR
4. Applies service chain (DPI, NAT, QoS, firewall)
5. Encapsulates if needed (GTP, GRE)
6. Transmits on the TX queue

// Simplified worker loop concept (from pktio/src/pktio.c)
while (running) {
    nb_pkts = rte_eth_rx_burst(port, queue, pkts, BURST_SIZE);
    for (i = 0; i < nb_pkts; i++) {
        classify_packet(pkts[i]);
        apply_service_chain(pkts[i]);
        transmit_packet(pkts[i]);
    }
    mbox_poll(mbox);  // Check for config updates
}

Chapter 6: Packet Processing Pipeline

The Fast Path

The "fast path" is the optimized packet processing pipeline that handles the vast majority of traffic without involving the controller:

Slow Path (Punt)

Some packets can't be handled on the fast path and are "punted" to the Input/Controller CPU:

• ARP requests/replies
• BFD (Bidirectional Forwarding Detection) keepalives
• ICMP (ping, unreachable)
• LACP (Link Aggregation Control Protocol)
• First packet of a new flow (if buffering is needed)
• Packets requiring fragmentation/reassembly

Network Instance (NWID)

A Network Instance (NI, also called NWID in code) is the data-plane's equivalent of a VRF. Each DNN/APN maps to a network instance with its own:

• Routing table (FIB)
• IP address space
• GRE tunnels
• Service chain configuration

Chapter 7: Traffic Spraying

Why Spraying?

With multiple worker CPUs, incoming traffic must be distributed. Spraying is how packets are assigned to worker cores. The goal: even load distribution while keeping related packets on the same core (for stateful processing).

Spraying Modes

Eventdev (Hardware-Assisted Spraying)

On supported NICs, DPDK's eventdev framework can do spraying in hardware/firmware, offloading the ingress CPU. See pktio/src/eventdev.c.

// From pktio/src/eventdev.h — eventdev configuration
// Eventdev distributes packets to worker cores based on flow_id
// This avoids software-based spraying overhead

Chapter 8: Packet I/O (PKTIO)

Overview

The pktio/ directory is the packet I/O subsystem — the interface between the data-plane and the network. It abstracts multiple backends behind a common API.

PKTIO Backends

Key PKTIO Concepts

• Packet classification (classifier.c) — Determines packet type (GTP, GRE, plain IP, ARP, etc.)
• Five-tuple extraction (five_tuple.c) — Extracts src/dst IP, ports, protocol for flow hashing
• Flow table (flow_table.c) — Tracks active flows for stateful processing
• NWID table (nwid_table.c) — Maps tunnel keys to network instance IDs
• Encapsulation (pktio_encap*.c) — GRE, MPLS, IP encap/decap
• Packet flow expiry (packet_flow_expiry.c) — Timeout inactive flows

// The main PKTIO public API (pktio/include/pktio/pktio.h)
// ~50K lines — this is the largest header in the project
// Key functions:
pktio_t* pktio_create(...);
void pktio_start(pktio_t* pktio);
void pktio_stop(pktio_t* pktio);
int pktio_rx_burst(pktio_t* pktio, pktio_packet_t* pkts, int max);
int pktio_tx_burst(pktio_t* pktio, pktio_packet_t* pkts, int n);

FIB (Forwarding Information Base)

The pktio/fib/ subdirectory contains the FIB — the routing lookup table used during packet forwarding. It's a longest-prefix-match (LPM) structure optimized for fast lookups.

Chapter 9: UPF Engine & DPI

UPF Module (upf/)

The upf/ directory handles UPF-specific session logic — PDR installation, FAR application, URR counting. It works closely with the service chaining engine.

DPI (Deep Packet Inspection)

DPI is handled by the cdpi-main/ directory and the external dpisf library. It classifies application-layer traffic (YouTube, Netflix, WhatsApp, etc.) using:

• Heuristics — Pattern matching on packet payloads
• SNI inspection — TLS Server Name Indication
• DNS correlation — Map DNS responses to flows

DPI Thread Controller

DPI can run on dedicated threads to avoid impacting forwarding latency. See main/dp_config_dpi_thread.c and main/dp_config_dpi_thread_controller.c.

Chapter 10: VRF, GRE & FIB

VRF Module (vrf/)

The VRF (Virtual Routing and Forwarding) module manages network instances — isolated routing domains, each with their own FIB, GRE tunnels, and interfaces.

// VRF source files (from vrf/CMakeLists.txt)
src/config.c                // VRF configuration handling
src/config_delta_builder.c  // Incremental config changes
src/controller.c            // VRF controller (control plane side)
src/engine.c                // VRF engine (data plane side)
src/routes.c                // Route management
src/mac_learning_mgr.c      // MAC address learning
src/cre_route.c             // CRE (Cloud Routing Engine) routes

GRE Tunnels

GRE (Generic Routing Encapsulation) tunnels connect the data-plane to the transport network. Each network instance can have multiple GRE tunnels. The GRE subsystem lives in vrf/gre/.

FIB Controller (vrf/fib-ctrl/)

The FIB controller manages route installation/removal. Routes come from:

• Static configuration (CM Mediator)
• Routing Agent (dynamic routes via gRPC)
• Connected routes (interface addresses)
• CRE (Cloud Routing Engine)

Route Announcements

The data-plane announces its routes to the Routing Agent so that external routers know how to reach UE addresses. See main/dp_config_route_announcements.c.

Chapter 11: CGNAT, Service Chaining & Protocols

CGNAT (cgnat/)

Carrier-Grade NAT translates private IPv4 addresses to shared public IPs. Key components:

• IP Pool Manager — Allocates public IPs and port ranges
• Translation Table — Maps (private IP:port) → (public IP:port)
• Service Functions — Per-protocol NAT (TCP, UDP, ICMP)
• Garbage Collector — Reclaims expired translations
• IPFIX logging — Reports NAT events for lawful intercept compliance

Service Chaining (sc/)

The Service Chaining engine is the heart of packet processing. It executes an ordered list of "service functions" on each packet:

Key files in sc/:

• sc.c — Main service chain controller
• engine.c — Fast-path service chain execution
• session.c — Session data management
• sf.c — Service function framework
• api.c — Service chain API (170K lines!)
• confighandler.c — Configuration handling
• tromboning.c — Traffic steering to external service functions

IPFIX (ipfix/)

IPFIX (IP Flow Information Export) generates NAT logging records. Required for lawful intercept — when CGNAT is used, operators must log which subscriber had which public IP:port at what time.

Protocol Handling

The data-plane handles several control protocols on the slow path:

Chapter 12: Module Lifecycle API

The Module Pattern

Every subsystem in data-plane follows a strict module lifecycle pattern defined in MODULE_GUIDELINES.md. This ensures controlled startup and shutdown.

State Machine

API Contract

// Every module exposes these functions:

// Constructor — allocate and initialize
my_module_t* my_module_create(dependencies...);

// Start — begin async operations, call on_started_cb when ready
void my_module_start(my_module_t* m, on_started_cb, cb_ctx);

// Stop — gracefully shut down, call on_stopped_cb when done
void my_module_stop(my_module_t* m, on_stopped_cb, cb_ctx);

// Destructor — free all resources (only after stopped)
void my_module_delete(my_module_t* m);

Key Rules

• Start must not fail. If resources are missing, retry until available.
• Start is called only once. Use module_state_t + abort_unless to detect violations.
• Stop must be callable before start completes. It inhibits the on_started_cb.
• After on_stopped_cb, the module is safe to delete. All timers, defers, and transactions must be complete.
• Dependencies are injected in create/start. Never expose them via getters.

Chapter 13: Control-Engine Pattern & Mailbox

Control-Engine Split

Many modules have two parts:

• Controller — Runs on the controller CPU, handles config, manages state
• Engine — Runs on worker CPUs, processes packets at line rate

Example: VRF has vrf/src/controller.c (control) and vrf/src/engine.c (data path).

Mailbox (mbox)

The mailbox is the inter-CPU communication mechanism. It's a lock-free SPSC (single-producer, single-consumer) or MPSC queue.

// Sending a message from controller to worker
mbox_msg_t msg = {
    .type = MBOX_MSG_CONFIG_UPDATE,
    .data = new_config_ptr
};
mbox_send(worker_mbox, &msg);

// Worker polls mailbox in its main loop
while (mbox_recv(my_mbox, &msg)) {
    handle_message(&msg);
}

Mbox Dispatcher

The mbox_dispatcher provides a higher-level API for synchronous and asynchronous cross-thread tasks:

// Execute a function on a specific worker thread
mbox_dispatcher_dispatch(dispatcher, target_cpu, my_function, arg);

// Used for operations that need specific thread context
// (e.g., timer start/stop must happen on the timer's owning thread)

Thread Context Rules

Chapter 14: RCU, Async Config & EVL Batch Iterator

RCU (Read-Copy-Update)

RCU allows the controller to update configuration while workers continue reading the old version without locks:

// Writer (controller thread)
new_config = build_new_config(...);
rcu_assign_pointer(global_config, new_config);
rcu_synchronize(rcu);  // Wait for all readers to finish with old
free(old_config);

// Reader (worker thread) — no locks needed!
rcu_read_lock(rcu);
config = rcu_dereference(global_config);
use_config(config);
rcu_read_unlock(rcu);

Async Configuration Model

From CONFIG_GUIDELINES.md: Configuration is applied asynchronously using a two-step model:

1. Step 1 — Build config: Parent module builds a module-specific config object
2. Step 2 — Apply config: Child module applies it via deferred jobs

// Every module exposes:
void module_set_config(
    module_t* module,
    module_config_t* config,
    on_done_cb_t on_done,      // Called when ALL deferred work completes
    void* on_done_ctx,
    void* on_done_arg
);

// Key rules:
// - All deferred work MUST be started inside set_config
// - on_done MUST be called exactly once, after all work completes
// - on_done may NOT be interrupted or cancelled

EVL Batch Iterator

For "large number objects" (e.g., thousands of network instances), you must NOT iterate in a blocking loop. Use the EVL Batch Iterator:

// BAD — blocks the event loop for too long
for (i = 0; i < 10000; i++) {
    configure_network_instance(ni[i]);  // ❌ Blocks!
}

// GOOD — process one at a time with low-priority defers
evl_batch_iterator_config_t cfg = {
    .defer_priority = EVL_DEFER_PRIORITY_LOW,
    .batch_size = 1,
    .on_item = configure_one_ni,
    .on_done = all_ni_configured_cb
};
evl_batch_iterator_start(&cfg, ni_list, count);

Chapter 15: up-common & EVL (Event Loop)

What is up-common?

The up-common/ directory is a git submodule containing shared libraries used across all UP (User Plane) microservices. Think of it as the "standard library" for the data-plane ecosystem.

Key Libraries in up-common

EVL (Event Loop)

EVL is the async I/O framework used by the controller thread. It wraps Linux epoll and provides:

// Create an event loop
evl_t* evl = evl_create();

// Register a file descriptor for read events
evl_add_fd(evl, fd, EVL_READ, my_callback, my_arg);

// Schedule a deferred task (runs on next iteration)
evl_defer(evl, my_deferred_fn, arg);

// Schedule a low-priority deferred task
evl_defer_low(evl, my_low_prio_fn, arg);

// Run the event loop (blocks until stopped)
evl_run(evl);

Chapter 16: Timers, Defers & Clients

Timer Framework (cbtimer)

The cbtimer library provides callback-based timers. Each CPU has its own timer framework instance (no cross-CPU timer operations without mbox).

// Create a timer
cb_timer_t* timer = cb_timer_create(fw, my_timeout_cb, arg);

// Start with 5 second timeout
cb_timer_start(timer, duration_from_seconds(5));

// Cancel
cb_timer_stop(timer);

// Destroy
cb_timer_delete(timer);

Timer Wheel (tw-client)

For scenarios with thousands of timers (e.g., session timeouts), the timer wheel is more efficient than individual cbtimers. It batches timer expirations.

Deferred Work

Defers are "do this later" tasks scheduled on the event loop:

• evl_defer() — Normal priority, runs soon
• evl_defer_low() — Low priority, yields to important work
• evl_defer_idle() — Runs only when nothing else is pending

REST Client & HTTP

The http library provides both server (for health endpoints) and client (for CM Mediator communication). The rest_client is a higher-level wrapper for making REST API calls.

tw_client (Timer Wheel Client)

Used for bulk session timers. The data-plane can have millions of active sessions, each with timeout timers. The timer wheel handles this efficiently with O(1) start/stop operations.

Chapter 17: ET Test Framework & dstrace

ET (Ericsson Test) Framework

All unit tests and SFTs use the ET framework — a C test framework from up-common. It provides:

// Test file structure
#include "et/et.h"

ET_TEST(my_module, test_create_and_delete) {
    my_module_t* m = my_module_create(deps);
    ET_ASSERT(m != NULL);
    my_module_delete(m);
}

ET_TEST(my_module, test_start_stop) {
    my_module_t* m = my_module_create(deps);
    my_module_start(m, on_started, ctx);
    // ... drive event loop until started ...
    my_module_stop(m, on_stopped, ctx);
    // ... drive event loop until stopped ...
    my_module_delete(m);
}

ET_TEST_SUITE(my_module) {
    ET_RUN_TEST(my_module, test_create_and_delete);
    ET_RUN_TEST(my_module, test_start_stop);
}

SFT (Signal Flow Tests)

SFTs test the interaction between modules. They use stub/mock versions of dependencies (note the -sft library variants in CMakeLists.txt):

• up-dp-gwu-sft — SFT variant of the main GWU library
• up-dp-cgnat-sft — SFT variant of CGNAT
• Compiled with ENV_UT and UP_SFT defines

dstrace (Distributed Tracing)

The dstrace library integrates with OpenTelemetry for distributed tracing across microservices. It allows you to trace a request from SMF → UPC → data-plane.

// Create a span for a config operation
dstrace_span_t* span = dstrace_span_start(tracer, "apply_config");
// ... do work ...
dstrace_span_end(span);

Chapter 18: CMake Structure & 3PP Management

CMake Organization

The project uses CMake as its build system. Each subdirectory has its own CMakeLists.txt:

data-plane/
├── main/CMakeLists.txt      # Main executable + libraries
├── pktio/CMakeLists.txt     # Packet I/O library
├── vrf/CMakeLists.txt       # VRF library
├── cgnat/CMakeLists.txt     # CGNAT library
├── sc/CMakeLists.txt        # Service Chaining library
├── ipfix/CMakeLists.txt     # IPFIX library
├── contest/CMakeLists.txt   # Integration tests
└── up-common/               # Submodule with its own CMake

Library Naming Convention

Libraries follow the pattern up-dp-{module} with SFT variants:

• up-dp-vrf — Production VRF library
• up-dp-vrf-sft — SFT variant (stubs, test hooks)
• up-dp-gwu — Main "gateway user-plane" library (the big one)
• up-dp-gwu-sft — SFT variant of GWU

Build Targets

# The main executable
add_executable(data-plane main.c)

# Links against everything
target_link_libraries(data-plane PRIVATE
    up-dp-gwu        # Main library
    up-dp-vrf        # VRF
    up-dp-cgnat      # CGNAT
    up-dp-sc         # Service Chaining
    up-dp-ipfix      # IPFIX
    up-dp-pktio      # Packet I/O
    up-evl           # Event loop
    up-mbox          # Mailbox
    up-rcu           # RCU
    ...
)

3PP (Third-Party Packages)

Third-party dependencies are managed via the staging directory. Running bob generate:3pp downloads and unpacks them:

# Key 3PP dependencies:
- DPDK          # Data Plane Development Kit
- jansson       # JSON parsing
- hiredis       # Redis client
- flatbuffers   # Serialization (for Kafka messages)
- grpc/protobuf # gRPC (routing agent communication)
- openssl       # TLS
- abseil (absl) # Google's C++ utilities
- yyjson        # Fast JSON parser

Chapter 19: Bob Tool & Rulesets

What is Bob?

Bob is Ericsson's ADP (Application Development Platform) build orchestration tool. It reads ruleset2.0.yaml files and executes build rules in Docker containers for reproducibility.

Key Bob Commands

# Initialize bob (downloads bob binary)
$ bob/bob init-dev

# Generate 3PP staging (download dependencies)
$ bob/bob generate:3pp

# Generate CMake build system
$ bob/bob -p build-dir=builds/san generate:cmake

# Build C++ targets
$ bob/bob -p build-dir=builds/san build:cpp

# Run tests
$ bob/bob -p build-dir=builds/san test:cpp

# Build Docker image
$ bob/bob -p build-dir=builds/release build image package

Ruleset Structure (ruleset2.0.yaml)

The ruleset defines build rules, Docker images, properties, and task sequences. Key sections:

• docker-images — Build environment containers
• properties — Variables (repos, versions, paths)
• rules — Named tasks (generate, build, test, image, etc.)
• env — Environment variables

Bob Parameters (-p)

Chapter 20: Makefile Developer Shortcuts

The Developer Makefile

The top-level Makefile wraps bob commands into convenient shortcuts. Run make help to see all targets:

# Quick test (assert build, fast)
$ make test

# Test with sanitizers (catches memory bugs)
$ make testsan

# Test a specific suite
$ make test t=my_module_SUITE

# Build a Docker image for system test
$ make image

# Run contest (container integration tests)
$ make contest

# Get test coverage report
$ make testcov

# Run linter (clang-tidy on your commit)
$ make lint

# Generate compile_commands.json for IDE
$ make lsp

# Clean build artifacts
$ make clean

# Nuclear option — delete everything
$ make realclean

Build Variants

ccache

The Makefile auto-detects ccache directories (/local/scratch/ccache or /local/persistent_docker/ccache) to speed up rebuilds. If you have ccache configured, subsequent builds are much faster.

Chapter 21: Unit Tests & SFT

Test Organization

Tests live in test/ subdirectories within each module:

cgnat/test/ut_cgnat_*.c         # CGNAT unit tests
pktio/test/test_pktio*.c        # PKTIO tests
sc/test/utest_*.c               # Service Chaining tests
ipfix/test/ut_ipfix_*.c         # IPFIX tests
vrf/test/                       # VRF tests
main/linux/test/test_linux.c    # Linux-specific tests

Running Tests

# Run all tests
$ make test

# Run specific test suite
$ make test t=ut_cgnat_ip_pool_SUITE

# Run with sanitizers
$ make testsan t=ut_cgnat_ip_pool_SUITE

# Under the hood, bob uses CTest:
$ cd builds/san && ctest -R ut_cgnat

Writing a Unit Test

#include "et/et.h"
#include "cgnat/cgnat.h"

// Setup/teardown per test
static cgnat_t* cgnat;

static void setup(void) {
    cgnat = cgnat_create(test_deps);
}

static void teardown(void) {
    cgnat_delete(cgnat);
}

ET_TEST(cgnat_pool, allocate_returns_valid_ip) {
    ip_addr_t addr = cgnat_pool_allocate(cgnat->pool);
    ET_ASSERT(addr.s_addr != 0);
    ET_ASSERT_EQ(addr.s_addr, expected_first_ip);
}

ET_TEST_SUITE(cgnat_pool) {
    ET_SETUP(setup);
    ET_TEARDOWN(teardown);
    ET_RUN_TEST(cgnat_pool, allocate_returns_valid_ip);
}

SFT vs UT

Chapter 22: Contest & VETO

Contest (Container Integration Tests)

Contest runs the actual data-plane binary in a Docker container with simulated network interfaces. It tests end-to-end packet forwarding.

# Run contest locally
$ make contest

# What happens:
# 1. Builds data-plane with sanitizers
# 2. Builds Docker image
# 3. Starts docker-compose with:
#    - data-plane container (SUT)
#    - test driver container
#    - mock services (Redis, CM Mediator, etc.)
# 4. Runs test scenarios

Contest Structure

contest/
├── contest.yaml          # Docker-compose definition
├── run.sh                # Test runner script
├── main.c                # Test driver entry point
├── framework/            # Test utilities
│   ├── contest.c         # Framework core
│   ├── pfcp.c            # PFCP message builder
│   ├── config.c          # Config injection
│   ├── session.c         # Session management
│   ├── io.c              # Packet send/receive
│   └── sut.c             # System Under Test control
├── tests/                # Test cases
│   ├── test_pgwu.c       # PGW-U scenarios
│   ├── test_colocated.c  # Co-located SGW-U + PGW-U
│   ├── test_vpngw.c      # VPN-GW scenarios
│   └── test_redis.c      # Redis/geo-redundancy tests
└── simulator/            # Mock external services

VETO (System Test)

VETO is the full system test that runs on real hardware or cloud infrastructure with the complete PCG deployment. It's triggered by the CI pipeline (JenkinsfileVeto).

Chapter 23: Fuzz Testing, ISSU & Benchmarks

Fuzz Testing

The project includes fuzz tests that feed random/malformed data to parsers:

• pktio/test/fuzz_pktio.c — Fuzz the packet parser with random bytes
• Uses libFuzzer (Clang's built-in fuzzer)
• Catches crashes, buffer overflows, undefined behavior

ISSU (In-Service Software Upgrade)

ISSU tests verify that the data-plane can be upgraded without dropping traffic. The new version takes over sessions from the old version via shared state in Redis.

Benchmarks

Performance benchmarks measure packets-per-second and latency:

• pktio/test/bench_pktio.c — Raw packet I/O throughput
• pktio/test/bench_encap.c — Encapsulation performance
• pktio/test/bench_encap_mpls.c — MPLS encap performance
• pktio/test/bench_ingress_spraying.c — Spraying overhead

Stress Tests

pktio/test/pktio_stress.c — Long-running stress test that exercises the packet path under load to find race conditions and memory leaks.

Chapter 24: PreCodeReview Pipeline

What Triggers It

Every time you push a patch to Gerrit, the JenkinsfilePreCodeReview pipeline runs automatically. It's the gatekeeper — your patch won't get merged if this fails.

Pipeline Stages

What Gets Checked

• Compilation: Must compile cleanly with Clang (warnings are errors)
• Tests: All tests must pass (assert build + sanitizer build)
• Clang-tidy: No new warnings in changed files
• Commit message: Must follow format (Jira ticket, proper subject line)
• Contest: End-to-end tests must pass

Chapter 25: Drop Pipeline

What is a "Drop"?

A drop is a versioned release candidate. The Drop pipeline (JenkinsfileDrop) runs on the main branch after patches are merged. It produces artifacts that can be deployed.

Drop Pipeline Stages

1. Build — Full release build (LTO, optimized)
2. Test — Complete test suite (UT + SFT + contest)
3. Image — Build production Docker image
4. Helm Package — Package Helm chart
5. Publish — Push image to Docker registry, chart to Helm repo
6. VA Scan — Vulnerability scanning (Trivy, Grype)
7. Design Rule Check — ADP compliance checks
8. Version — Tag with version number

Artifact Repositories

Chapter 26: PRA (Release) Pipeline

PRA = Product Release Approval

The PRA pipeline produces the official released version that goes to customers. It runs additional quality gates beyond the Drop pipeline:

• Full VA (Vulnerability Assessment) scan suite
• FOSSA license compliance check
• EVMS (Ericsson Vulnerability Management System) registration
• Munin/PLMS product lifecycle registration
• Documentation publishing
• Released artifact promotion (dev → released repos)

Version Scheme

# Version format: PREFIX + auto-increment
$ cat VERSION_PREFIX
CXU1012345

# Full version example: CXU1012345_1_R2A (drop 2, revision A)

Chapter 27: VA & SoC Pipelines

VA 2.0 (Vulnerability Assessment)

The VA pipeline scans for security vulnerabilities using multiple tools:

Daily Pipeline

The JenkinsfileDaily runs nightly and catches issues that don't block individual patches:

• Full test suite with extended timeout
• Memory leak detection (long-running tests)
• Performance regression detection

Readiness Check

JenkinsfileReadinessCheck verifies that the service meets ADP readiness criteria before release.

Chapter 28: Docker Image

Image Structure

The data-plane Docker image is built from a minimal base (ADP Base OS) and contains:

• The data-plane binary (statically linked, stripped for release)
• DPI heuristics packages
• DPDK drivers and configuration
• Health check scripts
• Minimal OS utilities

Building an Image for ST

# Build a release image
$ make image

# Output tells you the tag to use in ST:
# helm.flags=eric-pc-up-data-plane.imageCredentials.repoPath=proj-pc-dev,
#            eric-pc-up-data-plane.images.dataPlane.tag=1.2.3.unstripped

Image Variants

• Release (stripped) — Production image, no debug symbols
• Unstripped — Has debug symbols for crash analysis
• Debug — Built with -O0 for GDB attachment

Contest Dockerfile

Contest uses a separate contest/Dockerfile that includes the test driver alongside the SUT (System Under Test).

Chapter 29: Helm Chart & Kubernetes

Helm Chart: eric-pc-up-data-plane

The data-plane is deployed as a Kubernetes DaemonSet or Deployment via Helm. Key resources:

• DaemonSet/Deployment — The data-plane pods
• ConfigMap — Static configuration
• Service — Internal service discovery
• ServiceAccount — RBAC permissions
• NetworkAttachmentDefinition — SR-IOV interface definitions
• PodDisruptionBudget — Ensures availability during upgrades

Key Helm Values

# Typical values override for ST:
eric-pc-up-data-plane:
  replicaCount: 2
  resources:
    dataPlane:
      requests:
        cpu: "8"
        memory: "16Gi"
        hugepages-1Gi: "4Gi"
      limits:
        cpu: "8"
        memory: "16Gi"
        hugepages-1Gi: "4Gi"
  imageCredentials:
    repoPath: proj-pc-dev
  images:
    dataPlane:
      tag: "your-version"

Pod Resource Requirements

• CPU: Dedicated cores (guaranteed QoS class) — typically 4-16 cores
• Memory: Large allocation for packet buffers and session tables
• Hugepages: Required by DPDK for DMA-capable memory (1GB pages)
• SR-IOV VFs: Virtual Functions for direct NIC access

Chapter 30: SR-IOV, DPDK & Cloud-Native I/O

Why DPDK?

Standard Linux networking (kernel TCP/IP stack) adds too much overhead for millions of packets/second. DPDK bypasses the kernel entirely:

SR-IOV (Single Root I/O Virtualization)

SR-IOV allows a physical NIC to present multiple Virtual Functions (VFs) that appear as independent NICs to the pod. Each data-plane pod gets its own VF with direct hardware access.

Memory: Hugepages

DPDK requires hugepages (1GB or 2MB pages) for:

• Packet buffer pools (mbufs)
• Ring buffers (NIC queues)
• Hash tables (flow tables, session tables)

Without hugepages, TLB misses would destroy performance.

CPU Isolation

Data-plane pods use CPU Manager (Kubernetes) to get dedicated cores. The kernel's scheduler never runs other workloads on these cores, ensuring consistent latency.

Chapter 31: Linting & CodeChecker

Clang-Tidy

The project uses clang-tidy for static analysis. Configuration is in cdpi-main/.clang-tidy (29K lines!) which enables/disables specific checks.

# Run clang-tidy on your changes
$ make lint

# What it checks:
# - Bugprone patterns (use-after-move, suspicious comparisons)
# - Modernize suggestions (C++17 features)
# - Performance issues (unnecessary copies)
# - Readability (naming conventions)
# - Custom Ericsson checks

CodeChecker

For deeper analysis, CodeChecker runs Clang Static Analyzer:

# Run CodeChecker (slower but more thorough)
$ make codechecker

# Generates HTML report at:
# builds/assert/code-checker-report/
# View with: python3 -m http.server -d ./builds/assert/code-checker-report

Commit Linting

The CI also checks commit message format:

# Good commit message:
PCPB-12345: Add timeout handling to CGNAT garbage collector

Implement periodic cleanup of expired NAT translations.
The garbage collector now runs every 30 seconds and removes
translations that have been idle beyond the configured timeout.

# Bad (will be rejected):
fix stuff

Chapter 32: Design Rule Checks

ADP Design Rules

Ericsson's ADP (Application Development Platform) defines mandatory design rules for cloud-native services. The CI checks compliance automatically.

Helm DR (Design Rules)

Checked by adp-helm-dr-check Docker image. Validates:

• Chart structure and naming conventions
• Required labels and annotations
• Resource limits defined
• Security context (non-root, read-only filesystem)
• Service mesh compatibility
• Graceful shutdown support

Image DR (Design Rules)

Checked by adp-image-dr-check. Validates:

• Base image is ADP Base OS
• No unnecessary packages installed
• Non-root user
• Proper labels (version, name, vendor)
• No hardcoded secrets
• Minimal attack surface

PLM DR (Product Lifecycle Management)

Ensures proper product registration in Ericsson's systems (Munin, EVMS).

Chapter 33: VA Scans, FOSSA & EVMS

Vulnerability Assessment Flow

FOSSA (License Compliance)

FOSSA scans all dependencies for license compliance. It ensures:

• No GPL-licensed code in the product (incompatible with proprietary)
• All 3PP licenses are documented
• Attribution notices are included
• No license conflicts between dependencies

EVMS (Ericsson Vulnerability Management System)

All known CVEs in the product are registered in EVMS. The team must:

• Acknowledge each CVE
• Assess impact (exploitable in our context?)
• Plan remediation (upgrade dependency, apply patch, accept risk)

Munin / PLMS

Product registration system. Each release is registered with:

• Version number
• Bill of materials (all 3PP components)
• License information
• Security assessment status

Chapter 34: Gerrit Workflow & Making a Test Image

Gerrit Patch Lifecycle

Commit Message Format

PCPB-XXXXX: Short description (max ~72 chars)

Longer description explaining WHY the change is needed.
What problem does it solve? What approach was taken?

- Bullet points for multiple changes
- Reference related patches if needed

Change-Id: I1234567890abcdef  (auto-generated by git hook)

Making a Test Image for ST

# Step 1: Build the release image
$ make image

# Step 2: Note the output — it gives you the Helm override:
# helm.flags=eric-pc-up-data-plane.imageCredentials.repoPath=proj-pc-dev,
#            eric-pc-up-data-plane.images.dataPlane.tag=X.Y.Z.unstripped

# Step 3: In ST (Beets), paste that into "Custom Arguments"

# Alternative: Push to your personal repo
$ docker tag data-plane:latest serodocker.sero.gic.ericsson.se/proj-pc-dev/data-plane:my-test
$ docker push serodocker.sero.gic.ericsson.se/proj-pc-dev/data-plane:my-test

Amending a Patch

# After review feedback, amend your commit:
$ git add -p                    # Stage fixes
$ git commit --amend            # Amend (keeps Change-Id)
$ git push origin HEAD:refs/for/master  # Push new patchset

Rebasing on Latest Master

$ git fetch origin
$ git rebase origin/master
# Fix conflicts if any
$ git push origin HEAD:refs/for/master

Chapter 35: Debugging, Common Pitfalls & Useful Links

Debugging with GDB

# Build debug variant
$ make builds/debug

# Run under GDB (for unit tests)
$ cd builds/debug
$ gdb ./my_test_binary
(gdb) break my_function
(gdb) run

# Attach to running container (for contest/ST)
$ kubectl exec -it data-plane-pod -- /bin/sh
$ gdb -p $(pidof data-plane)

Reading Core Dumps

# With unstripped image:
$ gdb ./data-plane core.12345
(gdb) bt          # Backtrace
(gdb) info threads # All threads
(gdb) thread 3    # Switch to thread 3
(gdb) bt          # Backtrace of that thread

Common Pitfalls

Useful Make Targets Cheat Sheet

make test              # Quick test (assert build)
make testsan           # Test with sanitizers
make test t=FOO_SUITE  # Run specific suite
make contest           # Container integration test
make image             # Build Docker image for ST
make lint              # Clang-tidy on your commit
make lsp               # Generate compile_commands.json
make testcov           # Coverage report
make codechecker       # Deep static analysis
make clean             # Clean builds (keep configs)
make realclean         # Delete everything

Useful Links

• Gerrit: Your code review platform
• Jenkins: CI/CD dashboards
• Jira (PCPB project): Feature tracking
• Confluence: Team documentation
• Artifactory: Artifact repositories
• Grafana: Production metrics dashboards