As network bandwidth grows, packet processing can consume CPU capacity that would otherwise support application workloads. NIC offloading—moving selected computation from a host CPU to a programmable network interface controller (NIC)—offers one way to address that pressure. But choosing which work to move, how to exchange data between devices and how to preserve application behavior can make implementation time-consuming.
FLOEM is a programming system designed to make those choices easier to explore. Its data-flow language, compiler and runtime help developers describe packet-processing logic, map work to CPU or NIC resources, and manage communication between them. Although the system was developed for data-center server applications, its design offers useful lessons for technical teams evaluating how to structure network-intensive software and infrastructure.
Why NIC offloading takes more than moving code
A CPU-NIC application is not simply a CPU program with a few functions relocated. A change in placement can alter which packet fields need to cross the device boundary, how messages are queued, and how components run in parallel. Developers may need repeated design, implementation and testing cycles before an offloaded version performs better than a CPU-only baseline.
Communication is a central part of the problem. A design may need to steer packets to different CPU threads, share a physical communication channel among message types, or preserve message order to maintain correct application state. Moving computation can also make manually assembling the right packet fields error-prone. If an application caches state or computation on the NIC, it must account for consistency between the NIC and host.
These concerns also matter in industrial computing environments where network traffic and application processing compete for finite resources. The relevant question is not whether all processing belongs on the NIC, but whether a particular function benefits from a different execution location—and whether the surrounding data path remains correct and manageable.
How FLOEM structures NIC offloading
FLOEM represents an application as connected elements. Each element processes inputs and sends results through output ports. Developers can implement an element in C, connect elements into a data-flow program and assign groups of work to a CPU or NIC. They can also configure parallel execution across available cores.
A segment is a connected group of elements that runs to completion for a packet before processing the next one. Queues connect segments, including segments placed on different devices. This separation gives developers a way to change where work runs without redesigning the entire application as one monolithic program.
FLOEM also distinguishes logical queues—the communication paths developers use in the program—from physical queues implemented by hardware. Developers can map multiple logical queues onto physical queues to express different communication strategies. For example, multiple physical queues can support packet steering to separate processing resources, while sharing a physical queue can combine traffic and preserve ordering where needed. The appropriate mapping depends on the application and hardware configuration.
Reducing data-transfer and consistency work
Infer the packet data that must cross devices
When a packet moves between CPU and NIC processing, transferring an entire packet may be unnecessary. Yet identifying and assembling only the required fields can add work whenever an offloading design changes. FLOEM provides per-packet state: elements can refer to packet fields and metadata across the program without manually passing every value through each connection.
The compiler uses liveness analysis to determine which state fields are required beyond a queue. It then generates code to transfer the necessary data. This approach makes it easier to revise the placement of elements while avoiding the need to manually rebuild packet marshaling logic for each arrangement.
Use a cache construct for selected computation
FLOEM includes a caching construct for memoizing a program region, such as an expensive query. Developers identify the query’s input key and output value, then configure cache behavior, including write-through or write-back policies and how write misses are handled. The compiler expands this high-level construct into elements that implement the cache protocol.
Caching introduces correctness requirements as well as performance choices. In particular, the CPU and NIC must maintain consistent application state, and messages for operations such as misses and evictions may need to retain their order. FLOEM imposes requirements on how cache-related paths and queues are structured and reports errors when a program violates them.
Integrating with existing applications
Adopting a data-flow model does not require rewriting every part of an application. FLOEM allows developers to embed C code in elements, link to external C functions and expose selected segments as functions callable from an existing program. This supports a hybrid design: packet handling or other suitable work can use FLOEM, while complex application logic remains in its original codebase.
The compiler generates separate C programs for the CPU and NIC and infers data transfers across queues. Its runtime manages communication over PCIe, including DMA synchronization and batching. In the prototype, asynchronous transfers and batched operations help overlap communication with computation. These mechanisms are part of the system’s implementation; they do not remove the need to evaluate the target hardware, workload and communication pattern.
What the evaluations show—and what they do not
The FLOEM researchers applied the system to a key-value store and a distributed real-time analytics application. They reported throughput improvements of 1.3–3.6 times for the key-value store and 75–96% for the analytics system compared with CPU-only implementations. Results depended on the application and offloading design.
The key-value store case also illustrates why measurement matters: an initial CPU-NIC split performed worse than the CPU-only version because a blocking DMA read introduced an additional cost. A redesigned approach using NIC caching performed better under configurations with high cache hit rates. In the analytics case, an initial split likewise reduced throughput slightly; a subsequent redesign improved the result. Offloading decisions should therefore be validated with representative workloads rather than assumed to be beneficial based on placement alone.
Practical implications for engineering teams
FLOEM’s central contribution is a structured way to explore CPU-NIC designs while reducing repetitive implementation work. Its abstractions address several recurring tasks:
- Mapping packet-processing components to CPU and NIC resources.
- Controlling how logical communication paths use physical queues.
- Transferring only packet fields and metadata required downstream.
- Adding NIC-side caching with an explicit consistency protocol.
- Integrating selected offloaded components with existing C applications.
For industrial automation and infrastructure teams assessing network-intensive applications, the broader lesson is to treat offloading as a system-design decision, not a simple code-migration exercise. Define the data path, identify communication and ordering requirements, preserve application integration where practical, and measure the result on the intended workload and hardware. That disciplined approach helps determine whether a programmable NIC is a useful resource—and which parts of an application are suitable to place there.