Zephyr FIFOs: Protect Intrusive Link Storage and Prevent Double Enqueue

With Zephyr’s ordinary k_fifo_put(), the queued object carries the queue’s link inside its own first machine word. That storage is not application payload while the object is queued. The same object must not be active in two FIFO queues, or added again before removal. Debug this class of failure as a layout and membership problem before blaming queue ordering or scheduler priority.

This guide targets upstream Zephyr 3.7.0 intrusive FIFO insertion, with a separate discussion of k_fifo_alloc_put(). It assumes records remain valid throughout transfer. The distinct checks here are reserved-link placement, actual instance alignment and single active membership. General allocator sizing, DMA coherency and multi-reader payload ownership require separate analysis.

Reserve a pointer-sized leading field

The 3.7.0 FIFO guide requires word-aligned data items and reserves their first word for queue linkage. Use a dedicated pointer field first in the record, rather than allowing a message identifier or sample value to occupy those bytes. Do not hard-code a four-byte reservation for every target: pointer size and alignment follow the target ABI.

This original C11 layout example was compiled successfully with GCC 14.2.0 on the host. It checks the declaration only; it neither calls Zephyr nor proves that a custom allocator returns correctly aligned instances.

struct record {
    void *fifo_reserved;
    uint32_t sequence;
    uint8_t payload[24];
};
_Static_assert(offsetof(struct record, fifo_reserved) == 0,
               "link must be first");
_Static_assert(_Alignof(struct record) >= _Alignof(void *),
               "link alignment");

The test translation unit included the standard headers for these types and operators. Repeat the checks with the product compiler, then inspect allocation alignment. A correctly declared record placed in a packed enclosing object, unaligned byte array or incorrectly stepped pool can still violate the instance requirement. Check each record address, not only the pool’s initial address.

Why a second put can damage existing work

The tagged FIFO macros route ordinary insertion through the kernel queue implementation. In queue insertion, the intrusive path initializes linkage in the supplied object and inserts that node. The API does not provide a new independent node for each ordinary put.

If that object already links to another queued record, reinitializing its link can alter the first queue’s structure. Adding it to a different FIFO is equally invalid: both queues would need exclusive use of the same leading word. The documented outcome is undefined behavior, so do not depend on a particular symptom such as a self-loop, one missing item or a predictable assertion.

Queue locking does not fix this misuse. A lock can serialize two invalid insertions while leaving the same object active twice. The application needs a membership rule covering all insertion sites, including interrupt producers, retry paths and diagnostic injection.

Instrument membership outside the link field

1. Inventory every ordinary put and every source of record addresses. Identify shared static records and retry paths that can reuse an address too early.
2. Track each record as producer-owned, published or consumer-owned in a separate debug table or dedicated application metadata. Never use the reserved link as a state flag.
3. Validate and change producer-owned to published before calling put. Synchronize this check-and-transition appropriately for all participating contexts.
4. After a successful get returns the item, the consumer changes the record to consumer-owned. Only the defined recycling step returns it to a producer.
5. Reject a put from any other state before invoking the kernel. Record the address, current state, intended FIFO and sequence identifier.

The ordering in step three is important. If a consumer is already waiting, insertion can hand the item directly to that thread, and it may run before the producer continues after put. A producer-side debug assignment performed only after the call could overwrite the consumer’s newer state. Do not inspect or modify the handed-off record merely because the put call has just returned.

Do not use light-load success as a layout test

A hypothetical telemetry producer accidentally places its sequence number in the first word. In a light-load test, a receiver is usually already waiting, so direct handoff may conceal the intrusive-storage error. Under a burst, records begin accumulating in the FIFO and their leading bytes become linkage. The changed symptom reflects a different queue path, not random payload corruption.

Qualify both paths deliberately. First block a consumer before publishing one record. Then keep consumers stopped while publishing several distinct records, verify their application fields after removal, and compare the sequence list with the insertion log. The second case exercises stored links rather than relying only on direct handoff.

Understand the allocated-wrapper alternative

k_fifo_alloc_put() provides temporary bookkeeping storage instead of requiring a reserved field inside the data. The data itself is not copied. The wrapper is freed when the item is removed, and insertion can fail with -ENOMEM if the caller’s resource pool cannot supply the needed storage. Prepare the publication state before this call too: direct handoff can let the consumer run before success returns. On failure, retain or restore producer ownership according to the wrapper protocol; on success, do not touch the record because the consumer may already have processed it.

This alternative removes the intrusive layout requirement; it does not make shared payload mutation or repeated ownership transfer safe. Do not switch APIs just to suppress a double-enqueue assertion without specifying who may read, update and release the underlying record. Also test allocation failure when no consumer is waiting, because the direct-handoff path can avoid allocating the wrapper.

No Zephyr FIFO runtime test was executed for this article. The host check validates only the illustrative C declaration. Target qualification should include delayed consumers, burst insertion, producer retry, deliberate duplicate detection in the application guard and orderly recycling. A passing design has both correctly aligned link storage and an auditable proof that each record is published once per ownership cycle.