Linux eventfd Doorbells: Distinguish Counter Draining from Semaphore-Style Reads
Choose eventfd read semantics from the information your consumer must retain. An ordinary eventfd read returns the accumulated counter and clears it. An EFD_SEMAPHORE read returns one and decrements the counter by one. Neither mode carries your work payload or automatically makes a shared queue safe; the queue and its ownership protocol remain authoritative.
This guide uses Linux man-pages 6.12. Nonblocking and close-on-exec flags date from Linux 2.6.27, and semaphore-style reads from 2.6.30. The scope is userspace eventfds used by an embedded service dispatcher. A bounded host fixture exercised counters and saturation on x86-64 Linux 6.18.44; no concurrent target queue or kernel-originated event source was tested.
Specify whether the counter represents work or a doorbell
For a counting contract, every successful increment contributes to a quantity that the consumer must interpret. For a doorbell contract, a nonzero counter means “inspect the authoritative work state.” Several writes may collapse into one read without losing queued work, provided the consumer drains that work under a correct synchronization protocol.
Do not switch between these meanings implicitly. If the consumer reads an accumulated value of five and processes only one item while assuming another notification must follow, four items may remain stranded. Conversely, processing five queue entries solely because the counter was five is invalid if writes announce conditions rather than individual accepted entries.
Write down the publication boundary: when a producer commits an item to the queue, when it attempts the eventfd write, what a failed write means, and when the consumer may remove the item. A mutex or a reviewed atomic queue algorithm must establish payload visibility and ownership. Do not claim the syscall alone proves an arbitrary lock-free memory-order design.
Read the two modes literally
The eventfd interface maintains an unsigned 64-bit counter. Successful reads return an eight-byte native-order value. With ordinary semantics, a positive count is returned and reset to zero; with EFD_SEMAPHORE, each successful read yields one while reducing the count by one. At zero, a nonblocking read returns EAGAIN.
| Synthetic sequence | Ordinary mode | Semaphore mode |
|---|---|---|
| Start at zero; write 2, then write 3 | Counter becomes 5 | Counter becomes 5 |
| First read | Returns 5; counter becomes 0 | Returns 1; counter becomes 4 |
| Further nonblocking reads | EAGAIN until another increment | Four more ones, then EAGAIN |
Readiness is level-like with respect to the counter being positive, but an edge-triggered epoll registration still needs a correct drain/retained-readiness policy. A semaphore eventfd can remain positive after one read. Returning directly to a blocking edge wait at that point is not a reliable way to obtain the remaining tokens.
Handle writes as fallible admissions
A write adds its supplied eight-byte value. The ordinary userspace maximum counter is UINT64_MAX minus one. If the addition would exceed that limit, a nonblocking write returns EAGAIN; a blocking write can wait for a reader to make space. Writing UINT64_MAX is invalid. Use checked arithmetic in any surrounding application counters as well.
For strict counting, a failed write has not admitted that increment. Retain it under a bounded retry or explicit failure policy. Never count the attempted increment as successfully delivered. If the payload was already committed elsewhere, the design must explain how it remains discoverable despite notification failure.
For a deliberately coalesced doorbell, saturation means a notification is already outstanding, but treating EAGAIN as harmless still requires a proved queue/recheck protocol and valid descriptor ownership. It is not a blanket rule for all errors. EBADF, an invalid write size or a shutdown race cannot be relabeled as successful coalescing.
The manual also describes a theoretical overflow indication from kernel AIO signal posts. That differs from userspace writes, which cannot wrap the counter past the checked maximum. Do not report an ordinary saturated write as evidence that an overflow actually occurred.
Close the clear-and-sleep race
A simple doorbell consumer can drain the eventfd, then inspect and service the protected queue. If work remains after a budget slice, it retains local readiness rather than sleeping. Before sleeping, it needs a protocol that makes a concurrent producer’s publication either visible in the final queue check or accompanied by a notification that remains observable.
The exact proof depends on the queue and locking design. Document the interleavings around empty observation, notification clearing, publication and wait entry. Avoid a custom boolean “already notified” flag whose clearing can erase a producer’s new notification unless its synchronization is explicitly established.
When multiple consumers share an eventfd, ordinary reads can consume the whole count and semaphore reads can distribute individual tokens. Neither guarantees fair scheduling among readers or gives each consumer a copy of every event. Choose a single dispatcher when the work ownership does not require multiple competing consumers.
Audit inherited references and shutdown
Forked or duplicated descriptors refer to the same eventfd object. One process reading can change what another later sees. Close-on-exec prevents an unintended executed helper from retaining a reference, but a fork-only child still needs its own ownership decision. Closing one descriptor does not destroy the object while other references remain.
Stop new producers before releasing the dispatcher’s final reference, and resolve queued work under the shutdown policy. A producer that writes through a stale numeric descriptor after reuse can affect a different object. Keep descriptor lifetime protected alongside the queue lifetime.
Evidence and acceptance tests
The executed host fixture wrote two and three to an ordinary eventfd, read five and then observed EAGAIN. A separate semaphore eventfd initialized to three returned one three times before EAGAIN. Another ordinary counter was set to UINT64_MAX minus one; a nonblocking write of one failed with EAGAIN, and the subsequent read returned the unchanged maximum permitted value.
These checks establish the observed counter semantics, not a producer-consumer algorithm. Proposed target tests add publication before wait, publication during drain, budget exhaustion, multiple producers and shutdown with outstanding work. Require accepted items to equal processed plus still-owned items under a consistent snapshot. The useful result is a bounded work ledger and a sound wake protocol, rather than equal counts of writes, wakeups and callbacks.
