Edge-Triggered epoll: Drain Device-Service I/O Without Starving Other Connections
An edge-triggered epoll loop must remember readiness until nonblocking I/O establishes that no more progress is currently possible. For fairness, it may stop after a bounded amount of work, but it must keep that source on its own ready list. Stopping early and then sleeping for another edge can strand unread input; draining one busy connection forever can starve every other connection.
This guide uses Linux man-pages 6.12 and focuses on a single-owner dispatcher with nonblocking stream sockets. Device files need their driver’s read, write and poll contract checked separately. A small executed host fixture demonstrates the partial-drain problem; it is not a target throughput benchmark or proof of a finished scheduler.
Track application readiness independently
The epoll guide describes the kernel’s interest and ready lists, edge-triggered behavior and starvation risk. A returned event is a prompt to attempt I/O. It is not a reservation of bytes, one event per message, or evidence that a complete application frame is available.
Give each connection an application state containing generation, live/closed flag, input parsing state, pending output and readiness flags. Maintain a queue of connections eligible for another nonblocking service slice. An epoll event marks a direction ready and enqueues the connection only if it is not already queued. Repeated notifications must not create duplicate unbounded queue entries.
After receiving some bytes, retain them in the connection’s parser until complete frames can be decoded. One read can contain half a frame, several frames or the end of one followed by part of another. Transport readiness and protocol completeness require separate counters and limits.
Classify each nonblocking operation
| Read result on a stream | State transition |
|---|---|
| Positive byte count | Advance parser input by exactly that count; continue within the service budget. |
| Minus one, EAGAIN or EWOULDBLOCK | Clear input-ready state for now and await future readiness. |
| Zero | Handle stream EOF, including buffered complete and incomplete messages. |
| Minus one, EINTR | Apply the interruption/control policy without losing the partial-buffer position. |
| Another error | Classify and retire or recover the connection under its explicit policy. |
Use the corresponding write result to advance the output offset. A partial write does not authorize dropping the unsent suffix. On EAGAIN retain that suffix and arrange writable-interest handling. When the queue is empty, remove unnecessary writable interest under the chosen design; otherwise a permanently writable socket can cause useless dispatch work.
The conservative drain rule is to continue until EAGAIN, EOF or a terminal error. The documentation also describes limited stream-specific short-I/O reasoning, but do not transfer that shortcut to datagram, terminal or arbitrary device interfaces. Here the uniform EAGAIN rule avoids making a short read stand in for a capability the dispatcher has not established.
Bound service without forgetting work
Choose a byte, operation or elapsed-work budget per connection per pass. These are product scheduling choices, not epoll guarantees. If a budget ends after successful I/O but before EAGAIN, leave the connection marked ready and place it at the back of the userspace queue. Process another ready connection, then return to it without requiring a new kernel edge.
While that queue is nonempty, do not enter an indefinitely blocking epoll_wait. Poll for newly ready sources without blocking at a chosen cadence and merge them with retained work. Otherwise a busy retained source could prevent newly arriving control events from being considered even though the local queue itself is fair.
State the invariants in reviewable terms: no known-ready connection disappears before an exhausted/terminal observation; each live queue entry owns a valid generation; no connection receives unlimited consecutive service; and blocking waits occur only when the application has no retained runnable I/O. Queue limits and admission control still matter when offered traffic exceeds capacity.
Keep one-shot rearming and closure explicit
EPOLLONESHOT is a separate choice from EPOLLET. When used, it disables further notification after delivery until explicitly rearmed with EPOLL_CTL_MOD. The epoll_ctl contract must be part of the ownership design. Finishing one worker callback without rearming is not equivalent to finishing an edge-triggered drain.
A single-owner dispatcher simplifies closure, but it still needs to invalidate events already copied into a batch. If handling one event closes another connection, a later batch entry can be stale. Retain a safe object/generation reference until the batch cannot access it, mark it closed, remove registration as required, and avoid interpreting a reused descriptor number as the old connection.
Also audit duplicated descriptors. Closing one descriptor does not necessarily destroy the underlying open file description or remove all epoll interactions if duplicates remain. Do not rely on numeric closure alone as proof that no event can refer to the retired object.
Reproduce the missing-edge condition safely
An executed Python fixture on x86-64 Linux 6.18.44 used one owned Unix stream socket pair. It sent six bytes, received an EPOLLET input event, and read only two bytes. A second immediate epoll poll returned no events even though four bytes remained. Reading those four bytes succeeded; one additional nonblocking read then returned EAGAIN.
That observation demonstrates why the application must retain readiness after a budget-limited partial drain. It does not imply that every partial read always suppresses every subsequent event. Other arrivals and races can produce notifications, but relying on them would make correctness depend on unrelated traffic.
For target qualification, add two busy connections and one control channel. Track maximum consecutive service slices, backlog, parser progress, control handling and terminal cleanup. Inject partial writes, peer EOF with buffered data, duplicate readiness and closure during a returned event batch. These workload and fairness tests are proposed, not executed here.
Release the loop when every byte has a defined buffer position, every ready source has a future service opportunity, and every sleep is justified by exhausted local work. If those invariants are difficult to maintain, level-triggered epoll may be a better engineering choice than an edge-triggered loop whose correctness depends on new traffic arriving.
