inotify Queue Overflow: Rescan Configuration State Instead of Trusting Missing Events

After IN_Q_OVERFLOW, treat the affected inotify instance’s incremental view as incomplete and rebuild it from authoritative filesystem state. Continuing to apply later events cannot reconstruct the missing changes. A configuration service should enter a visible reconciliation state, repair its watches and validate a complete candidate configuration before publishing a new in-memory generation.

This guide follows Linux man-pages 6.12 for local filesystems monitored through inotify. It concerns observation and reload, not durable multi-file commit or power-loss recovery. The rescan protocol is an application design requiring an explicit writer consistency rule. No inotify overflow was induced and no filesystem watcher fixture was executed here.

Define what the cache claims to represent

List the watched directories, relevant filenames, validated configuration fields and current applied generation. Separate “an event occurred” from “the new configuration was accepted.” A malformed replacement should not overwrite the last known usable in-memory configuration merely because its close or rename was observed.

The inotify reference warns that identical pending events can coalesce. Therefore event counts cannot reliably count writes or edits. Treat notifications as reasons to inspect current state, not as a complete audit journal. By the time an event is processed, the named object may have moved or disappeared.

For a service consuming a single atomically replaced file, monitor the parent directory and resolve the current name under a safe open policy. A watch on the old file’s inode does not automatically become a watch on a replacement inode. Keep the pathname-selection rule distinct from the opened file’s identity.

Recognize overflow before looking up a watch

IN_Q_OVERFLOW is reported with watch descriptor minus one. Parse global event conditions before indexing a per-watch table. A decoder that rejects an unknown descriptor first can discard the very event that says the entire view needs repair.

The queue limit is selected from max_queued_events when an inotify instance is initialized. Events beyond that capacity are dropped and an overflow event is generated. Increasing a limit is not this article’s recovery procedure: it cannot recover already missing events and does not prove the consumer will keep up with every future burst.

On overflow, increment a local observation epoch, mark derived state dirty and cancel the authority of pending incremental reload callbacks. Keep the previously validated application configuration available only under the product’s explicit stale-configuration policy. Expose “reconciliation pending” instead of reporting a clean watcher merely because the descriptor remains readable.

Rebuild a coherent observation generation

1. Stop treating queued deltas as a complete account of changes. Retire any pending rename-pair assumptions from the old observation epoch.
2. Create a fresh watcher instance or rebuild its watch/cache state under one serialized owner. Check every watch-add result.
3. Install required directory watches before scanning their contents, then enumerate the relevant objects and validate candidate data.
4. Continue collecting events during the scan. Reconcile changes observed in that interval or mark the candidate dirty for another pass.
5. Use the writer’s consistency boundary to determine whether the candidate is one valid generation.
6. Publish the validated snapshot atomically to readers, with its new observation and configuration identities.

Adding watches before scanning narrows a common blind interval, but does not make a directory traversal an atomic snapshot. A writer may continue changing files throughout the scan. For strict consistency, use a cooperating writer’s immutable generation/selector, a shared locking protocol, or another verified snapshot mechanism. Without one, promise eventual reconciliation under stated quiescence assumptions, not instantaneous consistency.

Bound retries and work per pass. If changes never settle or validation repeatedly fails, keep a defined degraded state and report the cause. An endless rescan loop can consume CPU while the service appears healthy. Preserve enough diagnostics to distinguish watch-registration failure, read failure, invalid content and renewed overflow.

Handle rename and directory replacement as state changes

IN_MOVED_FROM and IN_MOVED_TO can share a cookie, but the reference warns that they need not be adjacent or inserted atomically. The second half may arrive in another read, and a move outside the watched scope may have no matching destination event. A cookie helps associate observations; it is not a guaranteed complete transaction record.

Use bounded pending-pair storage and a fallback that rechecks authoritative names when a pair remains unresolved. Do not wait forever for a second half before considering later configuration work. After overflow, discard the assumption that retained pair fragments can reconstruct the lost interval.

Directory watching is not recursive. When a directory is created or moved into a monitored tree, install its watch and scan its contents because children may already exist before registration. Replacing or renaming directories also changes cached path mappings. Keep descriptor-to-object associations scoped to the watcher generation so stale events cannot be applied to a rebuilt table.

Document the events you cannot observe

Inotify does not provide the responsible user or process identity. It also does not generally report remote network-filesystem changes or modifications made through memory mappings. Mounting another filesystem over a watched directory creates additional visibility limitations described in the reference. A configuration update path using those mechanisms needs separate observation or periodic verification.

A successful scan cannot prove that no unobservable change will occur later. State the supported local writer operations and deployment mount layout. If the service relies on periodic reconciliation as a safety net, define its bounded work and freshness expectation rather than presenting it as an undocumented fallback.

Qualify the recovery protocol without changing host limits

First inject a synthetic overflow record into the application’s event-decoder test seam. Require dirty-state entry, old callback invalidation and a complete authoritative rescan. Inject overflow during the rebuild too, and verify that the candidate is not falsely published as complete. This tests application control flow, not kernel overflow generation.

In an approved disposable integration environment, test file replacement by rename, directory move-in with preexisting children, deletion during scan, invalid replacement content and a deliberately delayed event reader. Preserve the watcher generation, validation outcome and applied configuration identifier. Do not modify shared host inotify limits merely to obtain an overflow quickly.

The acceptance condition is convergence to a valid authoritative snapshot, with missing observation intervals visible and stale callbacks unable to overwrite it. Event delivery is an optimization for timely reload. Correctness after loss comes from reconciliation and the configuration consistency contract.