Flash Ring Logs at Capacity: Define Which Evidence May Be Overwritten
A flash ring log needs an explicit erase-eligibility rule before it wraps: a sector may be reclaimed only when every record it contains is eligible for removal. If an incident pins even one record in that sector, ordinary rotation must stop or use a separately qualified retention path. Finite storage cannot guarantee unlimited recording and unlimited preservation simultaneously.
This article develops an application-level incident-retention model using Zephyr v4.2.0 Flash Circular Buffer (FCB) as a concrete storage boundary. The pin states are an original product design, not built-in FCB features. The proposed tests were not run on a target.
1. Put the policy at the actual erase unit
The v4.2.0 FCB description leaves the full-buffer decision to the caller: rotate old data or stop accepting new entries. Reclamation occurs by sector. A record-level policy therefore has to account for all other records sharing that sector.
The FCB API contract specifies that fcb_rotate() erases the oldest sector. It does not accept an arbitrary sector selected by an incident-retention manager. If the oldest sector is protected, calling rotate does not skip to a newer unprotected sector.
Build a sector ledger with physical sector identity, ring generation, contained record identities, pin references, and current eligibility. Never treat a reused sector address as the same historical object. A delayed release for an earlier generation must not unlock new evidence that now occupies that address.
2. Define the record and sector state model
| Application state | Allowed transition | Reclamation rule |
|---|---|---|
| Ordinary retained record. | Pin for an incident, or become eligible under ordinary retention policy. | May contribute to an eligible sector only after its policy permits removal. |
| Pin requested. | Persist and verify the pin decision, or reject the request. | Prevent conflicting rotation while the decision is pending. |
| Pinned record. | Release only through the approved incident disposition. | Its entire containing sector is protected. |
| Release requested. | Verify incident, record and generation identity; persist the release. | A request or transmitted export alone does not release evidence. |
| Unknown or inconsistent retention metadata. | Recover or explicitly resolve the uncertainty. | Do not silently classify the sector as unpinned. |
Permit multiple incidents to reference the same record. Releasing one incident must not remove another incident’s pin. A reference set or another qualified ownership model is safer than a single Boolean that loses the reason for protection.
Coordinate append, pin and rotation decisions under one application ownership mechanism. The v4.2.0 rotate implementation has its own mutex for FCB internals and erases the oldest sector. It does not inspect your incident ledger. An application pin check outside the relevant ownership boundary can race with a rotation decision.
3. Make the promise start at a durable boundary
Define when the product may display “incident preserved.” The evidence records must exist, and their protection metadata must survive the failures covered by the product’s requirement. A pin stored only in RAM may disappear at reboot while the log itself remains readable.
Design the ledger’s crash behavior explicitly. It may use a separate qualified store or an atomic relationship supported by the chosen storage design. Do not assume that writing a pin record somewhere in the same ring protects that record from rotation or makes a multi-object update atomic.
On recovery, reconstruct or validate pin ownership before enabling reclamation. If the ledger is incomplete, choose the documented conservative behavior: pause rotation, limit new admissions, and expose the unresolved state. The availability cost is real and belongs in the product decision.
4. Handle oldest-sector blocking and saturation
A protected oldest sector can prevent normal FCB rotation even when some newer sectors contain only disposable records. Distinguish this ordering constraint from the simpler condition “all bytes are pinned.” A free-space dashboard alone does not explain which reclamation operation is legal.
- Oldest sector eligible: rotate only after the eligibility decision and storage-operation result are both checked.
- Oldest sector protected: stop ordinary rotation. Reject, backpressure, or divert new records according to a predeclared policy.
- All permitted retention capacity occupied: report saturation and the refused admission. Do not silently evict a pinned incident.
- Separate incident store available: copy only through a qualified transfer that verifies completeness and establishes destination ownership before releasing the source.
- Metadata capacity exhausted: reject new preservation promises rather than creating pins that cannot be recovered.
Keep a bounded way to report saturation without relying on an unlimited sequence of new records in the full ring. Reserve the necessary control capacity and qualify it; an emergency reserve that shares the same blocked allocation path may not provide the escape route its name implies.
5. Specify what releases a pin
Choose a concrete disposition: an authorized operator decision, expiry under an approved retention rule, or verified transfer to a destination that has accepted responsibility. An upload starting, a socket write succeeding, or an export file being created is not automatically that disposition.
Bind release evidence to incident ID, record range, segment generation and content identity. Reject stale, duplicate or conflicting release requests according to a bounded idempotent policy. A delayed confirmation must never release whatever happens to be stored at the old physical offsets now.
Retention also needs privacy and access limits. Preserve only the incident data the product is authorized to retain, restrict export, and define the approved disposition. Indefinite pinning is not a substitute for a considered retention policy.
6. Qualify with a pinned-sector fixture
Use a disposable ring filled with synthetic numbered records. Put a protected record and ordinary records in the same oldest sector. Fill the remaining usable space, then request another append. The expected result is the declared saturation/admission outcome, with the protected bytes unchanged.
Repeat with two incidents pinning one record; release only one and verify that protection remains. Restart after pin persistence, during release, and before rotation. Replay a release from an old generation after sector reuse. Corrupt or omit ledger metadata in a copy and verify that recovery does not infer permission to erase.
Record sector identities before and after, exact record sets, pin references, rejected admissions and operation results. These tests evaluate erase authorization, not the torn-record parser or the transmission protocol.
Hypothetical shared-sector surprise
A device pins one small fault record in its oldest sector. Most bytes in that sector are routine telemetry, but the sector cannot be erased without losing the fault record. The device reaches its declared admission limit sooner than a record-count estimate suggested.
The correct response is the chosen bounded saturation policy, or a separately qualified incident-copy design. Release the feature with a sector-level eligibility ledger and restart tests, so “preserved” remains true precisely when the ring reaches its hardest capacity boundary.
