A Wi-Fi Firmware Crash Without a Kernel Panic: Preserve the Right Diagnostic Layer

A radio firmware fault can interrupt Wi-Fi while Linux continues running. Preserve the driver’s fault report and any device coredump promptly, then correlate them with the specific radio incident. A normal host uptime and an empty panic archive do not establish that the radio firmware stayed healthy.

This procedure follows Linux 6.12 Intel iwlwifi MVM and its regular operating firmware. It does not assume identical recovery in the older DVM driver, other vendors, initialization firmware or suspend transitions. The installed device, firmware image, kernel configuration and transport state must be recorded before applying the source-level interpretation.

Identify the failed layer before restarting anything

Separate four observations: a firmware error reported by the driver; a bus or transport failure; a failed association; and an application timeout. They can occur in one incident, but none is a substitute for the others. An unreachable access point alone does not prove a firmware crash. A firmware error log does not prove that the host kernel panicked.

In the pinned MVM implementation, iwl_mvm_nic_error() conditionally emits the NIC error log, handles shutdown collection separately and checks whether firmware is running before requesting the restart path. That path aborts notification waits and scan work, while recovery decisions depend on initialization/restart state, device registration, transport health and firmware restart controls. “The driver has a recovery function” is therefore insufficient evidence that this incident recovered. Linux 6.12 MVM error and restart paths

Build an incident key from the host boot identity, actual radio device identity and an application-assigned incident sequence. Add monotonic event times, exact kernel/driver build and the firmware version reported for that device. A firmware restart within one host boot needs a new radio incident record, not an invented new host boot.

Understand why the next dump may be missing

The pinned iwlwifi firmware dump code exports collected scatter-gather data through dev_coredumpsg(). This provides a transient device diagnostic object, not a permanent crash archive. Whether useful data can be collected still depends on the fault, firmware configuration and transport accessibility. Linux 6.12 iwlwifi firmware dump collection

The device-coredump framework has an important retention boundary: when an entry already exists for the same failing device, the new dump is discarded. The implementation tests for an existing entry; it does not test whether an operator has already read its bytes. Reading the data attribute alone does not remove the entry or make room for the next incident. Writing that attribute schedules deletion. Keep those actions separate in the collection runbook. Linux 6.12 dev_coredumpm_timeout(), data read and write handlers

The ordinary wrapper used by this scatter-gather path selects a five-minute default timeout. A separate custom-timeout API exists, so this is not a universal promise for every driver. With CONFIG_DEV_COREDUMP disabled, the stub frees supplied dump data instead of exporting it. Global disabling, allocation failure and failed collection can also leave no artifact. Absence of a dump cannot clear the firmware of responsibility. Linux 6.12 default timeout and configuration guards

Use a preserve-first incident procedure

1. Record the first observable failure. Save the bounded driver/kernel log interval before a manual reset destroys context. Identify whether the evidence says firmware error, transport failure, association failure or only service timeout. Retain uncertainty where the log cannot distinguish them.
2. Identify the correct device object. Examine available device-coredump entries and their failing-device relationship. The framework attempts to create a failing_device link, but link creation can fail. Cross-check the device and driver logs instead of treating the devcd ordinal as an incident identifier.
3. Copy the available data promptly. Use an approved collector with restricted output access. Save collection time, byte count, digest, device identity and the matching incident key. Verify that the saved copy is complete according to the collector’s supported read/error handling before relying on it.
4. Preserve the distinction between copy and release. Do not automatically delete the live object just because a read was attempted. Only the authorized retention procedure should release it after the copy is verified. Document that leaving it present can prevent the next same-device dump.
5. Observe recovery independently. Record firmware restart messages, the new association outcome, address state and one uniquely identifiable application exchange. A restart announcement is not proof that the service recovered.
6. Escalate with a matched package. Include exact software/firmware builds, incident timing, device mapping and a sanitized log excerpt. Transfer raw dumps only through an approved support path with appropriate access.

Device dumps can contain packet contents, identifiers or other sensitive runtime data. Treat them as restricted evidence, not material to paste into a public issue. Sanitizing the log summary and controlling raw-dump distribution are different tasks; do not claim a binary dump is anonymous merely because its filename contains no serial number.

Interpret repeated failures without relabeling old evidence

Consider a hypothetical gateway whose host boot remains unchanged. Radio incident 41 creates a dump; a collector reads and verifies it but leaves the live entry present. Incident 42 occurs shortly afterward. The framework can discard its new dump because the old entry still exists. A later collection timestamp does not turn the retained bytes into an incident-42 dump.

The correct record is “incident 42 observed; only incident 41 dump retained,” accompanied by incident 42’s logs and recovery observations. If the incident mapping is uncertain, say so. This avoids a particularly costly diagnosis: asking the firmware team to explain a second fault using the first fault’s memory image.

Qualify the runbook without manufacturing a crash

The source paths above were inspected; no firmware fault, radio recovery, sysfs deletion or hardware dump collection was executed for this article. Validate the collector’s file handling with harmless fixtures first, then exercise the runbook during authorized target qualification or a naturally reproduced defect. Do not inject undocumented debug commands or repeatedly power-cycle a production unit to obtain a cleaner story.

Acceptance requires a bounded capture delay, a verified retained copy, an unambiguous device/incident mapping or an explicit uncertainty label, and a separate service-restoration result. Keep “host survived,” “radio restarted,” “dump preserved” and “application resumed” as four independently supported statements.