Test Firmware Versus Shipping Firmware: Recheck Features That the Diagnostic Image Bypasses
Use diagnostic firmware for the hardware paths it genuinely exercises, then map its differences from the shipping image to the final verification still required. A direct peripheral test can establish useful assembly evidence while bypassing the scheduler, driver configuration, security policy or application logic that the customer will use.
This is an original configuration-delta procedure informed by NASA SP-2016-6105 Rev2. All firmware configurations and examples are hypothetical. No image was built, programmed or booted, and no claim is made that diagnostic firmware is inherently unreliable or that every test must be repeated unchanged.
Identify the two complete configurations
Record the diagnostic and shipping artifact identities, source revisions, build options, board configuration, boot chain and persistent-data assumptions. Include peripheral firmware and external assets when they affect the tested function.
Do not compare only two application filenames. A diagnostic image may run under a different bootloader, use another clock tree or depend on a debugger. The effective test configuration includes those conditions as well as the bytes in the main application.
The NASA handbook’s verification guidance records test-article configuration and differences from the intended configuration. Apply that principle by naming each difference and the claim it can affect, rather than treating “same board” as complete equivalence.
Map diagnostic shortcuts to shipping paths
| Hypothetical diagnostic behavior | Shipping behavior it may bypass | Final evidence to define |
|---|---|---|
| Writes a GPIO directly from a simple loop. | Application command, authorization and scheduled output path. | Required command produces the intended output through the released stack. |
| Uses a fixed peripheral clock. | Dynamic clock or low-power transitions. | Relevant operation after supported transitions. |
| Reads raw ADC values with a test coefficient. | Production calibration selection and scaling. | Correct unit-bound coefficients and final interpreted result. |
| Runs one peripheral at a time. | Concurrent application traffic and shared-resource behavior. | Required coexistence case in the shipping configuration. |
| Accepts unrestricted service commands. | Released access controls and disabled test hooks. | Permitted access works and disallowed service entry is unavailable as specified. |
The matrix does not assert that these differences exist in every product. Inspect the actual builds and paths. A diagnostic image can share the same driver implementation as shipping firmware, but that fact alone does not prove identical initialization or operating conditions.
Credit the evidence that remains applicable
For each diagnostic result, identify the physical function and conditions it establishes. A qualified continuity or loaded-output observation may remain useful after replacing firmware if the relevant hardware and interpretation assumptions stay valid.
Write the reuse argument. State which components, routing, stimulus and measurement method are unchanged, and which shipping behavior remains uncovered. Do not throw away valid assembly evidence merely because the image name changes.
Conversely, do not carry a broad pass forward when the diagnostic image used a different path. If production software never enables a peripheral correctly, a direct-register diagnostic pass cannot establish that the shipped feature works.
Control calibration and persistent-state handover
Identify every persistent value the diagnostic image creates or modifies: calibration, serial association, test flags, counters, boot selection or temporary configuration. Specify which values must survive and which must be removed before final verification.
Bind calibration to the physical unit, measurement method and schema expected by the shipping image. A valid coefficient under the diagnostic algorithm can be misinterpreted by a different production scale, unit convention or layout.
Test the handover on recoverable development hardware with preserved original data. Include missing, stale and incompatible records using an approved method. Do not erase production calibration as a routine way to obtain a clean test state.
Keep the handover result distinct from the subsequent functional result. Successfully copying a calibration record does not prove that the shipping image selected or applied it.
Verify the final image through the actual boot path
After replacing the diagnostic image, establish the final installed configuration and the image that actually starts. A programmer result alone cannot prove that a bootloader did not select an older slot or recovery image.
Use the required reset or power condition and bind the observed startup to the current unit and boot session. Remove or account for debugger effects where the shipping operating mode excludes them.
Record the final build identity and relevant persistent configuration after the replacement. A production label printed before the shipping image is installed should not become the only evidence that installation completed.
Select final checks from the delta
Choose at least one requirement-specific check for every material bypass or changed path that needs per-unit evidence. Some wider configuration differences belong to engineering qualification rather than every-unit screening; that allocation requires an approved rationale.
For a hypothetical controller, the diagnostic image directly drives an output and measures its loaded voltage. The shipping application receives a command, validates it, schedules the update and drives the same output through a configured driver. The diagnostic observation can support the physical output path, while the final check must exercise the released command-to-output chain.
If the shipping configuration also changes pin multiplexing or inversion, the physical-path reuse argument needs revision. The important question is the actual delta, not whether both builds call a function with the same name.
Include absence checks for test-only behavior
Review diagnostic commands, forced credentials, bypass flags, alternate boot entries and test data that must not remain in the shipping state. Verify their expected absence through the product’s supported inspection or negative-test method.
Do not weaken access controls merely to make the final test convenient. If a shipping image needs a manufacturing test interface, design and qualify its authorized lifecycle explicitly. A hidden debug path is not automatically a safe substitute.
Preserve a separate result when final cleanup or final-image verification fails. The earlier diagnostic pass remains historical evidence, but the unit has not completed the intended handover.
Deliver a two-configuration verification record
The completed package should contain both artifact identities, the diagnostic-to-shipping delta matrix, evidence credited from the first stage, persistent-state handover, final boot identity and final checks. Mark untested differences explicitly.
This approach keeps diagnostic firmware useful without allowing its shortcuts to define the product’s acceptance accidentally. The release claim belongs to the final configuration, supported by the earlier evidence only where the reuse argument is justified.
