Acceptance Evidence Has Missing Raw Data: Decide What Can Still Be Reconstructed
Identify which conclusion depends on the missing raw data, then classify what the surviving evidence can still support. A retained report may support a reported historical verdict without allowing independent recalculation. A new test can provide new evidence, but it cannot recreate observations that were never retained.
This article proposes a recovery review for engineering evidence. It does not alter a contract, fabricate missing measurements or certify a product. The numeric examples are synthetic, and no archive search, recalculation program or target test was executed.
1. Preserve what remains before attempting reconstruction
Identify the surviving report, chart, log fragment, procedure, requirement revision and product configuration. Keep original files unchanged and record any uncertainty about their provenance.
Search approved repositories, instrument exports and authorized owner records for linked originals. Do not assume a missing attachment was never created, but do not claim recovery until the actual file and its relationship to the run are established.
NASA’s verification-record guidance, section 5.3.1.2.4, links results to data and configuration versions. Those dependencies provide a useful starting point for determining which parts of a historical conclusion remain reviewable.
2. Draw the evidence dependency for each claim
For a timing verdict, dependencies might include the event definition, timestamp observations, time-base information, exclusions, calculation method and criterion. For a functional verdict, they may include stimulus, expected output, actual output and tested build.
Mark every dependency present, recoverable but not yet obtained, absent or uncertain. Avoid giving an entire report one completeness label when some requirements have full evidence and others retain only a summary.
| Surviving material | Potentially supported use | Important limit |
|---|---|---|
| Identified report with method and verdict | State what that report concluded | Not independent recalculation of missing observations |
| Raw observations without interpretation metadata | Preserve bytes and pursue missing definitions | Do not guess units, channels or event meaning |
| Complete observations and method | Recompute the defined result | Still assess provenance and method validity |
| Chart or isolated PASS label | Describe the limited retained statement | May not support configuration or criterion applicability |
3. Do not invent detail from a summary
Consider two fictional four-value datasets: 20, 40, 60, 80 and 0, 60, 60, 80. Both have a mean of 50 and a maximum of 80, but the number of values above 55 differs: two versus three.
Therefore, a retained count, mean and maximum cannot uniquely recover the distribution or that threshold count. These numbers are an illustrative arithmetic argument, not actual measurement data or an executed reconstruction.
The same principle applies to charts. Digitizing visible points may produce an approximate derived representation, but it does not recreate hidden samples, discarded outliers or precision lost during rendering. Label such work according to what it actually preserves.
4. Separate three evidence levels
Use “report-supported” when a conclusion is based on an identified surviving report and its stated scope. Use “independently recomputed” only when the necessary observations and method are available and the calculation was actually performed. Use “newly tested” for a later experiment with its own identity.
A trustworthy maximum in a sufficiently identified report may support the report’s bounded maximum claim. It does not automatically support a new percentile, an unrecorded operating condition or a fresh uncertainty analysis.
The NIST uncertainty discussion associates uncertainty with the result and measurement configuration. If that context is missing, do not supply a confident uncertainty statement merely because a summary contains many decimal places.
5. Choose recovery, limited reliance or repeat testing
| Evidence state | Decision path | Record to retain |
|---|---|---|
| Needed originals located and linked | Validate identity, then assess/recompute as appropriate | Recovery source and actual analysis result |
| Summary adequate for a bounded existing claim | Seek authorized reliance at that evidence level | Report scope and inability to recompute |
| Missing data essential to the claim | Hold the claim or plan a suitable new test | Dependency gap and required observation |
| Historical state cannot be recreated | Retain an explicit historical uncertainty | What a new test would and would not answer |
The responsible authority decides whether limited reliance is acceptable under the project process. This article does not turn a surviving summary into automatic acceptance or insist that every lost raw file invalidates every historical conclusion.
6. Design a new test around the unresolved question
When a repeat is needed, identify whether the original hardware, firmware, environment and workload can be recreated. If not, state the differences and the narrower or different question the new test will answer.
A current release passing a current bench test does not prove that an earlier release passed on an earlier installation. Retain the two evidence records separately and avoid backdating the new result.
Capture the observations and metadata that were missing before. The repeat plan should repair the specific evidence gap rather than merely produce another PASS screenshot.
7. Close with a claim-level reconstruction record
For each affected requirement, list the surviving artifacts, missing dependencies, recovery attempts, current evidence level and remaining action. Record derived analyses as new artifacts linked to their inputs.
Keep unavailable data visibly unavailable. A precise statement that one claim remains unproven is more useful than a reconstructed-looking spreadsheet filled with plausible values.
The outcome is a defensible account of what can still be known: which historical report can be relied on within scope, which calculation can be reproduced and which question needs new evidence. It preserves useful results without inventing the past.
