Yocto Downloads Directory: Preserve Complete Sources for an Offline Rebuild

A release image built successfully while the build server had internet access. Months later, the same checkout fails because an upstream repository is unavailable. Keeping a busy downloads directory is useful, but it is not sufficient evidence that the release can be rebuilt offline. The engineering question is whether the preserved inputs cover the exact targets and configurations being promised.

This procedure uses Yocto Project 5.0.12 and BitBake 2.8 documentation. It separates source availability from shared-state reuse and binary reproducibility. The checks below are proposed validation steps, not results from an executed build.

Define the rebuild boundary before copying files

Write down the image targets, machines, distribution configuration, layer revisions, and SDK targets that belong to the release. Include optional product variants that must remain supportable. A source snapshot prepared for one image does not establish completeness for another image with extra packages or different providers.

Preserve the metadata alongside the source archive. Downloaded source files cannot explain which patches, configuration fragments, or recipe options the release used. Local files stored inside a layer belong in the versioned layer snapshot; private development directories require separate treatment. Record the supported host environment and required host tools as another input set.

Assign a named owner to every input outside those sets. For example, a task that reads a generated file from a network share needs an explicit preservation decision. Finding such a dependency during an offline trial is valuable, even when it does not involve the normal source fetcher.

Populate an isolated download snapshot

The Yocto offline-build procedure documents a dedicated DL_DIR, source-control mirror tarballs, and a local source mirror. Use a new directory for the release snapshot instead of deleting a shared cache used by other builds. Set BB_GENERATE_MIRROR_TARBALLS = "1" while preparing it.

Fetch the complete target graph with bitbake product-image --runall=fetch, replacing the example target with the actual image. The BitBake command reference defines --runall as running the named task for recipes in that target's task graph. Repeat for the defined configuration matrix rather than treating one successful command as universal coverage.

Keep the fetch logs, resolved source revisions, and a file inventory with hashes. Do not prune source-control directories merely to reduce archive size until a separate restore trial has established that the retained representation works. Archive handling should preserve the directory structure expected by the fetcher.

Make the restore test unable to borrow missing inputs

Use a fresh build directory and a separate, initially empty working download cache. Point SOURCE_MIRROR_URL at the preserved local snapshot, inherit own-mirrors, and set BB_NO_NETWORK = "1". Keep the archived snapshot separate from the writable cache so the test does not silently modify the evidence being evaluated.

The BitBake fetcher documentation explains the role of local downloads and the premirror, upstream, and mirror search sequence. A warm working cache can hide omissions in the archive. The separate cache makes a missing source visible at the boundary being tested.

For a source-completeness trial, use bitbake product-image --no-setscene in the fresh build directory so shared-state restoration does not satisfy work that should exercise the preserved sources. Record any exceptions deliberately, rather than calling a mostly restored build a complete source rebuild. A successful fetch-only test is a useful first stage, but it does not cover tools that obtain inputs during later tasks.

The BB_NO_NETWORK definition concerns BitBake's fetcher. It is not an operating-system network sandbox for arbitrary task commands. If the acceptance requirement is no network access anywhere in the build, run the trial in an appropriately isolated test environment and retain that environment's network-control evidence as well.

Classify failures without weakening the requirement

Treat each failure as a missing input, an unstable reference, or an environmental dependency. A missing archive should lead back to the target configuration that required it. An authentication failure may indicate that source acquisition was never included in the preservation process. A task contacting an external service deserves inspection of the recipe and its upstream build logic.

Floating source revisions are a separate problem. The Yocto offline procedure warns that AUTOREV requires resolving the current upstream revision. Replace floating references with the reviewed immutable revisions for the release and preserve the resulting metadata. Do not substitute a different available commit simply because it makes an offline test pass.

After correcting an omission, repeat the trial with a fresh working cache. Reusing the cache from the failed attempt can retain manually supplied files and conceal the same gap on the next run. Keep the failed log and the corrective change together so the preservation process can be improved.

Accept a bounded, repeatable result

An acceptance record should identify the archive hashes, metadata revisions, configuration matrix, host description, command sequence, and resulting build status. Separate source acquisition success from complete image creation and from subsequent boot testing. These are different claims with different evidence.

Schedule a restore exercise when changing the preservation process or adding a new machine, package group, or SDK. The useful deliverable is an archive that a clean test can consume under the stated restrictions, with a clear account of what remains outside its scope. That is stronger evidence than the size of a downloads directory or the age of its oldest file.