Yocto Shallow Git Mirrors: Explain a Full Download Despite BB_GIT_SHALLOW

Setting BB_GIT_SHALLOW to 1 does not promise that a clean builder will transfer only one commit from every upstream Git server. In the examined BitBake release, the feature centers on shallow mirror tarballs. A missing or mismatched shallow artifact can lead to a full mirror or an upstream clone.

To explain a large download, identify which representation the fetcher selected. The size of the final source checkout alone cannot tell you how much history was transferred or which cache supplied it.

Separate the producer setting from consumer behavior

The BB_GIT_SHALLOW reference documents a shallow-mirror attempt followed by a full-mirror fallback. Without a suitable mirror tarball, the implementation can perform a full Git clone even with shallow support enabled.

Generation is another choice. BB_GIT_SHALLOW = "1" together with BB_GENERATE_SHALLOW_TARBALLS = "1" requests shallow mirror production. BB_GIT_SHALLOW_DEPTH controls how much parent history those generated tarballs retain; a depth of one keeps the referenced commit without its parents. Increasing that depth changes the artifact being requested.

Treat the producer and consumer as separate configurations. A mirror populated by a builder that never generated shallow artifacts cannot deliver them merely because a later consumer enables shallow mode. Likewise, a file produced for an older revision or different depth is not automatically the artifact requested by the current recipe.

Check the computed artifact identity

The examined Git fetcher constructs shallow filenames using repository identity, selected revisions, depth and applicable refs. Inspect the actual requested filename in fetch debugging rather than inventing a generic name for the server.

Compare that request with the mirror's inventory and routing. A successful HTTP response for another tarball does not establish a match. Look for a missing file, a rewrite that drops part of the name, or a producer that retained a different set of refs.

There is also a local-cache branch: an up-to-date clone can be preferred over tarballs. During unpacking, the examined implementation uses that usable clone first and tries the shallow tarball if the clone cannot supply the requested source. A warm builder can therefore behave differently from a new consumer with the same shallow settings.

Distinguish a full producer clone from a failed optimization

A producer may need a full repository to generate the shallow representation that consumers subsequently download. Seeing a full clone during mirror preparation is not, by itself, evidence that shallow generation failed.

Inspect the producer's resulting shallow artifact and its generation log. Then test consumption from a separate working download directory. Otherwise, the producer's retained full clone can satisfy the consumer experiment and obscure whether the published shallow artifact was usable.

Use the real recipe revision and depth in this experiment. Do not change SRCREV to a convenient branch tip merely to obtain a smaller transfer. That would test a different source input and could make an unavailable release commit appear to be a performance problem.

Check history-dependent build logic explicitly

After unpacking, git rev-parse --is-shallow-repository reports the repository's shallow status, as documented by Git's revision-query reference. Run it against the actual source checkout, not whichever repository happens to be the terminal's current directory.

A successful checkout proves that the selected tree was obtainable. It does not prove that a build-time version script has all the ancestry or tags it expects. Inspect any such script and exercise its exact query against the shallow tree. Decide whether to retain additional history or change the versioning input deliberately.

The useful outcome is a clear account of the selected source representation: current local clone, matching shallow tarball, full mirror or upstream clone. Once that branch is known, fix the missing artifact or configuration mismatch. Repeatedly reducing the depth will not repair a mirror URL that never serves the requested shallow file.