Precompiled Python Bytecode: Match Interpreter, Invalidation Mode and Runtime Paths
Precompiled Python bytecode is usable only when the interpreter format, import location and cache-validation policy agree. A pyc file can be present yet ignored, stale or unsuitable for the deployed interpreter. Diagnose those conditions separately before treating bytecode generation as complete.
This article pins CPython 3.12.14. The executed fixtures used that same interpreter on x86-64 Linux, owned temporary source files and fresh child processes. They tested cache invalidation and source presence, not startup speed, another Python release or an embedded image.
Record the producer and consumer contract
Keep a manifest containing the producing interpreter version, implementation, bytecode magic, cache tag, optimization level, invalidation mode and intended import location. Record the consuming interpreter independently. A filename that resembles the expected cache tag is not a substitute for validating the bytecode format.
The tagged import implementation checks the pyc magic and header flags before interpreting the cache. The format is interpreter-specific; a file produced by an arbitrary host Python is not automatically an acceptable target artifact.
Also record whether source files remain in the image. Source-associated caches under __pycache__ and deliberately sourceless top-level pyc modules follow different discovery paths. Decide which packaging model is intended before deleting sources to save space.
Select an invalidation policy deliberately
The 3.12.14 py_compile documentation defines timestamp, checked-hash and unchecked-hash invalidation. Timestamp caches compare source modification time and size. Hash-based caches carry a source hash, with the checked bit expressing whether normal import should validate that source.
Specify the mode explicitly for a release build. SOURCE_DATE_EPOCH affects the default mode, while an explicit invalidation_mode takes precedence in this version. An environment difference between developer and release jobs should not silently change the cache contract.
Unchecked hashing delegates source consistency to another mechanism, typically the image or package builder. It does not provide content authentication. Checked hashing is also a freshness mechanism rather than a signature or trust boundary.
Read the executed stale-source matrix
Each host fixture first compiled a module containing VALUE=1. It then replaced the source with VALUE=2 while preserving the eight-byte source size and the same integer modification time. Every import ran in a fresh isolated CPython 3.12.14 child, avoiding an already-imported module in sys.modules.
| Cache mode | Hash policy at import | Observed VALUE |
|---|---|---|
| Timestamp | default | 1: unchanged time and size accepted the stale cache. |
| Checked hash | default | 2: source mismatch caused source recompilation. |
| Unchecked hash | default | 1: the cached code remained selected. |
| Checked hash | never | 1: runtime policy skipped the hash check. |
| Unchecked hash | always | 2: runtime policy required source checking. |
The command-line policy was selected with –check-hash-based-pycs. These observations match the checked-bit and policy branches in the pinned loader. They show why the producer’s invalidation mode and the consumer’s command line belong in the same manifest.
The child processes used -B, which prevented writing new bytecode caches. It did not prevent reading an existing cache or compiling changed source in memory. A read-only image can therefore execute updated source without producing a replacement pyc, depending on its loader and permissions.
Test source presence independently
A separate owned fixture compiled VALUE=7 into its normal __pycache__ location and removed only that fixture’s source. An ordinary module import then failed with ModuleNotFoundError. The cache file’s continued presence did not make it a discoverable sourceless module.
Copying the same owned bytecode into a top-level fixturemod.pyc enabled the sourceless import, which returned 7. This is a narrowly controlled result for the documented CPython filesystem importer, not a recommendation to move arbitrary caches without qualifying the packaging layout.
The versioned import reference explains source cache behavior. Custom importers, zip layouts and application launch mechanisms need their own checks. Do not generalize the two filesystem cases into a universal rule for every Python deployment.
Separate stored filenames from import search
The compiler’s dfile argument changes the filename recorded in the code object. The fixture set it to /opt/app/fixturemod.py and verified that value in the raw, self-produced bytecode. It did not create that path or direct module discovery there.
Furthermore, the source-based loader in 3.12.14 calls _imp._fix_co_filename with the actual source path when loading cached code. A stored dfile value alone therefore does not guarantee the filename later shown by every runtime traceback. Test the actual loading route if stable diagnostic paths matter.
Inspect only trusted, self-produced caches when using marshal-based inspection. This procedure does not require loading untrusted bytecode into the interpreter. Keep import-path qualification separate from any diagnostic-path normalization.
Use a decision procedure on the image
First verify the deployed interpreter identity and intended module path. Next compare the cache header and optimization choice with that interpreter. Then check whether source exists and which validation policy the process actually receives. Finally run a fresh-process import that reports the expected application value.
If a timestamp cache unexpectedly survives a source change, compare both recorded time and size. If a hash cache behaves unexpectedly, inspect the checked bit and runtime hash policy. If removing source breaks discovery, qualify the sourceless layout rather than merely changing a cache suffix.
Retain positive and stale-source controls with the exact image manifest. The seven host cases above establish specific CPython 3.12.14 behaviors; they do not establish compatibility with another magic value or performance on a target. The release outcome should state which producer, consumer, layout and invalidation policy were actually exercised.
