Large Files on 32-Bit Embedded Linux: Audit off_t Across Library Boundaries
Qualify large-file support at every boundary that carries a file offset or size, including library interfaces and intermediate arithmetic. On a 32-bit glibc build, selecting _FILE_OFFSET_BITS=64 can widen the relevant interfaces, but it cannot repair an already compiled narrow library or an application field that still truncates the value to int.
This guide uses Linux man-pages 6.12 for glibc feature-test behavior and file interfaces. It focuses on file positions and sizes around the signed 32-bit boundary, not calendar-time migration. An executed sparse-file fixture ran on a 64-bit x86-64 host; no 32-bit target executable was built or run.
Define the supported file horizon
Identify the largest supported log, recording, database, update artifact and exported file. Include temporary growth and sparse offsets, not only the current compressed image size. A file size of 2 GiB is 2,147,483,648 bytes, one greater than the maximum positive signed 32-bit value. A 32-bit process can still perform large-file I/O when its ABI and implementation support it.
The off_t definition identifies it as a signed integer type for file sizes. Its width is architecture and feature-policy dependent. Record sizeof(off_t), relevant structure layouts and the exact compiler/sysroot settings from the target build. Do not infer these properties solely from CPU word size or a successful host test.
Keep total file size separate from an individual I/O request length. A process need not map or allocate the whole file to access a high offset. Bounded chunks can operate on a large file while respecting address-space and per-call count limits.
Set the feature policy before every system header
The glibc feature-test documentation says _FILE_OFFSET_BITS=64 redirects relevant 32-bit file operations and types to their large-file counterparts. It must be established before system headers are included, preferably through a consistent project build policy. On conventional 64-bit systems it has no effect on the already-wide file-size capability.
Check every translation unit, static library, shared library, plugin and binding that exchanges affected types. A private API taking off_t by value or embedding it in a structure requires matching declarations and compilation assumptions on both sides. Recompiling the main executable does not rewrite the ABI of a binary-only dependency.
A link can succeed despite an application-defined wrapper having incompatible parameter widths. Exported symbol names do not generally encode your custom C structure layout. Add compile-time layout assertions and a boundary-value call across the actual library interface, rather than accepting successful linking as ABI evidence.
Follow the offset through arithmetic and formatting
Audit casts to int, long, unsigned long and size_t. A wide destination does not save an intermediate expression that overflowed in a narrower type. For example, construct a high test offset in the intended wide type before shifting or multiplying, then check addition against the supported range before combining base and length.
Keep signed-error conventions intact. An API returning a negative error sentinel cannot safely be stored first in an unsigned size field and then tested for negativity. Preserve native types and return codes until the operation has been classified.
Text logging can also conceal a defect. Use a formatting approach appropriate to the actual type, such as a checked conversion to intmax_t with its corresponding format, instead of assuming %ld always matches off_t. A truncated diagnostic offset can make the file operation look wrong even when the underlying access was correct.
Audit the library calls, not just lseek
| Boundary | Check |
|---|---|
| open/stat/fstat | Can the selected ABI describe the file and its metadata? |
| lseek/pread/pwrite | Does the offset remain wide through wrappers and arithmetic? |
| stdio positioning | Use the appropriate off_t-based fseeko/ftello contract rather than assuming long-based functions are wide. |
| Library or plugin API | Both components agree on parameter and structure layout. |
| Persistent index or protocol | Its explicitly defined representation preserves the supported offset range. |
fseeko and ftello use off_t where fseek and ftell use long. The stat documentation also records EOVERFLOW when size, inode number or block count cannot fit the selected result types. Do not translate every metadata failure into “file missing” or assume EOVERFLOW always concerns size alone.
For exported indexes, specify units, signedness, byte order and range explicitly. Writing a native structure containing off_t to disk turns a build policy into a file-format dependency. A versioned fixed-width representation is easier to qualify across device and desktop readers, provided all conversions are checked.
Use sparse fixtures to test positions cheaply
An executed C fixture defined _FILE_OFFSET_BITS=64 before headers, asserted an at-least-eight-byte off_t, and used an owned temporary file. It wrote one byte at offset 2,147,487,744, verified a logical file size of 2,147,487,745, read the byte back and checked fseeko/ftello at the same position. GCC 14.2.0 compiled it with warnings treated as errors.
The host reported eight-byte off_t and eight-byte long. The test therefore proves the host path used, not that the macro changed a 32-bit ABI. It created a sparse logical extent rather than writing gigabytes of payload, and closed the temporary file afterward. Sparse-file allocation behavior remains filesystem-dependent.
On the target toolchain, build both a small type/layout probe and the real library consumer. Exercise offsets just below, at and above 2,147,483,647, plus the product maximum. Test stat, positioning, chunked read/write, serialization and reopen together. Keep a negative narrow-wrapper fixture so the suite detects a deliberately introduced truncation.
State the remaining limits
Large offsets do not guarantee enough address space for a whole-file mapping, enough storage for real data, or support from every filesystem and device interface. File-size limits, permissions and allocation failures remain distinct from an ABI width problem. Do not expand production files to diagnose those boundaries.
The deliverable is an offset-path inventory with build identities and per-boundary results. Release only when every participating component preserves the supported positions or rejects unsupported values explicitly. “The executable is 32-bit” and “the file is larger than 2 GiB” are compatible facts; the narrowest unchecked interface determines whether the product handles them correctly.
