Thread-Local Storage in Plugins: Check TLS Models Before Runtime Loading
A plugin can pass the shared-library link and still fail during dlopen because its thread-local storage model requires static TLS space that the running process cannot provide. Establish the model and loader allocation path before treating the failure as a missing file or generic heap exhaustion.
This guide pins GCC 14.2, Binutils 2.44 and glibc 2.41. An executed x86-64 fixture compared two owned plugins with the same 64 KiB thread-local object but different access models. It used fresh processes with zero or two preexisting worker threads; it did not change loader tunables or production services.
Inventory the model, not just the source declaration
A __thread or thread_local declaration describes per-thread storage at the language level. The generated access sequence and relocations determine what the loader must arrange. Record explicit -ftls-model options, per-variable attributes, visibility and the final object’s architecture.
The GCC 14.2 code-generation reference lists global-dynamic, local-dynamic, initial-exec and local-exec models. It documents different defaults with and without PIC and warns that optimization can select a more efficient model. Consequently, source flags alone are insufficient evidence for the final DSO.
Inspect readelf -lW for the TLS segment’s memory size and alignment, and readelf -rW for TLS relocations. Keep the initialized file size separate from the required memory size: a zero-initialized TLS array can occupy little file data while requiring substantial per-thread storage.
Understand the static-surplus boundary
In glibc 2.41’s relocation implementation, a dynamically loaded object requiring static TLS can use existing surplus only if its size, alignment and prior allocation state permit it. The allocation path can reject an object even when ordinary heap memory remains available.
The source also distinguishes mandatory static allocation from optional static allocation used to optimize dynamic access. Failure of an optional optimization has a fallback requirement; failure of a mandatory static relocation can produce the static-TLS allocation diagnostic. Those are not interchangeable outcomes.
The 2.41 TLS allocation source accounts for loader namespaces and several users of the surplus. Do not describe one documented optional-reserve value as the total universal capacity available to arbitrary plugins. Architecture, alignment, loaded objects and loader configuration affect the actual result.
Compare two bounded host plugins
The fixture defined an externally visible, zero-initialized __thread byte array of 65,536 bytes and a function that incremented its first byte. Both plugins were built with -fPIC. One explicitly selected global-dynamic; the other selected initial-exec.
The final x86-64 global-dynamic plugin showed R_X86_64_DTPMOD64 and R_X86_64_DTPOFF64 relocations plus a __tls_get_addr reference. The initial-exec plugin showed R_X86_64_TPOFF64. Both had a TLS memory size of 0x10000. These names describe this architecture and toolchain, not a portable relocation taxonomy.
| Plugin and fresh process | Observed host result |
|---|---|
| Global-dynamic; no worker threads before loading | dlopen succeeded; main’s first call returned 1. |
| Global-dynamic; two workers created before loading | Loading succeeded; main and both workers independently returned 1. |
| Initial-exec; zero or two preexisting workers | dlopen failed with a static-TLS allocation error in both cases. |
The workers waited at a barrier until loading completed. They stopped and joined before the owned plugin was closed. This tested a small, controlled lifecycle; it did not qualify arbitrary unloading while callbacks or TLS destructors remain active.
Do not infer a thread-count threshold
The initial-exec case already failed without extra workers. Adding two workers did not establish a capacity transition. The result therefore demonstrates the tested model’s static-space constraint, not that “more than two threads” causes the error.
Per-thread storage still matters to the memory budget. The glibc 2.41 dynamic-linking tunables documentation explains that optional static reserve is allocated per thread and cannot be resized after process startup. Increasing thread count can increase total memory use without changing the per-module fit rule in the way a simple threshold story suggests.
For a product investigation, record worker count, stack settings, TLS segment sizes, plugin sequence and process startup environment separately. Include resource-allocation failures outside TLS, but do not merge them into one unexplained “out of memory” category.
Build a load-order reproduction plan
Use fresh disposable processes for each case so prior loads do not contaminate the comparison. Test the intended plugin alone, then the real dependency closure and supported load sequences. Capture dlerror immediately when loading fails, together with the exact plugin and loader hashes.
Compare loading before and after worker creation, and test the actual supported startup-linked arrangement separately if the product uses one. A plugin that happens to fit an unused surplus in a minimal test may fail after another library consumes space. A successful startup-linked build is not proof of arbitrary late-loading behavior.
Keep alternate namespaces and audit modules as explicit additional cases if they are part of the product. The host fixture did not use dlmopen or auditing. Its clean environment and small object graph should not be mistaken for a complete service-process reproduction.
Choose a model or lifecycle repair deliberately
For a genuinely late-loaded plugin, rebuild its TLS accesses under a model supported by that loading contract, then inspect the resulting relocations again. Include static dependencies folded into the plugin; changing only the top-level wrapper’s flags can leave an initial-exec reference inside it.
If the design instead requires a fixed startup dependency, document and test that lifecycle. Do not treat preloading or a larger reserve as an unexplained universal fix. Any reserve adjustment needs a reviewed per-thread memory budget and exact loader-version qualification; no such adjustment was executed here.
The release artifact is a TLS inventory plus a load-sequence/thread-state matrix. It should distinguish successful discovery from successful relocation, explain the model actually emitted and preserve the negative control. That supports a specific plugin-loading promise without inventing a universal static-TLS capacity.
