ESP32-S3 PSRAM Integration Must Preserve Buffer Capabilities When Resizing

Byadmin

A buffer allocated in internal RAM can quietly lose that placement guarantee when a generic library resizes it. On an ESP32-S3 product with PSRAM enabled, checking the first allocation is therefore insufficient. The allocation contract must survive every resize, replacement and ownership handoff.

This source review targets ESP-IDF v5.2.5 on ESP32-S3 with external RAM available to the default allocator. It examines a common integration boundary: a product allocates memory with explicit capabilities, then passes it into code that calls ordinary realloc(). No firmware was compiled or executed for this article.

Follow the ordinary realloc path rather than the original request

The tagged newlib integration routes realloc() to heap_caps_realloc_default(). It does not forward a capability mask previously supplied by the application. The default allocator implementation selects an internal or PSRAM preference according to the configured size threshold when external default allocations are enabled, then tries a less restrictive default-capability fallback if necessary.

That is a new allocation policy decision, not a promise to preserve the caller’s original internal-memory requirement. A resize can move the allocation; a successful call only demonstrates that the new request was satisfied under the selected policy. Do not infer continued DMA suitability or cache-off accessibility from a non-NULL result.

For a hypothetical capture service, an internal control buffer may initially be small. A library expands it when a larger configuration arrives. The product can then fail only after that feature is used, even though startup memory checks passed. The important reproduction variable is the resize path and requested size, not merely whether the board contains PSRAM.

An explicitly internal buffer enters a resize boundary. Ordinary realloc selects the default allocation policy and may move it to PSRAM, while capability-aware resizing preserves the required internal-memory contract or reports failure.

Original allocation-policy diagram. Movement is a permitted outcome under the selected policy, not a measured result for every resize.

Write capabilities into the buffer’s interface

Describe each shared buffer with its byte capacity, owner, lifetime and required capabilities. If it must remain internal, that condition belongs in the interface documentation and resize helper, not only at the first allocation call. If a peripheral imposes alignment, region or transfer-size restrictions, record those separately; a generic heap capability is not a substitute for a driver’s complete buffer contract.

The capability allocator API provides heap_caps_realloc() with an explicit mask. For an ordinary byte buffer whose requirement is internal byte-addressable memory, a task-context resize can preserve that requirement as follows. This is illustrative integration code, not a compiled driver or a concurrency protocol.

#include <stddef.h>
#include <stdbool.h>
#include "esp_heap_caps.h"

bool resize_internal_bytes(void **buffer, size_t bytes)
{
    if (buffer == NULL || bytes == 0) {
        return false; /* zero-size policy is explicit */
    }
    void *next = heap_caps_realloc(
        *buffer, bytes, MALLOC_CAP_INTERNAL | MALLOC_CAP_8BIT);
    if (next == NULL) {
        return false; /* original allocation remains owned */
    }
    *buffer = next;
    return true;
}

This failure branch assumes the build does not abort on allocation failure; review CONFIG_HEAP_ABORT_WHEN_ALLOCATION_FAILS and registered failure hooks. Only call such a helper while no ISR, DMA engine or other borrower can still use the old address. Update the separately stored capacity only after success. The example deliberately does not promise preservation of an extra alignment requirement; use the selected driver’s supported allocation strategy when alignment must exceed the allocator’s ordinary guarantee.

Distinguish capacity failure from capability failure

The base reallocator checks whether the existing heap matches the newly requested capabilities. It can attempt a same-heap resize or allocate elsewhere with the requested mask and copy the contents. If a nonzero allocation fails, callers must keep the old pointer rather than overwrite it with NULL.

A system can have abundant free PSRAM yet lack a sufficiently large eligible internal block. Report the requested mask, requested bytes and largest free block for that capability class when investigating failure. Total free memory across different pools is not the capacity of one contiguous internal allocation. Treat this as a capacity-planning issue, not evidence that capability checking should be removed.

Reserve enough internal memory for critical product paths, and make oversized requests fail in a controlled way. Avoid an automatic fallback that changes a must-remain-internal requirement into a preference. If the product can use a slower or less capable alternative, make that a separate state with its own approved behavior.

Check the actual reason internal placement is required

The ESP32-S3 external-RAM guide explains cache-dependent PSRAM access and distinguishes data buffers from DMA descriptors, which cannot be placed in PSRAM in this baseline. It also documents configurations that keep cache available during SPI1 flash operations when both PSRAM instruction and read-only-data relocation options are enabled. Do not simplify this into “all S3 DMA rejects PSRAM” or “every flash write makes every configuration inaccessible.”

State the exact peripheral and build configuration behind each requirement. A buffer used only by a normal task may have no reason to consume scarce internal memory. Conversely, a control object reachable from a cache-off path needs a stronger argument than successful operation while no flash write occurs.

Exercise resize transitions before release

Test requests below, at and above the configured default-allocation threshold. Add a fragmented internal-heap condition using a controlled test harness, then observe address region, contents, success or failure, and the ownership of the original pointer. Compare the generic library path with the capability-preserving path without changing other workload variables.

Include allocation failure and repeated grow/shrink cycles. For peripheral-owned buffers, qualify resize only after the driver confirms that the old storage is no longer borrowed. Retain the sdkconfig, linker map, allocation trace and exact library versions.

No heap stress, peripheral transfer or cache-off experiment was run here. The release criterion is that every buffer either retains its declared requirements across resizing or produces an explicit, handled failure. Initial placement alone cannot establish that invariant.

Similar Posts