TinyML CMSIS-NN Workspace: Size Scratch Memory for the Selected Wrapper and Target

A direct CMSIS-NN convolution call starts failing after a model or processor configuration changes. Its input and output arrays still fit, but the temporary workspace was copied from an earlier example. That workspace belongs to a particular kernel selection and shape. It is not a fixed overhead for every int8 convolution.

This article examines direct calls to the pinned CMSIS-NN int8 convolution wrapper. A runtime such as TFLM normally performs its own scratch requests; do not replace that allocator with an unrelated application buffer merely because the underlying library uses one.

Ask the sizing function for the operation you will call

The public convolution interface pairs arm_convolve_wrapper_s8 with arm_convolve_wrapper_s8_get_buffer_size. Supply the same convolution parameters and input, filter and output dimensions to sizing and execution. The result is a byte count, not a count of int16 elements.

Use the ordinary getter in a target build. The header describes the explicitly suffixed MVE and DSP sizing variants as intended for host compilation. A host-side model planner must model the intended target rather than accidentally use its own compilation defaults.

Treat a changed shape, stride, padding or dilation as a reason to recompute the request. Likewise, an archive compiled for a different architecture is not interchangeable with a size calculated from an older library's source.

Understand why a copied number can be wrong

The wrapper implementation selects specialized 1-by-1 and 1-by-N implementations when their predicates match. Other shapes reach general convolution. A zero-workspace result for one specialization does not authorize a null buffer for another shape.

The examined sizing implementation also differs between general MVE and non-MVE paths. Consider an illustrative 3-by-3 filter with three input channels: the flattened patch contains 27 elements.

• The general non-MVE calculation rounds 27 to 28, then reserves two int16 columns: 112 bytes.
• The general MVE calculation rounds to two groups of 16 int8 elements and reserves four columns: 128 bytes.

These are source-derived arithmetic examples, not measured allocations. They exclude activations, weights, output, stack and other operations. Use the getter rather than embedding these formulas in production model integration code.

Do not mistake the context size field for protection

The context structure contains a buffer pointer and a size field. However, the examined general int8 convolution implementation checks whether the pointer is null and then uses the buffer. It does not compare the requested workspace against ctx->size.

Consequently, supplying a non-null, undersized buffer need not produce a clean argument error. Validate capacity before the call. Use storage with alignment appropriate for the selected kernel's typed accesses and target, rather than an arbitrary byte address in a packed structure. Check the operation's return status, but do not treat success as a bounds audit.

Make scratch reuse follow execution lifetime

For a sequential application, a shared region can be sized for the largest request among operations that never overlap. That is an application scheduling decision: simultaneous inference contexts or interrupt-driven work cannot safely reuse the same bytes just because their individual requests fit.

Keep scratch distinct from live input and output arrays unless the specific API explicitly permits overlap. In a development build, guard regions and repeated shape-specific tests can expose writes outside the intended allocation. They supplement the sizing contract; a passing sample does not prove every supported shape is safe.

The public interface also assigns buffer clearing to the caller when applicable for security. Clear it after its last legitimate use and before sensitive contents can cross an ownership boundary. The final fix should explain both the required capacity and who owns that capacity throughout execution.