TinyML CMSIS-NN Integration: Verify Which Kernel Path the Firmware Actually Uses

The firmware is rebuilt with CMSIS-NN enabled, but a particular model operation is no faster. The configuration flag is only one link in the integration. The model's data type, selected TFLM wrapper, linked library and processor-specific compilation all influence which code actually runs.

Trace one operation through those links before changing clock speed or replacing the model. A successful link establishes neither a specific optimized path nor a performance improvement.

Confirm that the build selected the TFLM wrappers

The pinned TFLM CMSIS-NN integration guide documents OPTIMIZED_KERNEL_DIR=cmsis_nn for its make workflow. The wrappers in that directory translate TFLM operations into the separate CMSIS-NN library interface.

Inspect the actual compile commands and resulting objects for the operation under investigation. If an IDE project was generated earlier and then copied into another build, changing a variable in the original make invocation does not automatically replace the sources in the copied project.

Check whether the firmware linked the intended wrapper or retained a reference implementation. A CMSIS-NN archive somewhere in the project directory is not evidence that the relevant object references it. Use the link map and symbol or disassembly inspection to follow the final executable, accounting for inlining and link-time optimization.

Keep downloaded sources and external libraries distinct

The integration guide supports downloaded CMSIS-NN sources, explicitly supplied source paths and a prebuilt external archive. These are different ways of providing the implementation, not interchangeable labels for an identical binary.

The examined download script pins CMSIS-NN commit 4ab83cc3cc98fb85ed6dafb55e8ca02f1628dcae. A separately downloaded latest branch need not be the source this TFLM checkout would otherwise use.

For an external archive, inspect the selected headers and its compilation settings as well as its filename. The guide warns that external-library compiler options can affect performance and size. Replacing only the archive while leaving headers or processor settings from a different build can create a misleading comparison.

Inspect the operation's actual data-type branch

The examined TFLM convolution wrapper is a useful counterexample to blanket acceleration claims. Its float32 branch calls the reference convolution implementation. Its quantized branches prepare CMSIS-NN parameters and dispatch through the corresponding wrapper functions.

Therefore, finding that source file in the build does not prove that a float convolution uses a CMSIS-NN float kernel. Library capabilities and the integration's actual calls are separate facts. Inspect the exported tensor types and the branch reached by the specific operation.

Within CMSIS-NN, the pinned int8 convolution wrapper chooses among shape-dependent implementations, including specialized 1-by-1 and 1-by-N paths. The same operator name can therefore select different functions when its dimensions or parameters change.

Verify the processor configuration before measuring

The matching CMSIS-NN documentation distinguishes pure-C, DSP-extension and MVE implementations selected through compiler architecture features. A pure-C path on a processor without the relevant extension is an expected configuration, not automatically a broken integration.

Verify that both the application and prebuilt library were compiled for features actually present in the device. Do not force an extension macro merely to make an optimized branch compile; the emitted instructions must be executable on the target.

Once the path is established, compare the same model, input and processor conditions with and without the intended optimization. Check output correctness before interpreting timing. If the optimized operation improves but total response time barely changes, investigate acquisition, preprocessing and the other model operations rather than assuming the selected kernel did not run.

Keep the conclusion narrow: identify the implementation exercised and the scope of any measured benefit. No firmware build, inference run or speed measurement was performed for this article.