TinyML Image Inputs: Preserve Edge Impulse’s Packed-Pixel Signal Before DSP

A correct camera preview does not prove that an image impulse receives the intended pixels. In the Edge Impulse path examined here, each floating-point signal element carries the numeric value of one packed RGB pixel. Returning normalized channel values at that interface changes what the image extractor interprets.

The source baseline is SDK v1.95.22, commit 732b2eac85c1874746fa6fa5a2876de207c59e39, specifically its standard image extractors. Custom DSP, direct tensor input and the separate DRP-AI path are outside this scope. No camera capture, example executable, bitmap generation, model inference or prediction comparison was run.

Name the representation at each boundary

Assume capture, dimensions and pixel coordinates have already been qualified. There are still three different representations to distinguish: camera storage, the signal consumed by image DSP, and the channel values supplied to the model.

A camera might store pixels as RGB565 words. The callback’s destination type is float *, but that does not make each destination element a normalized red, green or blue channel. For this path, one element represents the whole pixel as a numeric 0xRRGGBB value. DSP subsequently expands that element into channels or a grayscale value.

Edge Impulse’s RGB565 adapter example makes the boundary concrete: it decodes channel components, constructs the packed integer value and assigns that numeric value to a floating-point callback element. It advances once per pixel. This is numeric conversion, not reinterpretation of camera bytes as a floating-point object’s bit pattern.

The example is pinned separately to commit 29a15ccf992bf2048a0f021c4b04a5b8c67878c6. It was not compiled against the newer SDK here. Its README explains on-demand conversion, while the actual program constructs synthetic pixels and writes bitmaps rather than running a classifier.

Trace the numeric cast before the tensor

In the pinned image DSP source, extract_image_features numerically casts each callback element to uint32_t. It extracts red from bits 16 through 23, green from bits 8 through 15 and blue from bits 0 through 7. The three-channel branch divides each component by 255 and emits them in RGB order.

Consider a derived fixture, not observed output:

Decoded components: R = 17, G = 34, B = 51
One signal element: numeric value 0x112233, stored as float
Float RGB DSP output: 17/255, 34/255, 51/255

Returning those three fractions directly from the callback bypasses the required packed-pixel representation. A correct tensor-scale calculation applied at the wrong boundary does not restore the missing pixel encoding. Likewise, dividing the packed integer by 255 before handing it to DSP changes the integer that will be unpacked.

Keep input cardinality separate from channel count

The pinned example’s generated metadata describes a 96 × 96 input, 9,216 raw samples, one raw value per sample and 27,648 neural input features. Its signal therefore contains 9,216 pixel elements, while the RGB feature representation contains three times as many channel elements.

These are that export’s dimensions, not universal SDK constants. Read the actual export before allocating buffers or assigning a signal length. Check both counts explicitly; a buffer being large enough does not establish that its elements have the right meaning.

Grayscale selection still follows RGB unpacking

The standard float extractor chooses one output channel for grayscale, but first unpacks the same RGB word. It then applies the displayed weighted luma calculation. A grayscale model does not make an unadorned grayscale byte the expected callback representation.

For a proposed midlevel fixture, the packed value 0x7F7F7F supplies equal components R = G = B = 127. In ideal arithmetic, the shown luma weights preserve the normalized level 127/255. The bare numeric value 127, however, becomes 0x00007F: red and green are zero, and only blue is 127. Its ideal float-path luma is 0.114 * (127/255).

That comparison is derived from the source. It is not a measured brightness error or a universal promise about grayscale sensors and custom blocks.

Leave downstream scaling with the selected extractor

The quantized image extractor also begins with numeric packed-pixel decoding. Its later scaling and quantization depend on the selected branch. The component - 128 shortcut is conditional on a particular scale, zero point and image-scaling mode; it is not a general callback rule.

Do not pre-quantize the packed word merely because the model input is integer-valued. Inspect the configured path and verify the tensor after image DSP, separately from verifying the signal before it.

Test the boundary before judging predictions

A proposed adapter fixture should include distinct channel values, equal-channel gray, black and white in a complete immutable image. Record decoded camera components, packed callback values and copied DSP features. Assert pixel-element count separately from model-channel count.

Use the actual export’s scaling branch for any later tensor comparison, with a justified numerical tolerance. Only after the representation checks pass should unchanged-model predictions be evaluated. A plausible label or a visually correct preview cannot replace evidence from the intervening signal-to-DSP boundary.