4.1.0
Neural-ART support for STM32H7P vs STM32N6 devices


ST Edge AI Core

Neural-ART support for STM32H7P vs STM32N6 devices


for STM32 target, based on ST Edge AI Core Technology 4.1.0



r1.0

Overview

This article higliths the main differences to support the STM32H7P vs the STM32N6 through the STEdge AI core.

STM32H7P NPU sub-system

The STM32H7Pxxx series is the next generation of STM32 devices embedding a Neural ART accelerator™ running at 550 MHz with 90 Gops performance to deploy deep neural network (DNN) models. It is based on a 32-bit Arm® Cortex®-M7 core with DP-FPU units running up to 550 MHz. It also includes a Hardware Signal Processor (HSP) that supports matrix, vector, and digital signal processing operations with 32-bit floating-point support.

As illustrated in the following figures, the memory subsystem is based on a 64-bit AMBA® AXI interconnect and a 32-bit multilayer AMBA AHB matrix. This architecture allows interconnection between the MCU/NPU and various embedded and external memories through FMC or XSPI interfaces. The different internal memories, which are contiguous and memory-mapped, are accessible by the NPU and can be used to store activations and/or weights.

NPU subsystem

No memory is privileged for the NPU subsystem. Furthermore, since there is no dedicated NPU/AXU cache for external memory accesses, the use of off-chip memories to store runtime activations or weights should be limited.

NPU memory sub-system

HW NPU configuration

The Neural ART accelerator™ embedded in the STM32H7P is based on the same generation of IP as the one in the STM32N6. The configuration is defined for less complex models, it is limited to 1 CA+ (Neural-ART N10-72) and includes processing units capable of handling a wide range of quantized convolutional neural network (CNN) models. Supported operators are the same without restrictions. When an operator cannot be mapped to the NPU, a fallback mechanism executes it on the MCU.

Processing unit comment
CONV_ACC (x1) The main processing unit performing the convolution operations with up to 72 8x8 multiply accumulate operations per cycle (or 18 16x16 macc/cycle). This allows a theoretical peak processing of 72 x 1 x 2 (addition and multiplication) by cycles or @550MHz, 90 GOPs including the operations from the ACTIV/ARITH/POOL units.
POOL_ACC (x1) Perform the pooling operations like local 2D windowed (NxN), min, max, average pooling as well as global max, min, or average pooling.
ACTIV_ACC (x1) Perform the activation functions associated with the convolutional neural networks: Logistic, TanH, ReLU, PReLU, etc.
ARITH_ACC (x1) Perform the arithmetic operations: element-wise addition/subtraction/multiplication or any other affine operation.
STREAM_ENG (2x3) 3 per port, Key unit (smart half-DMA engine) to fetch/push data to/from memory subsystem.
  • The number of independent chains is limited by the hardware resources available. It cannot exceed the half of the number of available stream engines. On Neural-ART N10-72, maximum three chains are possible.

Additional specific units are also integrated:

Specific unit description
EPOCH_CTRL (x1) A finite state machine which is able to decode simple microinstructions from a “command stream” (blob object) to execute a set of epochs (see “Epoch definition”).
DEBUG & TRACE UNIT (x1) The debug unit allowing to collect various internal signals monitoring various internal signals (a set of specific 32b event counters).

Security aspects

No secutiry features are considered. The STM32H7P device does not include advanced security features (such as TrustZone) to protect a deployed model. All security measures must be managed at the firmware level.

End-to-end compilation flow

The same programming model and associated generation/compilation flow are used to deploy a model to an STM32H7P device. As with the STM32N6 target, the –st-neural-art option is used to invoke the specific passes required to deploy a model using the NPU. When combined with the stm32h7 target, the STEdge AI core set the requested NPU configuration files.

Typical command to generate the specialized files:

    stedgeai generate -m <mode_path> --target stm32h7 --st-neural-art <profile>@<conf-file> 

Default conf-file and associated mpool files are located at:

    %STEDGEAI_CORE_DIR%\Utilities\windows\targets\stm32\resources\NPU\STM32H7P5XX

Example of conf-file (user_neuralart.json) and associated mpool (./my_mpools/) files are located at:

    %STEDGEAI_CORE_DIR%\N6_scripts

Feature Comparison: STM32H7P vs STM32N6

Based on the contents of the STM32HP alpha release, following table summarizes the main supported features for the STM32H7P serie vs STM32N6.

Feature STM32H7P STM32N6
stedgeai generate with –st-neural-art yes with --target stm32h7 yes with --target stm32n6
stedgeai generate w/o –st-neural-art (MCU only) (*) yes with --target stm32h7 yes with --target stm32n6
runtime loadable/relocatable support no yes - N6_reloc scripts
stedgeai validate yes with --target stm32h7 yes with --target stm32n6
generated c-api: LL_ATON or st-ai yes yes
Build-in validation test application (IAR & GCC configuration) (see next section) yes (h7p_loader.py script) yes (n6_loader.py script)
Build-in validation test application with USBC support no yes (n6_loader.py script)
NPU profiler service (**) yes yes
Encryption service not applicable yes N6_encrypt scripts

(*) For STM32H7/N6 MCU only target, the standard MCU flow is considered. (**) npu_profiler.py script from ai_runner package (see the “How to profile a deployed mode” article)

Model evaluation

To deploy a model on an STM32H7P built-in validation application (the script configuration is similar to the N6 environment; see the article “Getting started - How to evaluate a model on STM32N6 board”), note that the IAR environment and/or STM32CubeProgrammer must support the STM32H7P series.

To generate and deploy the model

$> stedgeai generate -m mobilenet_v2_0.35_224_fft_int8.tflite --target stm32h7 --st-neural-art <profile>@<user conf file>
$> python $STEDGEAI_CORE_DIR/Utilities/scripts/N6_scripts/h7p_loader.py

To evaluate the inference time of the model:

$> export PYTHONPATH=$STEDGEAI_CORE_DIR/Utilities/scripts/ai_runner
$> python $STEDGEAI_CORE_DIR/Utilities/scripts/ai_runner/examples/checker.py -d serial:921600 --perf-only -b 10

To evaluate the accuracy of the deployed model (validate command):

$> stedgeai validate -m mobilenet_v2_0.35_224_fft_int8.tflite --target stm32n6 --mode target -d serial:921600