Neural-ART support for STM32H7P vs STM32N6 devices
for STM32 target, based on ST Edge AI Core Technology 4.1.0
r1.0
Overview
This article higliths the main differences to support the STM32H7P vs the STM32N6 through the STEdge AI core.
STM32H7P NPU sub-system
The STM32H7Pxxx series is the next generation of STM32 devices embedding a Neural ART accelerator™ running at 550 MHz with 90 Gops performance to deploy deep neural network (DNN) models. It is based on a 32-bit Arm® Cortex®-M7 core with DP-FPU units running up to 550 MHz. It also includes a Hardware Signal Processor (HSP) that supports matrix, vector, and digital signal processing operations with 32-bit floating-point support.
As illustrated in the following figures, the memory subsystem is based on a 64-bit AMBA® AXI interconnect and a 32-bit multilayer AMBA AHB matrix. This architecture allows interconnection between the MCU/NPU and various embedded and external memories through FMC or XSPI interfaces. The different internal memories, which are contiguous and memory-mapped, are accessible by the NPU and can be used to store activations and/or weights.
No memory is privileged for the NPU subsystem. Furthermore, since there is no dedicated NPU/AXU cache for external memory accesses, the use of off-chip memories to store runtime activations or weights should be limited.
HW NPU configuration
The Neural ART accelerator™ embedded in the STM32H7P is based on the same generation of IP as the one in the STM32N6. The configuration is defined for less complex models, it is limited to 1 CA+ (Neural-ART N10-72) and includes processing units capable of handling a wide range of quantized convolutional neural network (CNN) models. Supported operators are the same without restrictions. When an operator cannot be mapped to the NPU, a fallback mechanism executes it on the MCU.
| Processing unit | comment |
|---|---|
| CONV_ACC (x1) | The main processing unit performing the convolution operations with up to 72 8x8 multiply accumulate operations per cycle (or 18 16x16 macc/cycle). This allows a theoretical peak processing of 72 x 1 x 2 (addition and multiplication) by cycles or @550MHz, 90 GOPs including the operations from the ACTIV/ARITH/POOL units. |
| POOL_ACC (x1) | Perform the pooling operations like local 2D windowed (NxN), min, max, average pooling as well as global max, min, or average pooling. |
| ACTIV_ACC (x1) | Perform the activation functions associated with the convolutional neural networks: Logistic, TanH, ReLU, PReLU, etc. |
| ARITH_ACC (x1) | Perform the arithmetic operations: element-wise addition/subtraction/multiplication or any other affine operation. |
| STREAM_ENG (2x3) | 3 per port, Key unit (smart half-DMA engine) to fetch/push data to/from memory subsystem. |
- The number of independent chains is limited by the hardware resources available. It cannot exceed the half of the number of available stream engines. On Neural-ART N10-72, maximum three chains are possible.
Additional specific units are also integrated:
| Specific unit | description |
|---|---|
| EPOCH_CTRL (x1) | A finite state machine which is able to decode simple microinstructions from a “command stream” (blob object) to execute a set of epochs (see “Epoch definition”). |
| DEBUG & TRACE UNIT (x1) | The debug unit allowing to collect various internal signals monitoring various internal signals (a set of specific 32b event counters). |
Security aspects
No secutiry features are considered. The STM32H7P device does not include advanced security features (such as TrustZone) to protect a deployed model. All security measures must be managed at the firmware level.
End-to-end compilation flow
The same
programming model and associated generation/compilation
flow are used to deploy a model to an STM32H7P device. As with
the STM32N6 target, the –st-neural-art
option is used to invoke the specific passes required to deploy a
model using the NPU. When combined with the stm32h7
target, the STEdge AI core set the requested NPU configuration
files.
Typical command to generate the specialized files:
stedgeai generate -m <mode_path> --target stm32h7 --st-neural-art <profile>@<conf-file> Default conf-file and associated mpool files are located at:
%STEDGEAI_CORE_DIR%\Utilities\windows\targets\stm32\resources\NPU\STM32H7P5XXExample of conf-file (user_neuralart.json) and
associated mpool (./my_mpools/) files are located
at:
%STEDGEAI_CORE_DIR%\N6_scriptsFeature Comparison: STM32H7P vs STM32N6
Based on the contents of the STM32HP alpha release, following table summarizes the main supported features for the STM32H7P serie vs STM32N6.
| Feature | STM32H7P | STM32N6 |
|---|---|---|
stedgeai generate with
–st-neural-art |
yes with
--target stm32h7 |
yes with
--target stm32n6 |
stedgeai generate w/o
–st-neural-art (MCU only) (*) |
yes with
--target stm32h7 |
yes with
--target stm32n6 |
| runtime loadable/relocatable support | no | yes - N6_reloc
scripts |
stedgeai validate |
yes with
--target stm32h7 |
yes with
--target stm32n6 |
| generated c-api: LL_ATON or st-ai | yes | yes |
| Build-in validation test application (IAR & GCC configuration) (see next section) | yes (h7p_loader.py
script) |
yes (n6_loader.py
script) |
| Build-in validation test application with USBC support | no | yes (n6_loader.py
script) |
| NPU profiler service (**) | yes | yes |
| Encryption service | not applicable | yes N6_encrypt
scripts |
(*) For STM32H7/N6 MCU only target, the
standard MCU flow is considered. (**)
npu_profiler.py script from ai_runner package (see the
“How to profile a deployed
mode” article)
Model evaluation
To deploy a model on an STM32H7P built-in validation application (the script configuration is similar to the N6 environment; see the article “Getting started - How to evaluate a model on STM32N6 board”), note that the IAR environment and/or STM32CubeProgrammer must support the STM32H7P series.
To generate and deploy the model
$> stedgeai generate -m mobilenet_v2_0.35_224_fft_int8.tflite --target stm32h7 --st-neural-art <profile>@<user conf file>
$> python $STEDGEAI_CORE_DIR/Utilities/scripts/N6_scripts/h7p_loader.pyTo evaluate the inference time of the model:
$> export PYTHONPATH=$STEDGEAI_CORE_DIR/Utilities/scripts/ai_runner
$> python $STEDGEAI_CORE_DIR/Utilities/scripts/ai_runner/examples/checker.py -d serial:921600 --perf-only -b 10To evaluate the accuracy of the deployed model (validate
command):
$> stedgeai validate -m mobilenet_v2_0.35_224_fft_int8.tflite --target stm32n6 --mode target -d serial:921600