4.1.0
Neural-ART support for SR6P3E vs STM32N6 devices


ST Edge AI Core

Neural-ART support for SR6P3E vs STM32N6 devices


for STELLAR target, based on ST Edge AI Core Technology 4.1.0



r1.0

Overview

This article higliths the main differences to support the SR6P3E vs the STM32N6 through the STEdge AI core.

SR6P3E NPU sub-system

The SR6P3Exx series is the next generation of STELLAR devices embedding a Neural ART accelerator™ running at 500 MHz with 82 Gops performance to deploy deep neural network (DNN) models. It is based on a 32-bit Arm® Cortex® 52+ core with SP-FPU unit running up to 500 MHz. It includes 4 × 32-bit Cortex® R52+ cores (2 in split-lock configuration), configurable as either 4 cores (2 of them in lockstep configuration) or 3 cores (all of them in lockstep configuration). It also includes a 1 Cortex® M4 multipurpose accelerator running at up to 200 MHz, in lockstep configuration.

As illustrated in the following figures, the memory subsystem is based on a 128-bit Arm® CoreLink NIC-400 Network Interconnect, a 64-bit Network-on-chip (NOC) Interconnect and a 128-bit AXI crossbar. This architecture allows interconnection between the MCUs/NPU and various embedded memories. The different internal memories, which are contiguous and memory-mapped, are accessible by the NPU and can be used to store activations and/or weights.

NPU subsystem

No memory is privileged for the NPU subsystem. The clusters/cores NVMs and RAMs can be used for activations and/or weights.

NPU memory sub-system

HW NPU configuration

The Neural ART accelerator™ embedded in the SR6P3E is based on the same generation of IP as the one in the STM32N6. The configuration is defined for less complex models, it is limited to 1 CA+ (Neural-ART N10-72) and includes processing units capable of handling a wide range of quantized convolutional neural network (CNN) models. Supported operators are the same without restrictions. When an operator cannot be mapped to the NPU, a fallback mechanism executes it on the MCU.

Processing unit comment
CONV_ACC (x1) The main processing unit performing the convolution operations with up to 72 8x8 multiply accumulate operations per cycle (or 18 16x16 macc/cycle). This allows a theoretical peak processing of 72 x 1 x 2 (addition and multiplication) by cycles or @500MHz, 82 GOPs including the operations from the ACTIV/ARITH/POOL units.
POOL_ACC (x1) Perform the pooling operations like local 2D windowed (NxN), min, max, average pooling as well as global max, min, or average pooling.
ACTIV_ACC (x1) Perform the activation functions associated with the convolutional neural networks: Logistic, TanH, ReLU, PReLU, etc.
ARITH_ACC (x2) Perform the arithmetic operations: element-wise addition/subtraction/multiplication or any other affine operation.
STREAM_ENG (1x6) 6 per 1 port, Key unit (smart half-DMA engine) to fetch/push data to/from memory subsystem.
  • The number of independent chains is limited by the hardware resources available. It cannot exceed the half of the number of available stream engines. On Neural-ART N10-72, maximum three chains are possible.

Additional specific units are also integrated:

Specific unit description
EPOCH_CTRL (x1) A finite state machine which is able to decode simple microinstructions from a “command stream” (blob object) to execute a set of epochs (see “Epoch definition”).
DEBUG & TRACE UNIT (x1) The debug unit allowing to collect various internal signals monitoring various internal signals (a set of specific 32b event counters).

Security aspects

No secutiry features are considered. The SR6P3E device does not include advanced security features (such as TrustZone) to protect a deployed model. All security measures must be managed at the firmware level.

End-to-end compilation flow

The same programming model and associated generation/compilation flow are used to deploy a model to an SR6P3E device. As with the STM32N6 target, the –st-neural-art option is used to invoke the specific passes required to deploy a model using the NPU. When combined with the sr6p3e target, the STEdge AI core set the requested NPU configuration files.

Typical command to generate the specialized files:

    stedgeai generate -m <mode_path> --target sr6p3e --st-neural-art <profile>@<conf-file> 

Default conf-file and associated mpool files are located at:

    %STEDGEAI_CORE_DIR%\Utilities\windows\targets\stellar\resources\NPU\SR6P3E

Example of how to write a conf-file (user_neuralart.json) and its associated mpool (./my_mpools/) files can be find at:

    %STEDGEAI_CORE_DIR%\N6_scripts

Note: An adaptation of the mpool file is required considering the memories available for the SR6P3E. You can refer to the one delivered under the stedgeai core stellar targets folder.

Feature Comparison: SR6P3E vs STM32N6

Based on the contents of the SR6P3E alpha release, following table summarizes the main supported features for the SR6P3E serie vs STM32N6.

Feature SR6P3E STM32N6
stedgeai generate with –st-neural-art yes with --target sr6p3e yes with --target stm32n6
stedgeai generate w/o –st-neural-art (MCU only) (*) yes with --target sr6p3e yes with --target stm32n6
runtime loadable/relocatable support no yes - N6_reloc scripts
stedgeai validate yes with --target sr6p3e yes with --target stm32n6
generated c-api: LL_ATON or st-ai yes yes
Build-in validation test application (IAR & GCC configuration) (see next section) yes yes (n6_loader.py script)
Build-in validation test application with USBC support no yes (n6_loader.py script)
NPU profiler service (**) no yes
Encryption service not applicable yes N6_encrypt scripts

(*) For SR6P3E/N6 MCU only target, the standard MCU flow is considered. (**) npu_profiler.py script from ai_runner package (see the “How to profile a deployed mode” article)

Model evaluation

To deploy a model on an SR6P3E built-in validation application, there is the need to generate the network files and copy them in the right locations of the validation application.

To generate the model:

$> stedgeai generate -m mobilenet_v2_0.35_224_fft_int8.tflite --target sr6p3e --st-neural-art <profile>@<user conf file>

To deploy the model:

$> copy the network generated files in the built-in Stellar Studio NPU SR6P3E validation application, compile, flash and run

To evaluate the inference time of the model:

$> export PYTHONPATH=$STEDGEAI_CORE_DIR/Utilities/scripts/ai_runner
$> python $STEDGEAI_CORE_DIR/Utilities/scripts/ai_runner/examples/checker.py -d serial:115200 --perf-only -b 10

To evaluate the accuracy of the deployed model (validate command):

$> stedgeai validate -m mobilenet_v2_0.35_224_fft_int8.tflite --target sr6p3e --mode target -d serial:115200