Currently, developers are utilizing secure and performance enhancing technologies to develop small, low-power embedded systems, empowering previously unimaginable AI applications such as voice, vision, and vibration, which are changing the world.
The embedded field is undergoing a profound transformation. Connected devices are gradually evolving into systems that can make decisions based on the collected data. Compared to data processing in IoT gateways or the cloud, completing data processing closer to the collection source is expected to accelerate decision-making speed, reduce latency, address data privacy issues, reduce costs, and improve energy efficiency.
Many application fields are promoting the performance and function requirements of edge computing, such as industrial automation, robotics, smart city and home automation. In the past, sensors in such systems were much simpler and disconnected from each other. However, now artificial intelligence (AI) and machine learning (ML) have improved the level of local intelligence, allowing decision-making to be completed on the end side, which was not feasible with simple control algorithms used in the past.
The Evolution of General Purpose Processors in the AI Era
Years ago, developers focused on making logic and control algorithms the core of software development. However, with the emergence of digital signal processing (DSP) algorithms, support has been provided for many enhanced speech, visual, and audio applications.
This transformation in application development has entered a new era and is affecting the design of computing architecture. We have now developed to use reasoning as the main core of algorithm development, which has brought new or higher requirements for computing performance, energy efficiency, latency, real-time processing, and scalability.
The industry's demand is not only for new processor accelerators, but also for the improvement of general processing capabilities, in order to provide developers with the necessary balance and support applications such as feature checking or character detection in live videos.
A few years ago, developers could only rely on frequency based filters when creating noise cancellation applications. Nowadays, developers can improve the performance and functionality of applications by combining filtering with ML/AI models and inference. In order to make these development tasks more efficient and serve users as seamlessly as possible, the demand for processors and tools is also increasing day by day.
Promote the intelligence of edge and end devices
This evolution and innovation are driven by ML, but also face many technological challenges. After years of experimentation, attempting to create a development method that is universally applicable to the Internet of Things and embedded devices has prompted the industry to shift its approach to IoT development, in order to unleash the infinite possibilities of scale expansion.
Currently, developers are utilizing secure and performance enhancing technologies to develop small, low-power embedded systems, empowering previously unimaginable applications such as voice, vision, and vibration, which are changing the world. Various versions of programming languages and Transformer models will soon occupy a place in IoT edge devices with new computing capabilities. This undoubtedly brings more possibilities that developers have long dreamed of.
In the process of development evolution and innovation, in order to meet the hardware needs of developers, Arm introduced Arm in the Armv8.1-M architecture a few years ago ® Helium ™ Vector processing technology. Helium has brought significant performance improvements for ML and DSP applications in small low-power embedded devices. In addition, it also provides Single Instruction Multiple Data (SIMD) functionality, thereby integrating Arm Cortex ®- The performance of the M device has been improved to a new level and supports applications such as predictive maintenance and environmental monitoring.
Helium improves DSP and ML performance, accelerates signal conditioning (such as filtering, noise cancellation, and echo cancellation) and feature extraction (audio or pixel data), which can then be transmitted to classification using neural network processors.
Implement intelligent edge side functionality
We can see that many Arm partners have introduced Helium technology in their latest products, helping developers leverage ML capabilities on restricted devices at the farthest end of the network. In February 2020, Arm launched the Cortex-M55 processor using Helium technology, while Alif Semiconductor launched its first Cortex-M55 based chip in September 2021 and deployed the Cortex-M55 processor with Helium in its Ensemble and Crescendo product lines. In addition, Himax also uses Cortex-M55 equipped with Helium as its next-generation WE2 AI processor, targeting computer vision systems in battery powered IoT devices.
In April 2022, Arm launched its second CPU that supports Helium - Arm Cortex-M85. Renesas Electronics has conducted technical demonstrations on Cortex-M85 at embedded world 2022 and embedded world 2023. In the demonstration, Plumerai greatly accelerated its inference engine speed through Reza Electronics RA MCU technology. As a company that develops a complete software solution for person detection based on cameras, Plumerai believes that performance improvements will ensure that its customers can fully utilize the larger and more accurate Plumerai person detection AI version, while providing more product features and extending battery life. In November 2023, Arm launched its third CPU using Helium technology - Cortex-M52, which is a processor designed specifically for artificial intelligence Internet of Things (AIoT) applications. It can bring significant performance improvements to DSP and ML applications in small low-power embedded devices, allowing for the deployment of more computationally intensive ML inference algorithms at endpoints without the need for dedicated NPUs.
With the development of hardware, developers are facing increasing software complexity, thus requiring new development processes to create optimized ML models that combine efficient device drivers. The software development platform and tools provided for the ecosystem must also evolve closely with hardware, which is crucial.
Nowadays, various tools provided by Arm and third parties can be used to support end users in creating AI algorithms. After creating a model in an offline environment, data scientists can use the corresponding tools to optimize the model for use in Arm Ethos based environments ™- Run the model on the NPU of U, or use the Helium instruction on Cortex-M based processors.
Qeexo is the first company to implement end-to-end ML automation for edge devices. Its AutoML platform provides an intuitive user interface (UI), allowing users to collect, clean, and visualize sensor data, and use different algorithms to automatically build ML models. The Keil Microcontroller Development Kit (Keil MDK) and other traditional embedded tools are beneficial supplements to MLOps tools and help establish DevOps processes for validating complex software workloads. As a result, embedded, IoT, and AI applications ultimately converge into a single development process that software developers are familiar with.
The potential of the edges is gradually being explored. The demand for improving the performance of microcontrollers is still growing, especially for tasks such as voice controlled door locks, person detection and recognition, networked motor control with predictive maintenance, and countless other high-end AI and ML applications.
We believe that with the support of the right technology, developers can rethink edge and end-to-end devices, and strike an appropriate balance between key elements of limited devices such as performance, cost, energy efficiency, and privacy, enabling future embedded development to implement AI computing applications.
