How can you use edge AI for IoT?

As the Internet of Things (IoT) devices (networked embedded systems/cyber-physical systems (CPS)) are being stuffed with more processing capabilities and memory and storage capacities, IoT devices are destined to join mainstream computing. Also, there are containerization and cluster management platform solutions such as Kubernetes. Thus, establishing device clusters/clouds is being simplified and speeded up. IoT devices are, therefore, individually and collaboratively capable of running highly optimized artificial intelligence (AI) models to provide real-time insights. This concept of bringing forth swift and sagacious inferences by deploying competent AI models on IoT edge devices results in a series of real-world business innovations.

Edge AI main advantage is these devices can run without the need of internet connectivity so for places like ocean or remote areas can be very useful. This run data can later be synced with cloud services. All major cloud providers give options to use Edge AI.

Edge AI empowers IoT by processing data locally and minimizing latency... Applications include anomaly detection, image analysis, predictive analytics, energy efficiency, NLP, healthcare monitoring etc. Implementation relies on capable hardware and compatible software frameworks with attention to power efficiency and scalability.

Edge AI is the amalgamation of Artificial Intelligence techniques with edge computing, enabling real-time data analysis on or near IoT devices, bypassing the need for central data-processing hubs. By doing so, it cuts down the latency tied to cloud data transmission, paving the way for prompt decision-making. This framework not only amplifies privacy and security by keeping sensitive data on-site but also trims down data transmission expenses. It's especially pivotal in environments demanding swift insights and low latency like autonomous driving, industrial automation, and smart urban ecosystems.

IoT is a network of connected objects that generate tons of data. Sending all this data to the cloud for processing is slow and risky. That's where Edge AI steps in, like a turbocharger for IoT: 1. Speed: Edge AI makes devices lightning-fast, so they're not dependent on slow networks. 2. Privacy: Your data stays on the device, making it more secure. 3. Efficiency: It saves energy and money by processing data locally. 4. Reliability: Even without the internet, it keeps devices running smoothly. 5. Insights: Devices make instant decisions, improving automation. In a nutshell, Edge AI powers up IoT, making it faster, safer, and smarter.

Cost Reduction: Transmitting and storing vast amounts of data in the cloud can be expensive. Edge AI reduces operational costs by minimizing the volume of data that needs to be sent to the cloud. Bandwidth Efficiency: Edge AI conserves bandwidth by processing data locally. This is particularly advantageous in situations where network availability or speed is unreliable. Security and Privacy: Keeping personal data on edge devices and applying encryption or anonymization before any data transfer enhances security and privacy. Greater Resilience: Edge AI solutions are less susceptible to network outages. They can continue to function effectively even in the absence of an internet connection.

Edge AI in IoT enables local data processing, expediting decisions. For example, in smart factories, it can instantly detect anomalies, averting costly issues. Scalability is enhanced as it distributes processing tasks, easing IoT network expansion. Privacy is fortified as data stays on-site, reducing exposure. It cuts transmission costs, promoting sustainability by reducing network and energy demands. While challenges like setup complexity exist, the long-term gains in efficiency, cost reduction, and security make Edge AI on IoT a viable solution for modern, sustainable, and responsive IoT applications.

Edge AI for IoT can help smart homes for exemple become even smarter and more energy efficient. If we take the exemple of lighting, AI could help a lot to save energy while keeping great confort.

Nowadays cloud technology is widely used for IoT deployment on the edge. For example, the AWS services FreeRTOS, IoT Greengrass and IoT ExpressLink. FreeRTOS is an open-source, real-time operating system for edge devices. You can send API calls and receive a response with low latency which is a crucial factor when deploying on edge.

As a lighting pro, I see AI as the main brain in a building who will be able to collect and analyze all kind of information coming from HVAC, lighting, stores and different appliances. This way, the algorithm behind every building would be entirely handled by AI and would be dynamique and flexible according to the use of the building and the need of people inside it.

There are many toolkits that make it easy to implement edge AI for IoT; removing much of the burden from the developer. Most notable one is OpenVINO by Intel, which handles optimising models from popular frameworks like TensorFlow and PyTorch to run on x86 CPUs - meaning you can also use OpenVINO to run your models on AMD CPUs. OpenVINO also provides a unified API and runtime, making it easier for developers to target different devices without significant code changes. TensorFlow Lite is another great option, which makes it easy to deploy to IoT devices, Raspberry Pis, and mobile devices.

Successful implementation requires a holistic approach. Tailoring AI applications to the specific edge device's hardware capabilities is crucial for efficient operation. Optimizing AI models and code to match device specifications ensures optimal performance. Defining clear communication protocols and standards for interaction with the cloud and other devices guarantees interoperability. This approach demands a balance between AI model complexity and device resources, making effective model optimization and deployment essential.

Model size (impacting e.g. storage, memory, CPU requirements) and ways of reducing it, matter. Quantization is a technique to reduce the model size (and increase the inference speed by doing so) without decreasing accuracy too much. So far, research agrees that using a larger quantized model performs better than using a smaller model without quantization. However, the devil is in the details and it makes sense to carefully read up on the tradeoffs, possibly even benchmarking options yourself (and if you are nice, share the results back to the community :)). There are already many quantized models available to choose from.

Modern traffic light systems use AI to detect blue light runners and emergency services, setting green lights all the way for them and allowing traffic to flow freely with them. Companies like Deep Green are an 'edge data centre'. They place compute units in buildings that can utilise the heat they generate, thus having a benefit to sustainability and being more resourceful. For example, an edge data centre can be heating a public swimming pool while it is training an LLM model. The edge is excellent for being resourceful on space utilisation - less land can be used for high-multi million pound data centres.

At @DryadNetworks we are using edge AI to detect wildfires with solar-powered gas sensors detecting patterns if the gas composition of the air typical for a wildfire.

Edge AI's versatility is showcased through various examples across different industries. For instance, in the smart home and smart city domain, edge AI can enable lighting control systems to adjust based on local conditions, optimizing energy use. In healthcare, wearable devices can analyze vital signs, allowing for early detection of health issues without the need for continuous cloud connectivity. The applications span across manufacturing, agriculture, retail, and entertainment, demonstrating the broad potential of edge AI.

In industrial IoT, the main edge AI use case at the moment is machine vision (e.g., automated camera-based detection of anomalies) mostly in conjunction with production line quality control. People also often cite Predictive Maintenance but there are only some instances that actually require "edge AI". In many cases a more central AI is sufficient because there is enough time between the detection of an upcoming failure and the actual failure. (Some companies still consider it for cost-related architectural reasons) Augmented Reality is another upcoming industrial edge AI use case where some vision data needs to be rendered locally and in near real-time...

For outdoor lighting, there are video sensors that are able to analyze different scenarios and adjust the light brightness and color temperature based on the situation. For exemple, if there's an accident at midnight when light is dimmed at 10%, the AI would detect the accident and switch to 100% to allow an easier intervention for ambulance and police.

Edge AI for IoT encounters challenges in terms of managing power and scalability: 1. Power Pinch: Many edge devices run on limited power sources, demanding energy-efficient AI solutions. 2. Scalability Snags: Extending AI capabilities to a large number of devices can be complex and resource-intensive. These hurdles highlight the need for clever solutions to make edge AI work effectively in IoT setups.

Data privacy is one of the biggest concerns when it comes to deploying Edge AI in IoT ecosystems. Securing data on edge devices is really challenging, as they are susceptible to physical attacks or unauthorized access. So, ensuring end-to-end encryption and robust authentication mechanisms is essential for protecting sensitive information.

Challenges for Edge AI solutions include: - Model design: The goal is to reduce the model’s inference time on the IoT device. Deep Neural Networks (DNNs) often require storing and accessing a large number of parameters that describe the model architecture. So we need DNN architectures with a reduced number of parameters. SqueezeNet is a good example of efficient DNN architecture, optimized for Computer Vision use-cases. - Model compression to address memory constraints of IoT devices. There are mainly two ways to perform model compression: Lowering precision and fewer weights (pruning). E.g., TensorFlow Lite, can be used to reduce the model parameters from float32 bits to unit8, at the expense of (slightly) lower precision.

While the benefits are evident, challenges must be addressed. Resource limitations on edge devices, such as limited computing power and memory, can affect the performance and quality of AI applications. Ensuring data quality and diversity is essential to maintain accuracy, especially when dealing with noisy or incomplete data. Security and privacy concerns remain significant, and robust encryption and authentication mechanisms are needed to protect sensitive data. Finally, the balance between edge and cloud processing is a critical consideration to optimize both latency and data privacy.

If you're familiar with the history of AI, you know that we faced an AI winter in the 90s because of a limit on compute power. Thankfully, that is a relic of the past for traditional AI. Now, this is a problem Edge AI for IoT may face. It's no longer just about compute power though, it is also about compute *efficiency*. We only have so much power that can fit into an IoT device, and AI is very compute heavy. So, how do you get around this? 1. Use more efficient models, perhaps quantized models 2. Evaluate just how much of your process needs AI vs just nice to have 3. Make sure you're only running the AI when you need to

It needs to be clear what is meant by edge AI for IoT? 1. Which edge? The far edge (i.e. the sensor), the thick edge (e.g., the local data center), or something in between (the gateway, the IPC, the PLC, or other hardware) 2. Which AI? Generative vs. Predictive vs. Autonomous vs. ... (there are several classifications). 3. What kind of AI? Training or inference or both. Each has quite different requirements. 4. What kind of IoT? Consumer IoT vs. industrial vs. enterprise? A constraint IoT device vs. a tethered device.... --> All of these are important things to be clear about in order to ensure we compare apples with apples and do not have people talking to each other lost in translation.

Edge AI acts as a transformative catalyst, revolutionising the way devices process and utilise data. By leveraging the computational power of local hardware, it empowers IoT devices to analyse and interpret data without the need for constant connectivity to the cloud. One of its key advantages is its capacity to facilitate real-time decision-making at the source of data generation. This real-time responsiveness is crucial for applications where split-second actions are imperative. Moreover, the deployment of AI algorithms at the edge enhances the overall efficiency of the system, reducing the volume of data that needs to be transmitted to the cloud for processing. This is how Edge AI and IoT are reshaping the landscape of technology!!

Optimize performance and resource utilization by determining which tasks are best suited for edge processing and which for cloud processing. It's a strategic decision that impacts system efficiency, latency, and scalability. Insight: Striking the right balance between edge and cloud processing is akin to conducting an orchestra of data. It's about making informed choices to ensure efficient data handling, analysis, and, ultimately, the success of your Edge AI for IoT implementation. Edge AI empowers IoT devices to become intelligent, real-time decision-makers, driving efficiency, security, and innovation across various industries.

Don't forget that using the edge - when it fits the use case - is way more sustainable then unnecessarily sending everything to the cloud and back (which incidentally also means it will be cheaper). Taking the huge and ever-growing amounts of data and the increasing digitization into consideration, this is a relevant factor for sustainability.