The deployment of Artificial Intelligence (AI) algorithms at the Edge (i.e., IoT devices, sensors or interfaces, gateways, robots) is generally referred to as AI at the Edge. Being an extension of the Edge computing paradigm, in which data is stored and processed directly in devices or infrastructures at the edge of the network, AI at the Edge spreads the capabilities of Edge computing with Machine Learning and other types of AI competences. This means that the AI models are integrated and executed directly on Edge devices so that the data sources and the points of action of those models are as close as possible to where these AI algorithms are deployed. While the convergence between AI and Edge computing has great advantages considering it reduces the bottleneck of asking for predictions to a Cloud server where the AI models are deployed, it also comes with a set of new challenges that developers need to deal with.
Some of these challenges include, for instance, overcoming the lower resource capabilities of Edge devices, which results in the requirement of more lightweight software solutions, or dealing with new security vulnerabilities that introducing the AI systems onto the Edge devices might produce due to their direct impact on the physical world. However, there are already different types of solutions to these challenges. For instance, recently several technology companies have developed new hardware chips designed to fasten AI operations, such as Google’s Edge TPU on Google’s USB Accelerators1, or Intel’s Movidius Myriad X Vision Processing Unit on their Neural Compute stick2. Other solutions to these challenges are addressed by means of container technologies on the Edge, thanks to the advances of frameworks such as Docker, Kubernetes, or even the novel Unikernels which were explained in a recent IoT-NGIN post , that have raised the DevOps methodology to new standards.
Nonetheless, not every progress on AI at the Edge comes from “external” sources to AI, like different hardware designs or different infrastructure providers; forasmuch as there have been novelties on different kinds of AI techniques and algorithms which, applied to the Edge, are improving the efficiency and accuracy of AI systems. One of these techniques, which the IoT-NGIN project is leveraging to be part of its Machine Learning as a Service (MLaaS) platform, is the well-known Federated Learning (FL)3 discipline. While traditional Machine Learning techniques require the training data to be centralized on a central server, FL builds models based on datasets that are spread across several devices, preventing the data of those devices to be leaked outside its scope. Thanks to this approach, FL inherently enhances privacy as the data never leaves the original data sources. Notwithstanding, this distributed way of training the AI models also introduces other security vulnerabilities, such as malicious attacks on the training servers that will result in undesired predictions from the trained AI models. Luckily, there exist already some techniques to detect these attacks, such as the one explained in a recent IoT-NGIN blog post. Another ML discipline that clearly benefits from being as close to the data sources as possible is the automatic improvement of the AI models after their initial deployment, in other words, the self-adaptation of those models to the new patterns of the generated data. One of the techniques that is included within this discipline is the Online Learning methodology, where the new data is continuously processed by the AI models while it is generated, without forgetting past knowledge and without the need to retrain the model with an entire dataset. Certainly, this grants some advantages for the Edge computing paradigm, including reducing to very low storage demands as there is no need to cache a complete dataset; reducing the time required to deploy the ML model since it does not need to be initially trained and it can be deployed as soon as the data stream is available; among others. However, it also comes with some risks, as for example the so-called catastrophic forgetting4, in which some ML algorithms tend to forget past learnt patterns as new data arrives, or the fact that the initial predictions of the ML models will be quite inaccurate until some data is already processed by the model.
In addition to this Online Learning methodology, another procedure that can be applied on the Edge is the automatic data labelling of the newly generated data. One of the biggest issues when dealing with ML is to label the data to obtain a proper training dataset to be processed by the models. In this regard, automatic data labelling tries to overcome this issue by making use of a (previously trained) ML model to label the new data, which is then stored in a cache where human annotators can validate those labels. After this validation, the originally deployed ML model is then retrained with a larger dataset to produce an enhanced version which, in turn, will produce better labelling for the new data. The final goal would be that the model is so well trained that the predicted labels for new data are good enough to avoid the human validation so that the model can automatically improve itself. Nevertheless, this methodology comes with some disadvantages and risks. On the one hand, the first version of the ML model needs to be already trained with some pre-obtained data so that it is able to produce at least a few labels of the new data. On the other hand, in the cases in which the patterns of new data vary as time goes by, the predicted labels by the model may result inaccurate, so there appears a risk of having deployed a model which is not working as expected.
In addition to these methodologies, other AI techniques exist that may be applied at the Edge, which come always with some advantages and some risks. For this reason, it is always important to measure and monitor metrics for the deployed ML models, which is one of the main capabilities that the IoT-NGIN MLaaS platform plans to cope with. Check further documentation of the project on https://iot-ngin.eu/ to know more about how AI is being used to increase the capabilities of the Edge spectrum.
1https://coral.ai/products/accelerator/
2https://software.intel.com/content/www/us/en/develop/hardware/neural-compute-stick.html
3Google AI Blog: Federated Learning: Collaborative Machine Learning without Centralized Training Data.” https://ai.googleblog.com/2017/04/federated-learning-collaborative.html (accessed Jun. 29, 2021
4I. J. Goodfellow, M. Mirza, D. Xiao, A. Courville, and Y. Bengio, “An Empirical Investigation of Catastrophic Forgetting in Gradient-Based Neural Networks,” 2nd Int. Conf. Learn. Represent. ICLR 2014 – Conf. Track Proc., Dec. 2013, Accessed: May 31, 2021. [Online]. Available: http://arxiv.org/abs/1312.6211