Autonomous crop disease prediction has attracted the attention of IoT systems and AI developers, as an essential enhancement in existing agricultural solutions, even in smart ones. IoT-NGIN could be no exception in this race, offering a set of next-generation internet tools, that can effectively and efficiently support the technological delivery of such autonomous UAV-based use cases.
Let’s have a look at the components used in IoT-NGIN for crop disease prediction one by one.
UAV prototype
The UAV solution has been mainly based on the DJI Matrice 600 PRO drone, on which advanced processing capabilities have been added through a processing unit and a camera that have been integrated to enable on-device development and processing. A brief description of the UAV components is provided in Table 1.
Table 1. The components of the UAV.
| Component | Brief description |
| Smart Agri Drone | DJI Matrice 600 PRO, ideal for professional aerial photography with an extended flight time and a 5km long-range transmission, intelligent batteries, and a maximum payload of 6kg. |
| Multispectral camera | Parrot Sequoia, multi-band sensor designed for agriculture, featuring excellent precision, flexible integration, and small size and weight, compatible with the Smart Agri Drone. |
| Processing unit | NVIDIA Jetson Nano 4GB, is a small, powerful computer which can run multiple neural networks in parallel for applications like image classification, which is desired for the crop disease prediction application. |
The project enhancements on the UAV aim at enabling on-device decision support during flying operations. The flying drone should be able to detect on-the-fly diseased plant units, to be able to support aerial spraying. The detection of plant diseases within the device requires that the UAV has adequate processing capabilities to be able to run Machine Learning models able to classify images of the plants as healthy or diseased. Moreover, as accurate classification relies on the clarity of the input images, the UAV should be able to capture images of proper quality and content, allowing such an ML model to work properly. Thus, the UAV is equipped with a high-resolution camera, while the processing unit can run inference ML processes on high-definition images. A tradeoff between image resolution and processing capability has been of course considered, as the inference time and energy consumption should remain at low levels in order to have a viable working prototype. To this, lab experiments have been made on the investigation of the ML operation on the processing unit, while test flights of the UAV (Figure 1) have been made to collect test datasets from the Smart Agriculture LL vineyard.
Crop Disease Prediction Service
The Crop Disease Prediction service is based on a Deep Learning model, developed to identify Esca-diseased or healthy grapevines. The ML model has been trained on the Esca dataset [1] in NVIDIA TITAN X Graphics Processing Unit (GPU) available in the Smart Agriculture LL premises and tested in Jetson nano and Jetson Xavier GPUs for the inference service, running on the drone. After validation in the LL, a new class has been added in the ML model to improve the model’s ability to handle real-world scenarios and better classify captured images when it encounters an object or scene that does not belong to any of the two classes of grapevine leaves. The model has been retrained, adding data samples from the “Nature Image Classification” dataset into the training dataset. These model updates have been reported in D7.3 [2]. In addition, the Crop Disease Prediction service gets further enhanced, based on feedback received from the LL test activities, which provide new datasets of the LL vineyard at different crop ages. The ML model performance can vary significantly, depending on the age and state of the grapevines appearing in the training dataset. Figure 2 presents sample images of healthy and Esca-diseased vineyards of the UAV-acquired datasets. The crop disease prediction service is evaluated on its capability to correctly classify the acquired images and gets retrained on the new datasets whenever possible. The final results of the service evaluation will be reported in D7.4.
IoT Device Indexing Service
The good performance of the ML service depends on the availability of datasets for proper training. In addition, the configuration, management, and updates are critical aspects for the integration of the UAV into an IoT system. Thus, in IoT-NGIN, this integration has been approached through the IoT Device Indexing service, which supports digital twin services for connected IoT devices.
Through this service, the UAV-acquired data can be stored at an edge device and accessed through a protected API. The data could be either captured images from the vineyard or even logs providing traceability on the device status and activities. In addition, over-the-air updates and configuration may be done programmatically through API calls which end up in communicating control messages to the UAV. Such control messages could suggest an ML model update or de-/activation of services running on the UAV.
IoT Device Access Control
The integration of the UAV into an IoT system cannot be done without considering security. The IoT Device Access Control module caters for protecting the programmatic APIs that provide access to the UAV data and controls. The service also provides user and identity management, ensuring that access to each resource is granted only to eligible users or services.
MLaaS platform
The MLaaS platform of IoT-NGIN is utilized for both the development and the delivery of the ML service which supports crop disease prediction. The MLaaS platform provides a set of MLOps functions which facilitate these operations. Specifically, for this service, learning, storage, and model serving are offered through the MLaaS platform.
Privacy-Preserving Federated Learning
For increased data privacy and security of ML operations, privacy-preserving Federated Learning (FL) is employed for the ML model development. In addition to plain FL, which ensures that data do not leave the data owner’s premises, IoT-NGIN FL incorporates privacy preservation techniques that aim to address malicious behaviour in the FL system, such as membership attacks. In this way, models are individually trained in local edge servers and are then aggregated in a secure way in the Smart Agriculture LL cloud infrastructure. The final (aggregated) model is stored in the MLaaS storage component and served to the UAV prototype via the MLaaS model serving component and the IoT Device Indexing module.
In conclusion, the UAV prototype developed in IoT-NGIN has utilized a set of modular software tools of IoT-NGIN, in order to yield an autonomous UAV able to run ML services, which can detect crop diseases in vineyards in the particular case executed in the project. Beyond that, the project contribution expands to the development and piloting of an autonomous, modular, and configurable UAV, which is integrated with a next-generation multi-node infrastructure. Thus, the UAV prototype can be easily exploited in other use cases that involve AI execution on the device and integration in a wider IoT, edge and cloud ecosystem.