Computer vision and neural networks applied to the growth of crops

Abstract

This thesis presents an intelligent crop monitoring alternative aimed at automating visual analysis. It comprises a low-cost, low-power hardware system that operates algorithms capable of determining specific crop parameters, such as crop height, wheat spiking and stubble detection, as well as identifying the contours of leafy vegetables, in crops such as lettuce and chard. The hardware integrates a Raspberry Pi for algorithmic processing and a LoPy for power management and data transmission via LoRaWAN. The device schedules image capture based on seasonal light variations to compensate for its lack of automatic light adaptation. For height estimation, a kernel-based algorithm was developed, combining stereoscopic vision with reference-based techniques. The Hough Transform enables the identification of crop boundaries in dense plantations, followed by 3D reconstruction using stereoscopic vision to measure height. Alternatively, a reference-based approach circumvents the calibration challenges associated with stereoscopic vision in field conditions. For the detection of spiked and stubble areas, methods from the YOLOv8 and YOLOv10 family were employed, trained using proprietary images captured in low resolution with the hardware proposed in this study and previously annotated using Roboflow. For leafy vegetable contours for lettuce, a custom metric was developed to balance computational cost and accuracy, and was used to evaluate the performance of different detection methods. The YKMS method (YOLOv10 + K-means + Superpixel) enhances precision by utilising bounding box labels, while YOLOv8 and Detectron2 were also evaluated. Additionally, a combined approach named YKMSY8 (YOLOv10 + K-means + Superpixel + YOLOv8) was applied to lettuce. In the case of chard, the YKMS method was used; however, YOLOv10 was replaced by YOLOv11, demonstrating its adaptability to irregular crop shapes. The results confirm the feasibility of these techniques, which provide a cost-effective and real-time crop monitoring system to optimise decision-making and sustainability. The following outcomes were obtained: • The device was designed with a low-cost approach, prioritising minimal energy consumption and ensuring continuous operation in the field for three months with two captures per day. • Regarding height estimation, using the kernel-based algorithm combined with stereo vision, tests conducted on banded planting crops yielded a 3% error in height calculation. Similarly, an alternative method that combines the kernel-based algorithm with a reference-based approach, applied to wheat crops, also demonstrated a 3% height estimation error. • For the detection of the spiked and stubble area, YOLOv8 and YOLOv10 were employed. YOLOv8 achieved a recall of 71.1% and a precision of 79.5%, while YOLOv10 attained a recall of 70% and a precision of 77%. • To determine the contour of lettuce, neural networks such as Detectron2, YOLOv8, and a proprietary method based on YOLOv10, termed YKMS, were applied. Additionally, a hybrid approach combining YKMS with YOLOv8, referred to as YKMSY8, was introduced. A performance metric was presented to evaluate the efficiency of each neural network, incorporating both contour identification accuracy and computational cost. The following results were obtained: an accuracy of 81.9% was achieved by YOLOv8, 84% was achieved by Detectron2, 70.26% was achieved by YKMS, and 87.3% was achieved by YKMSY8. Mean error rates of 3.3% were recorded for YOLOv8, 3.9% for Detectron2, and 5.2% for both YKMS and YKMSY8. Mean inference times were recorded as 1.01s for YOLOv8, 4.12s for Detectron2, 4.07s for YKMS, and 0.46s for YKMSY8. • Preliminary results on chard were obtained with the YKMS methodology, modified to enable the detection of multiple objects within the same image, with YOLOv11 employed instead of YOLOv10.