MOBILE VISION-BASED TOOLS FOR YIELD ESTIMATION IN WINE GRAPES USING RGB AND 3D IMAGING

Priyanka Upadhyaya

doi:10.7273/000007489

Early season precision yield estimation is a critical challenge in modern viticulture, influencing vineyard operation management, resource allocation, and productivity. Traditional methods, such as manual cluster and berry counting, are labor-intensive, prone to sampling bias, affected by vine-to-vine variability, and limited in accuracy due to occlusions and environmental conditions. These factors contribute to inconsistent and low throughput yield estimation, limiting their practical use. Therefore, this study developed a smartphone-based machine vision system that integrates RGB imaging, 3D point cloud analysis, and deep learning models to improve both early- and harvest-season yield prediction. This study first investigated early-season yield estimation by automating lag-phase detection, a critical stage in berry development when growth slows, and berry size reaches approximately half of its final weight. Using RGB images and a Mask Region-based Convolutional Neural Network (Mask R-CNN) model, the system detected and measured berry size throughout the season. The model achieved a Root Mean Square Error (RMSE) of 0.473 mm and an R2 of 0.837 against ground truth berry measurements. The system successfully tracked berry growth and identified the onset of the lag-phase with a Mean Absolute Error (MAE) of seven days. Automating this traditionally labor-intensive process allows growers to predict the onset of lag phase leading to estimating current crop and predict harvest yield. Such estimates can be used to make timely production management decisions including crop thinning, irrigation, and fertigation, ultimately optimizing yield and berry quality. Early forecasting/prediction of harvest yield also helps growers with effective harvest and post-harvest planning. This dissertation also investigated on ways the 3D point cloud data could be used to extract canopy and cluster volumes to improve harvest yield estimation. It involved segmenting canopy and cluster structures from 3D point cloud data acquired using a mobile device. Canopy volume was found to be a critical feature for yield prediction, which was segmented with 98% accuracy using a Gradient Boosting Classifier. For grape cluster segmentation, a YOLO11 deep learning model with 0.98 mean Average Precision (mAP) was employed to detect and isolate clusters from RGB images before generating 3D models. The detected cluster images were processed using Structure-from-Motion (SfM) to reconstruct dense 3D point clouds, providing segmented cluster point clouds necessary for volume estimation. Once segmentation was completed, surface reconstruction techniques were applied to compute canopy and cluster volumes. An alpha shape reconstruction algorithm was then used to generate a watertight 3D surface from the segmented point clouds, enabling precise volume calculations. The reconstructed canopy surface, when compared to ground truth, demonstrated a relative error of 21.68%, confirming its accuracy in representing canopy structure and maintaining measurement reliability. For cluster volume estimation, the SfM-derived 3D models were analyzed against ground-truth volume measurements obtained from manually measured clusters. However, occlusions and irregular cluster geometries introduced challenges, leading to an RMSE of 46.47 cm3 (44.72% relative error) for partially occluded clusters. When only fully visible clusters were analyzed, RMSE improved to 24.50 cm3 (22.28% relative error), highlighting the importance of reducing occlusions and improving reconstruction accuracy for more precise cluster volume estimation. Finally, a comprehensive yield estimation model was developed combining RGB imagery features and 3D canopy volume data. Deep learning models effectively detected key vineyard attributes, with the YOLO11 model achieving a mAP of 0.76 for grape cluster detection and segmentation and 0.65 for shoot detection, despite challenges posed by overlapping and occluded clusters and shoots. The yield estimation model, developed using Linear Regression, demonstrated moderate predictive performance at the individual vine level, achieving an R2 of 0.375 and an RMSE of 1.35 kg per vine. The low R2 suggests that yield at the individual vine level was influenced by factors beyond the model’s predictors, such as variability in vine vigor, soil properties, and microclimate effects. However, when aggregated at the vineyard scale, the model’s performance improved significantly. The total harvest yield prediction error at the vineyard level decreased from 4.21% to 1.61% as the sampling percentage increased from 15% to 40% of vines. These results suggest that errors and variabilities at the individual vine level may be balanced across the vineyard, making the system practical for vineyard management operations such as harvest logistics and planning. By leveraging both 2D and 3D imaging techniques, the model successfully addressed challenges such as occlusion, lighting variability, and irregular cluster structures, demonstrating its high accuracy, scalability, and adaptability across diverse vineyard conditions. The use of smartphones to collect RGB and 3D images ensures that this system is highly practical, enabling growers to implement precision agriculture techniques without the need for specialized equipment. Overall, this dissertation demonstrated the feasibility of an automated machine vision system for both early-season (lag-phase-based) and harvest-season yield prediction. By addressing key challenges in early and harvest season yield estimation, these systems enhance the precision of vineyard management operations, providing a foundation for improved sustainability, reduced resource waste, and higher productivity.

MOBILE VISION-BASED TOOLS FOR YIELD ESTIMATION IN WINE GRAPES USING RGB AND 3D IMAGING

Files and links (1)

Abstract

Metrics

Details