Abstract
Background and goals Measuring evapotranspiration (ETc) in vineyards is important to optimize vineyard irrigation and water management practices. Previous work demonstrated a strong correlation between the amount of shaded area under the vine at high noon and crop coefficient. This parameter can be measured with a photovoltaic sensor (Paso Panel) or by hand using grid paper. We aimed to develop a low-cost and easy-to-use smartphone-based alternative to measure shaded area under a vine.
Methods and key findings Videos of the ground under a row of vines were recorded with a smartphone camera on a sunny day in the presence of resident vegetation which consisted of grasses and weeds. A novel computer vision-based algorithm using a segmentation machine learning model and structure-from-motion was developed to estimate the amount of shaded area present. Measurements were collected using a Paso Panel at the same time for comparison. Other Paso Panel measurements were collected to measure the relationship between electrical current and shaded area. Linear regression of this CV-based method to Paso Panel readings yields R2 = 0.68.
Conclusions and significance A new model for relating Paso Panel current readings to shaded area was derived empirically. Adoption of the CV-based crop coefficient estimation method could improve spatial resolution of ETc estimates, potentially aiding adoption of variable rate irrigation.
Introduction
Farmers commonly use a two-step crop coefficient approach to estimate evapotranspiration under standard conditions. This method models evapotranspiration as ETc = KcETo, where ETc is the evapotranspiration, Kc is the crop coefficient, and ETo is the evapotranspiration of a reference crop. The commonly accepted reference crop is a large field of grass, where the soil is completely covered and the grass is kept short and well-watered under ideal growing conditions. Several tools exist to model (estimate) the evapotranspiration of a grass field based on meteorological data, such as the FAO Penman-Monteith equation. While this method performs best when extensive and detailed meteorological data are available, it is possible to achieve good estimates on a 10-day or monthly basis when only daily maximum and minimum temperatures are available (Allen et al. 1998, de Carvalho et al. 2013).
Growers estimate the crop coefficient of their field to determine irrigation and water consumption needs. However, there are no simple-to-use, inexpensive, and commercially available methods to estimate crop coefficients in the vineyard with high spatial resolution. Crop coefficients for a variety of crops have been reported for early, mid, and late season (Allen et al. 1998). These values are, however, a rough estimate and actual crop coefficients change throughout the season with plant phenology (Allen and Pereira 2009).
Some efforts have been made toward modeling crop coefficient for various horticultural crops as a function of the ratio of ground cover-to-height, yielding reasonable results of 0.60 > R2 < 0.90 (Grattan et al. 1998), but these methods lack validation for grapes. Efforts to use multispectral satellite images to estimate crop coefficient in vineyards achieved a root mean square error (RMSE) of 0.10 (Allen et al. 2007, Carrasco-Benavides et al. 2012). More recent efforts have achieved similar results (RMSE = 0.098) fusing optical normalized difference vegetation index (NDVI) with synthetic aperture radar (SAR) using Sentinel-2, Sentinel-1, and Landsat-8 satellites (Beeri et al. 2020). Better results (RMSE = 0.062) were achieved using unmanned aerial vehicle (UAV)-based multispectral imaging (Gautam et al. 2021). However, to our knowledge, there have been no research efforts to measure crop coefficient directly using computer vision ground-based sensors, which would offer advantages such as speed and ease of use to farming practices.
Williams and Ayars (2005) demonstrated that crop coefficient in vineyards is a linear function of both leaf area index (R2 = 0.87) and the percentage of shaded area in the vineyard (R2 = 0.95) (as shown in Figure 1A). The shaded area under the vine was measured using a large piece of graph paper that was placed under the vine at high noon, with the shaded grid cells then counted by hand (Figure 1C). The correlation between crop coefficient and shaded area became the basis for the Paso Panel method (see the University of California Cooperative Extension San Luis Obispo County website: https://ucanr.edu/county/county-san-luis-obispo/crop-coefficients-paso-panel), as shown in Figure 1B. A Paso Panel is a device composed of a long, rectangular, lightweight solar panel connected to a current meter. This method assumes a 1-to-1 correlation between the fraction of the solar panel exposed to light and the intensity of current produced (https://ucanr.edu/county/county-san-luis-obispo/crop-coefficients-paso-panel; Gautam et al. 2021). This principle, combined with the linear correlation (Williams and Ayars 2005), yields the following equation to estimate crop coefficient with the Paso Panel:
where Lp is the length of the Paso Panel, Wr is the row spacing, and Is and Ic are the full sun and under canopy current readings, respectively. This assumption is only true when the object blocking direct sunlight to the panel is flush with the surface of the panel. Otherwise, small amounts of diffraction cause significant nonlinearities in the system. It should also be noted that while the relationship between the percentage of shaded area and crop coefficient has been validated (Williams and Ayars 2005), to our knowledge, the Paso Panel method for estimating the percentage of shaded area has not been validated as a stand-alone method. It is, however, commonly used by researchers to measure crop coefficient or monitor canopy size during field trials (Pagay 2016, Stout et al. 2017, Coniberti et al. 2018).
A) Camera view of video data captured. The camera is walked or driven down the row and pointed at the ground under the vine to capture the shaded area under the vine. B) The Paso Panel can be used to estimate the amount of sunlight/shade under the vine. C) An example of how the shaded area could be measured manually using a piece of paper that is laid down beside the vines to enhance the contrast of the shadow.
Despite being faster than the grid paper method, the Paso Panel can still be slow and tedious compared to modern sensing techniques such as computer vision or LiDAR, due to its bulky hardware, manual data notation, and susceptibility to error if the panel is not held perfectly perpendicular to the ground and to the row. As a result, industry has not widely adopted the Paso Panel method to estimate crop coefficient, and its use is primarily limited to research domains.
Because there is no easy-to-use, low-cost, point-based method to estimate crop coefficient in vineyards, we propose a novel approach that uses a smartphone camera to directly measure the percentage of shaded area in the field. It is capable of sampling at 15 to 50× faster than the Paso Panel, depending on the speed of the vehicle or person holding the camera, and does not require specialized equipment. Herein, we describe the method and operation of this approach, compare it to the Paso Panel method, and conclude with a discussion and plans for further work.
Materials and Methods
Paso Panel characterization
The theoretical model of a Paso Panel would suggest that the current output of the solar panel is linearly proportional to the percentage of the solar panel that is directly exposed to the sun. This assumption holds true in cases where the object blocking the sun is completely flush to the surface of the solar panel. In practice, because much of the shade hitting the solar panel is cast by parts of the vine which are not in direct contact with the solar panel, there is room for trace amounts of indirect sunlight to refract through the atmosphere and hit shaded parts of the solar panel. To characterize this behavior, the Paso Panel was tested with different percentages of the solar panel shaded using a sheet of plywood that was placed 0.5 m above the panel to approximate the effect of shoots that shaded the panel without being in direct contact with it. The output current was measured and plotted as a percentage of the current in full sun exposure against the percentage of the panel that was shaded. These data were fit to a polynomial and used to calibrate future Paso Panel readings.
Crop coefficient measurement
This work was conducted at the Cornell University teaching vineyard in Lansing, NY (42°34′N; 76°35′W). Mature Vitis vinifera (cv. Riesling/110R) with vine spacing 1.83 m × 2.74 m, cane-pruned, and vertical shoot-positioned (VSP) according to local practices were used (Wolf 2008). On 17 Aug 2023, corresponding to Eichhorn-Lorenz stages 35/36, Paso Panel and video data were collected for 40 vines separated into 10 panels. Data were collected at 1120, 1220, and 1330 hr, for a total of 30 data points. Because we sought only to estimate the shaded area under the vine, the actual crop coefficient of the vines was not validated and video and Paso Panel measurements were not collected exactly at solar noon. Videos were taken with an iPhone 13, which was held by hand, walked down the row, and pointed toward the ground at the shadow underneath the vine. Given the tedious and time-consuming nature of the graph paper method (originally deployed by Williams and Ayars [2005]), we opted to use the Paso Panel method instead. After each video was taken, Paso Panel data were collected using a flexible 100-w solar panel (192 cm long and 52 cm wide) that was attached to a multimeter through a 100-ohm resistor. Ten to 12 readings were taken between each post, which were averaged to compare against the computer vision method.
Computer vision pipeline
The data processing method was composed of three distinct steps: 1) structure-from-motion (SfM) (Figure 2A and 2B), 2) surface reconstruction (Figure 2C to 2E), and 3) image and surface segmentation (Figure 3) cropping and scaling. The goal of the data processing pipeline was to produce a uniformly sampled reconstruction of the surface of the ground under the vine, segment the reconstruction into shaded/unshaded regions, and crop and scale the reconstruction to real-world measurements, from which the total surface area of shade could be determined.
A) Camera view of shadow under the vine. The black boxes correspond to the camera images plane overlaid on the 3-D view of the point cloud. B) Sparse reconstruction as generated by the OpenSfM pipeline. C) Dense reconstruction generated by OpenMVS, using the sparse reconstruction. D) Mesh generated with OpenMVS, using the dense reconstruction. E) Uniform point sampling of mesh using Open3D’s implementation of Poisson disk sampling. This creates a uniformly sampled point cloud of the structure of the ground, vine trunk, and posts.
A) The original camera view images. B) The output of the segmentation network. For each pixel in the image, a classification distribution vector of length 3 is generated by the softmax output layer of the network, with each value corresponding to a classification probability. This classification probability distribution is visualized using a BGR color scheme of OpenCV: blue, sunlight; green, shade; red, posts. C) The uniformly sampled point cloud projected onto the image plane using the camera intrinsic and extrinsic matrices that are bootstrapped from the structure-from-motion (SfM) pipeline. Each 3-D point is classified according to the corresponding pixel from the image segmentation mask. D) Each point in the 3-D point cloud is visible from multiple camera views throughout the video. For each pixel, the classification distributions from each video frame are averaged together and the resulting classification distributions are visualized as a 3-D point cloud. The RGB color scheme of Open3D is blue, posts; green, shade; red, sun. Note that red and blue colors are swapped due to the RGB color scheme of Open3D.
The video was down-sampled to 20 frames per second (fps) and a point cloud was generated using OpenSfM (https://opensfm.org), an open-source SfM pipeline. The OpenSfM pipeline extracts Hessian affine invariant points (Mikolajczyk and Schmid 2004) and generates histogram of gradients feature descriptors (Lowe 1999). Image features were then paired with feature matches from the 10 nearest images in the video sequence. Matches were found using Fast Library for Approximate Nearest Neighbors (Muja and Lowe 2009) and outliers were removed using random sampling consensus (Fischler and Bolles 1981) with a homography transform using the five-point algorithm (Faugeras and Maybank 1990). The result was a sparse point cloud, with estimations for camera intrinsics and extrinsics for each video frame (Figure 2B).
The sparse reconstruction and camera intrinsics/extrinsics were passed into OpenMVS (https://github.com/cdcseacave/openMVS) for a more thorough surface reconstruction. OpenMVS is an open-source implementation of multiview reconstruction algorithms loosely based on several works (Shen 2013, Jancosek and Pajdla 2014, Barnes et al. 2023). This uses the output of the SfM pipeline to generate a densified point cloud (Figure 2C) which is converted to a mesh surface (Figure 2D). The mesh surface was then sampled using an Open3D (https://www.open3d.org) implementation of a Poisson disk sampling algorithm (Yuksel 2015). Poisson disk sampling attempts to generate a set of sample points that lie on the surface of a mesh, where each point is a uniform distance from its neighboring points. This forms a relatively uniformly sampled point cloud that represents the surface of the ground under the vine (Figure 2E). Video demonstration of this part of the 3D reconstruction is provided (Supplemental Video).
Each image of the video sequence was passed into an image segmentation model which produced a classification probability vector for each pixel in the image. A U-Net convolutional neural network (Ronneberger et al. 2015), implemented using Keras and Tensorflow, was trained to classify images based on three classes: shaded, unshaded, and post. A total of 93 images were segmented by hand using the Pixel Annotation Tool (https://github.com/abreheret/PixelAnnotationTool), which is a watershed algorithm-assisted labeling tool. The data set was divided into 65 training images and 28 validation images. An example of the output of the segmentation network is shown (Figure 3B). For each frame of the video, points generated in the previous step were projected onto the image plane using the camera intrinsic/extrinsic parameters generated from the SfM pipeline. Each point present on the image plane was classified according to the image classification probability vector from the segmentation model (Figure 3C). Any given point on the surface reconstruction was visible from multiple video frames. Accordingly, the classification vectors from each video frame where a given point was present were averaged together and the highest probability class was selected for that point. The result was each point in the point cloud was classified as one of three classes: shaded, exposed to sun, or post (Figure 3D). Video demonstration of this part of the reconstruction is provided (Supplemental Video).
For the last step of the data processing method, the location of the posts was detected in reconstruction. Each row of vines consisted of 10 panels of four vines each, separated by 11 posts. Points classified as posts in the reconstruction were isolated and a K-means algorithm was used to determine the location of each post. The point cloud reconstruction could then be cropped into individual panels using the post locations. The resulting point cloud was then voxelized (Figure 4), and the number of voxels classified as shadow were counted and multiplied by the size of each voxel. Each voxel is a cube with a side length equal to 1/1000th of the total length of the point cloud of the panel. This was done to account for variations in scaling that occur as part of the SfM pipeline. The total shaded area was normalized by the length of the panel and compared to the average Paso Panel reading for each panel of vines.
Data collected over three days, demonstrating the relationship between shaded area and current generated by the photovoltaic panel in the Paso Panel. The theoretical model used in previous work assumed a linear 1-to-1 relationship between the percentage of sunlight and percentage of maximum current. Using the curve generated from this data allows a more accurate estimate of shaded area as a function of current.
Results
A plot of the percentage of shaded area versus percentage of current generated by the Paso Panel is shown (Figure 4). The data were fit to a fourth order polynomial (y = −5e − 6 × 4 + 8e − 4 × 3 − 0.0451 × 2 + 0.3961x + 98.56), as shown in the plot. This equation was used to determine the amount of shade as measured by the Paso Panel by substituting the fraction of Is and Ic in the Paso Panel equation with values calibrated by the polynomial in Figure 4.
An example of a reconstruction of the area under the vine including voxelation is shown (Figure 5). The UNet image segmentation network was trained to achieve a validation intersection over union of 0.91. The total number of voxels classified as shade were counted and plotted against the average Paso Panel estimations. Paso Panel current readings for each panel were averaged together, divided by the full sun current reading, and converted to a percentage of area shaded measurement using the calibration curve from Figure 4. The results comparing these two methods are shown, along with a regression line with correlation R2 = 0.68 (Figure 6).
An example of a voxelized segmented panel of vines. The posts are detected using a K-means clustering algorithm and panels are segmented along the posts. The total number of voxels corresponding to shade are counted and divided by the length of the panel. The color scheme is blue, posts; green, shade; red, sun.
Plot of Paso Panel readings versus computer vision-estimated shaded area. The total shaded area as estimated by the computer vision algorithm was normalized by the length of the panel. This allows direct comparison to the Paso Panel readings.
Discussion
As technological advances in variable rate irrigation systems are made and such systems become more ubiquitous in vineyards, crop coefficient estimation methods such as that proposed in this paper are needed to measure the geospatial variance of water usage throughout the vineyard. Because modern smartphones are GPS enabled, mapping geolocation to crop coefficient measurements can be easily integrated into decision support systems for automated variable-rate irrigation. Recent work in GPS sensing has demonstrated the capacity for network real-time kinematic positioning on smartphones, enabling submeter spatial accuracy without additional sensors (Dabove and Di Pietra 2019).
This method provides a lower-cost and easier-to-use alternative to multispectral aerial imaging (Gautam et al. 2021). Aerial-based methods such as those described by Gautam et al. (2021) provide reliable, individual vine-scale crop coefficient measurements. However, UAV operation may prove challenging to farmers who do not want to invest in or learn how to use expensive hardware such as drones or airplanes. Our method also provides greater spatial and temporal resolution than a method using Landsat 8 images, which have a spatial resolution of 30 m and a temporal resolution of 16 days (Mondal et al. 2022). Our method is on par with remote-sensing NDVI/SAR-based methods (Beeri et al. 2020). If individual vine measurements are needed, or cloud coverage does not permit the use of remote sensing, UAV or ground-based methods are necessary.
Our method is significantly faster and easier than the Paso Panel method. Collecting data for a 10-panel row took 12 to 14 min with two people: one operating the Paso Panel and the other recording the values. In contrast, collecting a video of a single row took ~1 min per row, the limiting factor being the operator’s walking speed. The videos were recorded at 60 fps and down-sampled to 10 fps for processing. It follows, theoretically, that videos could be recorded without down-sampling at a speed of 10 sec per row and still obtain the same results, suggesting the possibility of using a vehicle such as a tractor or ATV for rapid data collection.
The entire computer vision pipeline took ~30 min to process a single row on an Intel Xeon workstation with a GTX 1080. While this is not ideal for widespread deployment, the algorithm is currently unoptimized. For example, given the overlap in images, image segmentation does not need to be performed on every video frame. The frequency of image segmentation can be determined dynamically from the SfM output. In addition, OpenSfM performs global bundle adjustment for each image in the video sequence. Given that this pipeline does not need to perform any sort of loop closure, local bundle adjustment should suffice and, in conjunction with GPU acceleration, should offer a 2 to 5× speed increase (Gopinath 2023). Lastly, a variety of other open-source SfM pipelines are available and will be tested before deploying the system. These include ColeMap (https://colmap.github.io), OpenMGV (https://github.com/openMVG/openMVG), and Meshroom (https://alicevision.org). Moreover, recent advancements in the UNet architecture have demonstrated increased efficiency and faster inference times (Wang et al. 2019). While the data generated from this pipeline can be quite large (~10 gb per row), in practice, the input and output data structures of each step of the pipeline can be dynamically allocated and deallocated. This means that the maximum amount of hard drive space needed by the pipeline would not exceed 5 gb. These factors and current trends in engineering methods demonstrate the potential for increased computational efficiency for each component of this computer vision pipeline. More work is needed to integrate these newer engineering methods into this pipeline. Moreover, we are working toward improving the validation of this computer vision method. One possibility is to borrow from another study (Williams and Ayars 2005) and use a large sheet of grid paper on the ground. This would remove any potential errors introduced by the Paso Panel.
The theoretical upper limit of the performance of this method is determined by Williams and Ayars’ work (2005) on the use of shaded area to predict crop coefficient. Their method achieved R2 of 0.95, with crop coefficients ranging from 0.1 to 1.3. This denotes the potential upper limit of the proposed computer vision method as improvements are made. Despite the potential of the proposed method to provide easy and reliable crop coefficient estimates after the necessary engineering work, there remain shortcomings to this approach. The proposed method is a point-based method and requires traversing the entire region to collect large-scale data. While regular pesticide and fungicide applications make large scale data collection feasible, more work is needed to automate video processing before this approach is commercially viable. In the near term, this method will be limited to small vineyards or researchers looking to take point measurements, while larger vineyards should rely on UAV or satellite-based NDVI methods.
The relationship between crop coefficient and shaded area under the vine is due to the specific canopy structure of VSP vines (Williams and Ayars 2005). The current authors are investigating ways to leverage the proposed method and the “wall-like” structure of VSP vines to estimated exposed leaf area using a slightly modified version of the proposed computer vision pipeline. One way to do this is to measure the surface area of the shadow produced later or earlier in the day, when the sun is lower in the sky, and use geometric models to estimate exposed leaf area of the canopy casting the shadow.
Conclusion
This work demonstrated the viability of using smartphone cameras to estimate crop coefficient in vineyards. Given the simplicity of data collection needed to deploy this method, we anticipate that farmers could use this method with very little training. Modifications to the image segmentation model training set must be made for applications in vineyards using different cover crops (e.g., vegetation, bare soil, mulch, etc.). We are working toward making the software publicly available to grapegrowers through partnership with the Efficient Vineyard Project, based out of the Cornell Lake Erie Research and Extension Laboratory, which is leveraging the existing MyEV tool (my.efficientvineyard.com).
Correlation between computer vision estimates and Paso Panel estimates depends on the accuracy of both methods. To our knowledge, there is no literature that conducts rigorous independent verification of the Paso Panel method as it relates to the accuracy of measuring shaded area in vineyards. For this reason, when correlating the Paso Panel readings with computer vision-generated measurements, it is impossible to disambiguate how much each method contributes to the measurement noise. Nevertheless, the Paso Panel is a commonly used tool to measure crop coefficient and monitor canopy size, despite the lack of thorough validation. Given the fidelity of the computer vision shadow reconstruction and the correlations with the Paso Panel data, this proposed method likely works as well as the Paso Panel. Moreover, this method is significantly simpler and faster to deploy than the Paso Panel and requires no extra hardware besides a smartphone camera. Our proposed method offers greater spatial resolution than the Paso Panel, simply through ease of measurement.
Supplemental Data
The following supplemental materials are available for this article in the Supplemental tab above:
Supplemental Video Video demonstration of the CV pipeline illustrated in Figures 2, 3, and 5.
Data Availability
The data underlying this study are available on request from the corresponding author.
Footnotes
We thank Michael Lovier and Will Melancon for technical assistance in the vineyard, as well as Mark Battany, UCCE Water Management and Biometeorology Advisor for San Luis Obispo and Santa Barbara Counties, and Karl T. Lund, UCCE Viticulture Advisor for Madera, Merced, and Mariposa Counties, for their expertise with the Paso Panel. Funding for this research was provided by NSF grant #1837367 and a NIFA grant #1014705.
Jaramillo J, Vanden Heuvel J and Petersen K. 2025. Toward estimating the crop coefficient of vineyards using a smartphone camera. Am J Enol Vitic 76:0760020. DOI: 10.5344/ajev.2025.24068
By downloading and/or receiving this article, you agree to the Disclaimer of Warranties and Liability. If you do not agree to the Disclaimers, do not download and/or accept this article.
- Received October 2024.
- Accepted May 2025.
- Published online August 2025
This is an open access article distributed under the CC BY 4.0 license.












