  Agricultural applications such as yield prediction, precision agriculture and
automated harvesting need systems able to infer the crop state from low-cost
sensing devices. Proximal sensing using affordable cameras combined with
computer vision has seen a promising alternative, strengthened after the advent
of convolutional neural networks (CNNs) as an alternative for challenging
pattern recognition problems in natural images. Considering fruit growing
monitoring and automation, a fundamental problem is the detection, segmentation
and counting of individual fruits in orchards. Here we show that for wine
grapes, a crop presenting large variability in shape, color, size and
compactness, grape clusters can be successfully detected, segmented and tracked
using state-of-the-art CNNs. In a test set containing 408 grape clusters from
images taken on a trellis-system based vineyard, we have reached an F 1 -score
up to 0.91 for instance segmentation, a fine separation of each cluster from
other structures in the image that allows a more accurate assessment of fruit
size and shape. We have also shown as clusters can be identified and tracked
along video sequences recording orchard rows. We also present a public dataset
containing grape clusters properly annotated in 300 images and a novel
annotation methodology for segmentation of complex objects in natural images.
The presented pipeline for annotation, training, evaluation and tracking of
agricultural patterns in images can be replicated for different crops and
production systems. It can be employed in the development of sensing components
for several agricultural and environmental applications.
