  Field robotic harvesting is a promising technique in recent development of
agricultural industry. It is vital for robots to recognise and localise fruits
before the harvesting in natural orchards. However, the workspace of harvesting
robots in orchards is complex: many fruits are occluded by branches and leaves.
It is important to estimate a proper grasping pose for each fruit before
performing the manipulation. In this study, a geometry-aware network, A3N, is
proposed to perform end-to-end instance segmentation and grasping estimation
using both color and geometry sensory data from a RGB-D camera. Besides,
workspace geometry modelling is applied to assist the robotic manipulation.
Moreover, we implement a global-to-local scanning strategy, which enables
robots to accurately recognise and retrieve fruits in field environments with
two consumer-level RGB-D cameras. We also evaluate the accuracy and robustness
of proposed network comprehensively in experiments. The experimental results
show that A3N achieves 0.873 on instance segmentation accuracy, with an average
computation time of 35 ms. The average accuracy of grasping estimation is 0.61
cm and 4.8$^{\circ}$ in centre and orientation, respectively. Overall, the
robotic system that utilizes the global-to-local scanning and A3N, achieves
success rate of harvesting ranging from 70\% - 85\% in field harvesting
experiments.
