  Due to labor shortage and rising labor cost for the apple industry, there is
an urgent need for the development of robotic systems to efficiently and
autonomously harvest apples. In this paper, we present a system overview and
algorithm design of our recently developed robotic apple harvester prototype.
Our robotic system is enabled by the close integration of several core modules,
including visual perception, planning, and control. This paper covers the main
methods and advancements in deep learning-based multi-view fruit detection and
localization, unified picking and dropping planning, and dexterous manipulation
control. Indoor and field experiments were conducted to evaluate the
performance of the developed system, which achieved an average picking rate of
3.6 seconds per apple. This is a significant improvement over other reported
apple harvesting robots with a picking rate in the range of 7-10 seconds per
apple. The current prototype shows promising performance towards further
development of efficient and automated apple harvesting technology. Finally,
limitations of the current system and future work are discussed.
