---
abstract: 'Developing personal robots that can perform a diverse range of manipulation tasks in unstructured environments necessitates solving several challenges for robotic grasping systems. We take a step towards this broader goal by presenting the first RL-based system, to our knowledge, for a mobile manipulator that can (a) achieve targeted grasping generalizing to unseen target objects, (b) learn complex grasping strategies for cluttered scenes with occluded objects, and (c) perform active vision through its movable wrist camera to better locate objects. The system is informed of the desired target object in the form of a single, arbitrary-pose RGB image of that object, enabling the system to generalize to unseen objects without retraining. To achieve such a system, we combine several advances in deep reinforcement learning and present a large-scale distributed training system using synchronous SGD that seamlessly scales to multi-node, multi-GPU infrastructure to make rapid prototyping easier. We train and evaluate our system in a simulated environment, identify key components for improving performance, analyze its behaviors, and transfer to a real-world setup.'
author:
- |
    Yasuhiro Fujita, Kota Uenishi, Avinash Ummadisingu, Prabhat Nagarajan,\
    Shimpei Masuda, and Mario Ynocente Castro [^1]
bibliography:
- 'IEEEabrv.bib'
- 'citations.bib'
title: '**Distributed Reinforcement Learning of Targeted Grasping with Active Vision for Mobile Manipulators** '
---

Introduction {#sec:intro}
============

Background {#sec:background}
==========

Problem Description and Formulation {#sec:formulation}
===================================

System {#sec:system}
======

Experiments {#sec:experiments}
===========

Related Work {#sec:relatedwork}
============

Conclusion {#sec:conclusion}
==========

We have presented a distributed deep reinforcement learning system for targeted grasping with active vision that can grasp unseen objects in dense clutter. We have shown that some of our proposed extensions and the large-scale distributed training are key to learning efficiently on this challenging task.

While we also have shown a proof-of-concept demonstration of real-world transfer, simulation-to-real (sim2real) transfer of a vision-based manipulation system is a difficult challenge in itself. We expect our system can benefit from recent advances in sim2real transfer to improve its performance in the real-world setup.

Acknowledgment {#acknowledgment .unnumbered}
==============

We thank Koichi Ikeda, Kunihiro Iwamoto, and Takashi Yamamoto from Toyota Motor Corporation for their support of the HSR robots, and Kentaro Imajo for his help in designing and implementing the distributed training software.

[^1]: All authors are with Preferred Networks, Inc., Tokyo, Japan. [@preferred.jp]{}
