  In order to explore robotic grasping in unstructured and dynamic
environments, this work addresses the visual perception phase involved in the
task. This phase involves the processing of visual data to obtain the location
of the object to be grasped, its pose and the points at which the robot`s
grippers must make contact to ensure a stable grasp. For this, the Cornell
Grasping dataset is used to train a convolutional neural network that, having
an image of the robot`s workspace, with a certain object, is able to predict a
grasp rectangle that symbolizes the position, orientation and opening of the
robot`s grippers before its closing. In addition to this network, which runs in
real-time, another one is designed to deal with situations in which the object
moves in the environment. Therefore, the second network is trained to perform a
visual servo control, ensuring that the object remains in the robot`s field of
view. This network predicts the proportional values of the linear and angular
velocities that the camera must have so that the object is always in the image
processed by the grasp network. The dataset used for training was automatically
generated by a Kinova Gen3 manipulator. The robot is also used to evaluate the
applicability in real-time and obtain practical results from the designed
algorithms. Moreover, the offline results obtained through validation sets are
also analyzed and discussed regarding their efficiency and processing speed.
The developed controller was able to achieve a millimeter accuracy in the final
position considering a target object seen for the first time. To the best of
our knowledge, we have not found in the literature other works that achieve
such precision with a controller learned from scratch. Thus, this work presents
a new system for autonomous robotic manipulation with high processing speed and
the ability to generalize to several different objects.
