  Aiming at the difficulty of extracting image features and estimating the
Jacobian matrix in image based visual servo, this paper proposes an image based
visual servo approach with deep learning. With the powerful learning
capabilities of convolutional neural networks(CNN), autonomous learning to
extract features from images and fitting the nonlinear relationships from image
space to task space is achieved, which can greatly facilitate the image based
visual servo procedure. Based on the above ideas a two-stream network based on
convolutional neural network is designed and the corresponding control scheme
is proposed to realize the four degrees of freedom visual servo of the robot
manipulator. Collecting images of observed target under different pose
parameters of the manipulator as training samples for CNN, the trained network
can be used to estimate the nonlinear relationship from 2D image space to 3D
Cartesian space. The two-stream network takes the current image and the
desirable image as inputs and makes them equal to guide the manipulator to the
desirable pose. The effectiveness of the approach is verified with experimental
results.
