  Classical pixel-based Visual Servoing (VS) approaches offer high accuracy but
suffer from a limited convergence area due to optimization nonlinearity. Modern
deep learning-based VS methods overcome traditional vision issues but lack
scalability, requiring training on limited scenes. This paper proposes a hybrid
VS strategy utilizing Deep Reinforcement Learning (DRL) and optimal control to
enhance both convergence area and scalability. The DRL component of our
approach separately handles representation and policy learning to enhance
scalability, generalizability, learning efficiency and ease domain adaptation.
Moreover, the optimal control part ensures high end-point accuracy. Our method
showcases remarkable achievements in terms of high convergence rates and
minimal end-positioning errors using a 7-DOF manipulator. Importantly, it
exhibits scalability across more than 1000 distinct scenes. Furthermore, we
demonstrate its capacity for generalization to previously unseen datasets.
Lastly, we illustrate the real-world applicability of our approach,
highlighting its adaptability through single-shot domain transfer learning in
environments with noise and occlusions. Real-robot experiments can be found at
\url{https://sites.google.com/view/vsls}.
