  Quantifying the parameters and corresponding uncertainties of hundreds of
strongly lensed quasar systems holds the key to resolving one of the most
important scientific questions: the Hubble constant ($H_{0}$) tension. The
commonly used Markov chain Monte Carlo (MCMC) method has been too
time-consuming to achieve this goal, yet recent work has shown that convolution
neural networks (CNNs) can be an alternative with seven orders of magnitude
improvement in speed. With 31,200 simulated strongly lensed quasar images, we
explore the usage of Vision Transformer (ViT) for simulated strong
gravitational lensing for the first time. We show that ViT could reach
competitive results compared with CNNs, and is specifically good at some
lensing parameters, including the most important mass-related parameters such
as the center of lens $\theta_{1}$ and $\theta_{2}$, the ellipticities $e_1$
and $e_2$, and the radial power-law slope $\gamma'$. With this promising
preliminary result, we believe the ViT (or attention-based) network
architecture can be an important tool for strong lensing science for the next
generation of surveys. The open source of our code and data is in
\url{https://github.com/kuanweih/strong_lensing_vit_resnet}.
