  We construct the Baxter operator $\boldsymbol{ \texttt{Q} }(\lambda)$ for the
$q$-Toda chain and the Toda$_2$ chain (the Toda chain in the second Hamiltonian
structure). Our construction builds on the relation between the Baxter operator
and B\"acklund transformations that were unravelled in {\cite{GaPa92}}. We
construct a number of quantum intertwiners ensuring the commutativity of
$\boldsymbol{ \texttt{Q} }(\lambda)$ with the transfer matrix of the models and
the one of $\boldsymbol{ \texttt{Q} }$'s between each other. Most importantly,
$\boldsymbol{ \texttt{Q} }(\lambda)$ is modular invariant in the sense of
Faddeev. We derive the Baxter equation for the eigenvalues $q(\lambda)$ of
$\boldsymbol{ \texttt{Q} }(\lambda)$ and show that these are entire functions
of $\lambda$. This last property will ultimately lead to the quantisation of
the spectrum for the considered Toda chains, in a subsequent publication.
