  We study a system of $n$ Abelian vector fields coupled to $\frac 12 n(n+1)$
complex scalars parametrising the Hermitian symmetric space $\mathsf{Sp}(2n,
{\mathbb R})/ \mathsf{U}(n)$. This model is Weyl invariant and possesses the
maximal non-compact duality group $\mathsf{Sp}(2n, {\mathbb R})$. Although both
symmetries are anomalous in the quantum theory, they should be respected by the
logarithmic divergent term (the ``induced action'') of the effective action
obtained by integrating out the vector fields. We compute this induced action
and demonstrate its Weyl and $\mathsf{Sp}(2n, {\mathbb R})$ invariance. The
resulting conformal higher-derivative $\sigma$-model on $\mathsf{Sp}(2n,
{\mathbb R})/ \mathsf{U}(n)$ is generalised to the cases where the fields take
their values in (i) an arbitrary K\"ahler space; and (ii) an arbitrary
Riemannian manifold. In both cases, the $\sigma$-model Lagrangian generates a
Weyl anomaly satisfying the Wess-Zumino consistency condition.
