  It is known that the model based on U(1)$_{L_\mu-L_\tau}$ gauge symmetry can
explain not only the discrepancy between the measured value of muon $g-2$ and
the theoretical prediction, but also the structure of the neutrino mass and
mixings. We revisit the analysis of the mass matrix structure in the minimal
U(1)$_{L_\mu-L_\tau}$ models based on the latest experimental result, where the
minimal stands for the symmetry breaking caused only by a single scalar field.
We find that the model called type ${\bf 2}_{+1}$, where an SU(2)$_L$ doublet
scalar $\Phi_{+1}$ with the U(1)$_{L_\mu - L_\tau}$ charge $+1$ and the
hypercharge $+1/2$, predicts the $\bf B_3$ texture and is marginally acceptable
under the current neutrino oscillation data and cosmological observation. When
the U(1)$_{L_\mu - L_\tau}$ gauge symmetry is broken by the vacuum expectation
value of the standard model non-singlet representation such as $\Phi_{+1}$,
there are additional contributions to the flavor-changing meson decay process
and atomic parity violation via the $Z-Z'$ mixing. We newly evaluate the
model-dependent constraints on the model and conclude that the type ${\bf
2}_{+1}$ model is robustly ruled out. The model is extended to have an
additional vacuum expectation value of a standard model singlet scalar in order
to avoid the stringent constraint from the flavor-changing meson decay.
Finally, we find the allowed range of the ratio of these vacuum expectation
values.
