  Possible violations of Lorentz invariance (LIV) can produce vacuum
birefringence, which results in a frequency-dependent rotation of the
polarization plane of linearly polarized light from distant sources. In this
paper, we try to search for a frequency-dependent change of the linear
polarization angle arising from vacuum birefringence in the spectropolarimetric
data of astrophysical sources. We collect five blazars with multiwavelength
polarization measurements in different optical bands $(UBVRI)$. Taking into
account the observed polarization angle contributions from both the intrinsic
polarization angle and the rotation angle induced by LIV, and assuming that the
intrinsic polarization angle is an unknown constant, we obtain new constraints
on LIV by directly fitting the multiwavelength polarimetric data of the five
blazars. Here we show that the birefringence parameter $\eta$ quantifying the
broken degree of Lorentz invariance is limited to be in the range of
$-8.91\times 10^{-7}$ < $\eta$ < $2.93\times10^{-5}$ at the $2\sigma$
confidence level, which is as good as or represents one order of magnitude
improvement over the results previously obtained from ultraviolet/optical
polarization observations. Much stronger limits can be obtained by future
multiwavelength observations in the gamma-ray energy band.
