  The cosmological dispersion measure (DM) as a function of redshift, derived
from localized fast radio bursts (FRBs), has been used as a tool for
constraining the cosmic ionized fraction and cosmological parameters. For these
purposes, the DM in a homogeneous cosmological model has typically been used,
neglecting the inhomogeneity of matter distribution. In this study, we derive a
bias in the ensemble average of the DM over many FRBs owing to gravitational
lensing by the inhomogeneous matter distribution based on cosmological
perturbation theory. We demonstrate that the ensemble average is $0.4 \,
\%$--$1\,\%$ smaller than the DM in the corresponding homogeneous model for a
source redshift of $z_{\rm s}=1$, according to recent cosmological hydrodynamic
simulations of IllustrisTNG and BAHAMAS. This reduction occurs because light
rays from FRBs tend to avoid high-density regions owing to lensing deflection.
We also discuss another selection effect, magnification bias, where demagnified
FRBs with low DMs, fainter than the detection threshold, are excluded from the
observed sample, leading to a selective observation of magnified FRBs with high
DMs. Lensing bias, including magnification bias, must be considered to achieve
percent level accuracy in the DM--redshift relation.
