  Tight limits on the photon mass have been set through analyzing the arrival
time differences of photons with different frequencies originating from the
same astrophysical source. However, all these constraints have relied on using
the first-order Taylor expansion of the dispersion due to a nonzero photon
mass. In this work, we present an analysis of the nonzero photon mass
dispersion with the second-order derivative of Taylor series. If the arrival
time delay corrected for all known effects (including the first-order delay
time due to the plasma and photon mass effects) is assumed to be dominated by
the second-order term of the nonzero photon mass dispersion, a conservative
upper limit on the photon mass can be estimated. Here we show that the
dedispersed pulses with the second-order time delays from the Crab pulsar and
the fast radio burst FRB 180916B pose strict limits on the photon mass, i.e.,
$m_{\gamma,2} \leq5.7\times10^{-46}\;{\rm kg}\simeq3.2\times10^{-10}\; {\rm
eV}/c^{2}$ and $m_{\gamma,2} \leq6.0\times10^{-47}\;{\rm
kg}\simeq3.4\times10^{-11}\; {\rm eV}/c^{2}$, respectively. This is the first
time to study the possible second-order photon mass effect.
