  We propose a coronagraphic system with fourth-order null for off-axis
segmented telescopes, which is sufficiently insensitive to the telescope
pointing errors and finite angular diameter of the host star to enable
high-contrast imaging of potentially habitable planets. The inner working angle
of the coronagraphic system is close to $1\lambda/D$, and there is no outer
limit. The proposed coronagraphic system is made up of a new focal plane mask
and an optimized Lyot stop with the second-order null. The new focal plane mask
is an extension of the band-limited masks with a phase modulation. We construct
a coronagraphic system with fourth-order null by placing two of the new
coronagraph systems in succession to be orthogonal to each other. The proposed
system is limited to narrow-band usage. The characteristics of the proposed
coronagraph system are derived analytically, which includes: (1)the leak of
stellar lights due to finite stellar diameter and pointing jitter of a
telescope, and (2)the peak throughput. We achieve the performance simulations
of this coronagraphic system based on these analytical expressions, considering
a monochromatic light of 0.75$\mathrm{\mu}$m and off-axis primary mirror with a
diameter of 8.5m. Thanks to the wide working area of the mask, the result shows
that terrestrial planets orbiting K and G-dwarfs can be detected under the
condition that the telescope pointing jitter is less than
$0.01\lambda/D\approx240$as. The proposed coronagraphic system is promising for
detection of potentially habitable planets with future space off-axis
hexagonally segmented telescopes.
