  Polarization of starlight induced by dust grains aligned with the magnetic
field (hereafter B-field) is widely used to measure the two-dimensional
B-fields projected onto the plane-of-sky. Here, we introduce a new method to
infer three-dimensional B-fields using starlight polarization. The B-field's
inclination angle or line-of-sight (LOS) component of B-fields is constrained
by the starlight polarization efficiency from observations and the alignment
degree provided by the magnetically enhanced radiative torque (MRAT) alignment
theory. We first perform synthetic observation of starlight polarization of
magnetohydrodynamic (MHD) simulations of a filamentary cloud. We then test the
new technique with our updated POLARIS code and find that the B-field's
inclination angles can be precisely determined by the starlight polarization
efficiency from synthetic observations. Regardless of grain magnetic
properties, the technique can provide an accurate constraint on B-field's
inclination angles in low-density regions $N_{\rm H} < 5\times
10^{21}\,\cm^{-2}$ (or visual extinction $A_{V}< 3$) using optical
polarization, whereas the technique can infer further to high-density regions
$N_{\rm H} \sim 5 \times 10^{22}\,\cm^{-2}$ (or $A_{V}\sim 30$) using
near-infrared polarization. Our new technique opens the full potential of
tracing 3D B-fields and constraining dust properties and grain alignment
physics on multiple scales of the diffuse interstellar medium and star-forming
regions using multi-wavelength starlight polarization observations.
