  We study the statistical nature of primordial fluctuations from an
anisotropic inflation which is realized by a vector field coupled to an
inflaton. We find a suitable gauge, which we call the canonical gauge, for
anisotropic inflation by generalizing the flat slicing gauge in conventional
isotropic inflation. Using the canonical gauge, we reveal the structure of the
couplings between curvature perturbations, vector waves, and gravitational
waves. We identify two sources of anisotropy, i.e. the anisotropy due to the
anisotropic expansion of the universe and that due to the anisotropic couplings
among variables. It turns out that the latter effect is dominant. Since the
coupling between the curvature perturbations and vector waves is the strongest
one, the statistical anisotropy in the curvature perturbations is larger than
that in gravitational waves. We find the cross correlation between the
curvature perturbations and gravitational waves which never occurs in
conventional inflation. We also find the linear polarization of gravitational
waves. Finally, we discuss cosmological implication of our results.
