  Fisher vector (FV) has become a popular image representation. One notable
underlying assumption of the FV framework is that local descriptors are well
decorrelated within each cluster so that the covariance matrix for each
Gaussian can be simplified to be diagonal. Though the FV usually relies on the
Principal Component Analysis (PCA) to decorrelate local features, the PCA is
applied to the entire training data and hence it only diagonalizes the
\textit{universal} covariance matrix, rather than those w.r.t. the local
components. As a result, the local decorrelation assumption is usually not
supported in practice.
  To relax this assumption, this paper proposes a completed model of the Fisher
vector, which is termed as the Completed Fisher vector (CFV). The CFV is a more
general framework of the FV, since it encodes not only the variances but also
the correlations of the whitened local descriptors. The CFV thus leads to
improved discriminative power. We take the task of material categorization as
an example and experimentally show that: 1) the CFV outperforms the FV under
all parameter settings; 2) the CFV is robust to the changes in the number of
components in the mixture; 3) even with a relatively small visual vocabulary
the CFV still works well on two challenging datasets.
