HEp-2 Cell Classification using Multi-Resolution Texture and Discriminative Shape Features
Xianbiao Qi
Guoying Zhao
~\IEEEmembership{Senior Member
~IEEE
....lastname \thanks{...
Indirect Immunofluorescence (IIF) HEp-2 cell image is an effective evidence for diagnosis of autoimmune diseases. Recently computer-aided diagnosis of autoimmune diseases by IIF HEp-2 cell classification has attracted great attention. However the HEp-2 cell classification task is quite challenging due to large intra-class variation and small between-class variation. In this paper we propose an effective and efficient approach for the automatic classification of IIF HEp-2 cell image by fusing multi-resolution texture information and richer shape information. To be specific, we propose to: a) capture the multi-resolution texture information by a novel Pairwise Rotation Invariant Co-occurrence of Local Gabor Binary Pattern (PRICoLGBP) descriptor, b) depict the richer shape information by using an Improved Fisher Vector (IFV) model with RootSIFT features which are sampled from large image patches in multiple scales, and c) combine them properly. We evaluate systematically the proposed approach on the IEEE International Conference on Pattern Recognition (ICPR) 2012, IEEE International Conference on Image Processing (ICIP) 2013 and ICPR 2014 contest data sets. The experimental results for the proposed methods significantly outperform the winners of ICPR 2012 and ICIP 2013 contest, and achieve comparable performance with the winner of the newly released ICPR 2014 contest. % % %This paper proposes an effective method for automatic classification of Indirect Immunofluorescence (IIF) HEp-2 cell images via using discriminative %texture and shape features. \textcolor{red{Our contributions to HEp-2 cell classification are three folds: (1) we propose a novel Pairwise Rotation Invariant %Co-occurrence of Local Gabor Binary Pattern (PRICoLGBP) feature to represent discriminative texture information. Compared to its counterpart (PRICoLBP), %the proposed PRICoLGBP can capture multi-resolution texture information more effectively. %(2) we propose an effective method to depict discriminative shape information. We fully investigate three key steps in this framework, including patch sampling, feature description and encoding method. We propose to sample large patches that covers about $\frac{1{3$ area of the whole image in multiple Gaussian scales. Then, we use RootSIFT feature to depict the sampled patches. Finally, motivated by the success of recent %computer vision technology, we propose to use Improved Fisher Vector (IFV) to encode the features. The IFV can preserve the strong discriminative power of the used features. \textcolor{red{(3) The proposed discriminative %texture and shape features prove to be complementary in the experiments. % %We have evaluated the proposed methods on IEEE International Conference on Pattern Recognition (ICPR) 2012, IEEE International Conference on Image Processing (ICIP) 2013 and \textcolor{red{ICPR 2014 contest data sets. The experimental results for %the proposed methods significantly outperform the winners of %ICPR 2012 and ICIP 2013 contest, and achieve comparable performance with the winner of the newly released ICPR 2014 contest. %Specifically, on ICPR %contest, our methods improves the ICPR winner by about 7\%, and %on ICIP 2013 contest \textcolor{red{data set, our method improves the accuracy of the winner's method %by 4.1\%. It should be noted that the difference between the %$1$-st and $2$-nd places in ICIP 2013 \textcolor{red{contest is only about 0.1\%. %In this paper, we firstly introduce an effective HEp-2 cell classification system that ranks the first in HEp-2 Cells Classification contest in the international conference of image processing 2013 (ICIP 2013). %Then, we further introduce some new methods for HEp-2 cell classification motivated by some new development of the computer vision area. %indirect immunofluorescence (IIF) image analysis
algorithm.tex




%\begin{multicols}{2}
\begin{figure}[t]
\begin{center}
 \includegraphics[width=1.0\linewidth]{figures/ulbp.eps}
\end{center}
   \caption{All 58 uniform LBP patterns are divided into 9 groups. In each group, any one pattern can rotate to any another pattern. The patterns in the same group has the same number of ``1'' and ``0''.}
\label{fig:ulbp}
\end{figure}
%\end{multicols}
\section{A Brief Review of LBP}
\label{sec:LBP}
LBP is an effective texture descriptor that depicts the local structures of natural images, such as edge, contour, flat region and so on. For each pixel in an image, the LBP pattern can be computed by comparing its pixel value with the values of surrounding neighbors:
%\begin{equation}
%    LBP_{P, R} = \sum_{p=0}^{P-1} s(v_p-v_c)2^p, \ \ \ \ \
%    s(x) = \begin{cases}
%    1, &  x \ge 0 \\
%    0, &  x < 0, \\
%    \end{cases}
%\end{equation}
%
%\begin{equation}
%\mathcal{P}_{c,i}(s) = \sum_{p=0}^{P-1} s(v_p-v_c)2^p, \ \ \ \ \
%s(x) = \begin{cases}
%1, & x \ge 0 \\
%0, & x < 0, \\
%\end{cases}
%\end{equation}


\begin{equation}
LBP(\vec{s}) = \sum_{k=0}^{n-1} \phi(v_k-v_c)2^k, \ \ \ \ \
\phi(x) = \begin{cases} 1, & x \ge 0 \\
0, & x < 0, \\
\end{cases}
\end{equation}
where $\vec{s} = [n, r]$, $n$ is the number of neighbors and $r$ is the radius of the neighbors. $v_c$ is the gray value of the central pixel, and $v_k$ is the value of its $k$-th neighbor.






In \cite{ojala2002multiresolution}, Ojala et al. observed that these patterns with very few spatial transitions described the fundamental properties of the image, and they called these patterns as ``uniform patterns''. The number of spatial transitions can be calculated as follows:
\begin{equation}
    \Phi(LBP(\vec{s})) = \sum_{k=1}^{n} |\phi(v_k-v_c) - \phi(v_{k-1}-v_c)|,
\end{equation}
where $v_n$ equals to $v_0$. The uniform patterns are defined as these patterns with $\Phi(LBP(s)) \le 2$. For instance, LBP patterns ``00000000'' and ``00001110'' are the uniform patterns, while ``00100100'' and ``01001110'' are non-uniform patterns.

The uniform LBP ($LBP^{U}$) depends on the start point of the binary sequence. Defined on different start points, $LBP^{U}$ will have different uniform patterns. For instance, the binary sequence ``00001000'' defined on 1$st$ position or 2$nd$ position, their corresponding uniform LBP patterns are different. Here, we denote {{$\bf{LBP^{U}(s, i)}$}} as the uniform LBP pattern on the scale $s$ with $i$ as the start point of the binary sequence, where $0 \le i \le n-1$.

%The image rotation usually changes the LBP value.

To achieve great robustness to image rotation, Ojala et al. also introduced the concept of rotation invariant LBP ({$\bf{LBP^{RI}}$}) and rotation invariant uniform LBP ({$\bf{LBP^{RIU}}$}), where $LBP^{RIU}$ is popularly used for texture classification. The $LBP^{RIU}$ can be defined as:
\begin{equation}
\label{eq:rui}
    LBP^{RIU}(\vec{s}) = \begin{cases}
    \sum_{k=0}^{n-1} \phi(v_k-v_c), &  \Phi(LBP(\vec{s})) \le 2 \\
    n+1, &  otherwise, \\
    \end{cases}
\end{equation}


\begin{multicols}{2}
\begin{figure*}[t]
\begin{center}
 \includegraphics[width=1.0\linewidth]{figures/groupCoLBP.eps}
\end{center}
   \caption{ An illustration of two groups of multi-scale co-occurrence LBPs. Each row denotes one group. In each group, with image rotating, one co-pattern will turn into another one in the same group.}
\label{fig:RotationInvariantCoLBP}
\end{figure*}
\end{multicols}


For the number of neighbors $n=8$, LBP has $2^8=256$ patterns, in which there are 58 uniform patterns and 198 non-uniform patterns. Usually, all 198 non-uniform patterns are regarded as the same pattern. Thus, in practice, $LBP^{U}(8, 1)$ has 59 patterns as shown in Fig. \ref{fig:ulbp}.
According to Eq. \ref{eq:rui}, the $LBP^{RIU}(8, 1)$ has 10 patterns.

%
%\begin{multicols}{2}
%\begin{figure*}[t]
%\begin{center}
% \includegraphics[width=0.85\linewidth]{figures/rulbp.eps}
%\end{center}
%   \caption{A group of LBPs. }
%\label{fig:rlbp}
%\end{figure*}
%\end{multicols}

Multi-scale strategy is widely used in most texture relevant tasks, and it usually boost the classification performance. For instance, the work \cite{ojala2002multiresolution} used three scales ( LBP(8, 1), LBP(16, 2) and LBP(24, 3) ) for texture classification task. To obtain rotation invariance and reduce the feature dimension, $LBP^{RIU}$ is used. Similar with traditional LBP, most LBP variants ( CLBP, LBPHF\_S/M, VLBP, LCP ) also used the multi-scale strategy. The multi-scale strategy has been widely recognized as an indispensable means to achieve superior performance.





\section{Globally Rotation Invariant Multi-scale Co-occurrence LBPs}
\label{sec:RIA}
In this section, we firstly analyze the correlation existing in multiple scales. The strong correlation among the scales motivates the joint encoding of multi-scale co-occurrence. Then,  we introduce the multi-scale co-occurrence of uniform LBP. And then, we detailly introduce and analyze three effective rotation invariant encoding methods. After, we analyze its relationship with single scale LBP. Finally, we introduce the classifier used in this paper.

\subsection{Correlation Analysis between Different Scales}
\label{sec:CADS}
As indicated before, each LBP pattern characterizes a local structure, but its discriminative power is limited by its small described region. For instance, LBP(8, 1) depicts a $3\times 3$ regions.
Though the multi-scale strategy can be used to depict multi-scale information, the traditional multi-scale methods ignore the correlation between different scales.

However there exists strong correlations between different scales. Ignoring the correlation will lead to huge information loss. As shown in Fig. \ref{fig:lbp_and_mslbp}, being effectively encoded, the patterns in multiple scales jointly reflect strong local structures. Below, we will firstly investigate the correlation between different scales.

The LBP patterns in different scales not only contains relevant information, but also has strong complementary information. In the literature
\cite{ojala2002multiresolution, maenpaa2003multi}, the classical multi-scale LBPs proves to significantly outperform the single-scale LBP. This observation shows that the LBPs in multi-scale are complementary. In \cite{qibmvc2}, Qi et al. use information theory to demonstrate the high correlation between different channels. Here, we also use the same tool to analyze the correlation between the LBP patterns in different scales. Table \ref{table:statistics} shows some statistics using  Outex\_00012 dataset \cite{ojala2002outex}.


%
%\begin{figure}[t]
%\begin{center}
% \includegraphics[width=1.0\linewidth]{figures/groupCoLBP.eps}
%\end{center}
%   \caption{An illustration of a group of multi-scale co-occurrence LBPs.}
%\label{fig:RotationInvariantCoLBP}
%\end{figure}




\begin{table}[h]
\caption{Texture Correlation between LBP(8, 1), LBP(8, 2), and LBP(8, 3).}              % title of Table
\centering                                     % used for centering table
%\small
\begin{tabular}{| @{\ }c@{\ } | @{\ }c@{\ } | @{\ }c@{\ } | @{\ }c@{\ } | @{\ }c@{\ } |}                       % centered columns (4 columns)
\hline
((8, 1), (8, 2))                 & H((8, 1))      & H((8, 2))      & H((8, 1), (8, 2))       & I((8, 1), (8, 2))       \\
\hline
Information Entropy    & 4.37      & 4.35      & 6.87          & \bf{1.84}          \\
\hline
((8, 1), (8, 3))                 & H((8, 1))      & H((8, 3))      & H((8, 1), (8, 3))       & I((8, 1), (8, 3))       \\
\hline
Information Entropy    & 4.37      & 4.37      & 7.82          & 0.91          \\
\hline
((8, 2), (8, 3))                 & H((8, 2))      & H((8, 3))      & H((8, 2), (8, 3))       & I((8, 2), (8, 3))       \\
\hline
Information Entropy    & 4.35      & 4.37      & 8.05          & 0.67          \\
\hline

\end{tabular}
\label{table:statistics}                   % is used to refer this table in the text
\end{table}

From Table \ref{table:statistics}, we have the following two key observations: (1): the LBP patterns in different scale have strong correlation. The mutual information is an effective indicator of the correlation. The correlation between LBP(8, 1) and LBP(8, 2) is higher.  (2): With the increase of the distance, the correlation decreases. For example, the correlation between LBP(8, 1) and LBP(8, 2) is greatly higher than that between LBP(8, 1) and LBP(8, 3).





Compared with single LBP, the joint encoding of multi-scale LBPs has two obvious advantages. Firstly, multi-scale LBPs depicts larger regions, thus they can capture stronger structures. 
Secondly, the joint encoding of multi-scale LBPs can capture richer patterns. For instance, for LBP(8, 1), it has 256 patterns. But for the co-occurrence of LBP(8, 1) and LBP(8, 2), it has 65536 co-patterns. Thus, if well encoded, the multi-scale co-occurrence of LBPs will have strong discriminative power.
%As shown in Fig. \ref{fig:lbp_and_mslbp}, the LBPs in multi-scales captures stronger curvature variation.




%\section{Rotation Invariance for Multi-scale Co-occurrence LBPs}
%\label{sec:MCLBP}
%\subsection{LBP and Multi-scale Co-occurrence LBPs}


\subsection{Multi-scale Co-occurrence LBP (MCLBP)}

%Then, we will present the MCLBP in detail.






In natural image, 58 uniform LBP patterns account for most of all 256 patterns. It is observed that the ratio is about 80\% to 90\%. Thus, instead of the original LBP, the uniform LBP is widely used. In this work, we also use uniform LBP though our method can be directly applied to the original LBP.

First, the LBP and ULBP value for each point in required scale was calculated. Then, the multi-scale co-occurrence of  $LBP^U$ (MCLBP) can be denoted as follows:
\begin{equation}
\label{eq:msjLBP}
CoLBP^{UU}(\vec{s_1}, \vec{s_2}) = [LBP^{U}(\vec{s_1}, i), LBP^{U}(\vec{s_2}, i)]_{co},
\end{equation}
where $i$ can be fixed to be any one in $[0, n-1]$. Here, we just set it to 0.

Thus, for each point, its MCLBP pattern can be denoted as a pattern in $59\times 59$. Any image can be denoted into a histogram distribution of these 3481 patterns. Note that This co-occurrence pattern is not rotation invariant. Image rotation will greatly change the histogram distribution.



%To achieve rotation invariance, our task is to extract the rotation invariant feature from the histogram distribution.
There exist some interesting rules among some co-patterns. As shown in Fig. \ref{fig:RotationInvariantCoLBP}, for the patterns in each row, with the image rotating, the patterns will turn into another one in the same row. In this paper, we call all patterns of each row as ``Group''.

According to the defined rule of groups, we can divide all 3841 co-patterns into 433 groups.

% Similar with the single scale LBP, the co-occurrence LBP patterns can be divided into some patterns. The co-occurrence patterns own the same property with single LBP pattern when the images rotate. As shown in \ref{fig:RotationInvariantCoLBP}, the shown eight patterns can be divided into a group. When all co-occurrence patterns are grouped, the same encoding methods in Sec. \ref{sec:RIA} can be applied.
%
%{\bf{Problem Formulation:\ }}In most current applications, LBP is usually divided into the uniform and non-uniform patterns. For the uniform patterns, we can further divide the patterns into different groups according to their properties. Here, we define a concept called ``Group'' for the uniform patterns. We define the $i$th group $G_i$ as:
%\begin{equation}
%\label{eq:set}
%    G: \{G_i \ |\ LBP^{RIU}(\vec{s}) = i \}
%\end{equation}
%%G: \{G(\vec{s},\ i)\ |\ i \in [0, n] \ \}
%where $\vec{s} = [n, r]$. Here, the $i$ can be considered as the number of ``1'' in the binary sequence.


\subsection{Globally Rotation Invariant Encoding Strategy}

%Properties of the uniform groups: (1) with the image rotation, one pattern will change to another pattern which belongs to the same group as the refer pattern.

For an image, its histogram distribution $H$ of all LBP patterns can be calculated. To make the integral probability to ``1'', we normalize the $H$ using $L_1$ norm.
Let $f$ be a feature extraction function, $G_i$ denotes the $i$th group, $H(G_i)$ denotes the probability distribution of each pattern in the group out of all LBP patterns. The $H(G_i)$ has the same dimension with $G_i$. Here, we use $F_i$ to denote the feature extracted from the distribution $H(G_i)$ using the $f$ feature extraction function. We can define the rotation invariant problem as follows:
\begin{equation}
\label{eq:set}
    f: H(G_i) \rightarrow F_i.
\end{equation}
%
%For an image, its histogram distribution of all LBP patterns can be calculated as $H$. Let $H(G_{ij})$ denote the probability distribution of $j$th pattern in $i$th group out of all LBP patterns.
%In ideal condition, the pattern in one group will not turn into another group under image rotation. Thus, the cumulative probability of one group in all patterns is constant under arbitrary image rotation.

There exists strong law between the probability distribution of the patterns in same group under different image rotation. As shown in Fig. \ref{fig:distribution}, when the image rotates 45 degrees, the probability distribution of one group just circularly shifts one unit. Thus, when we design the feature extraction function, the property of the distribution should be well considered. Actually in practice, the variation raw of the probability distribution in each group under different image rotation will facilitate our feature design.



%To achieve rotation invariance, we should extract the invariant features from each group.



For each group $G_i$, we can extract a feature $F_i$ according to the function $f$. The final feature $F$ can be denotes as follows:
\begin{equation}
\label{eq:set}
    F = [F_0,\ F_1, ......,\ F_n,\ F_{n+1}],
\end{equation}
where $n$ is the number of the neighbors.

\begin{figure}[t]
\begin{center}
 \includegraphics[width=1.0\linewidth, height=0.4\linewidth]{figures/aa.eps}
\end{center}
   \caption{The variation of probability distribution of patterns in one group when image rotates 45 degrees. }
\label{fig:distribution}
\end{figure}

Many methods can be used to encode the histogram distribution. Some may have strong discriminative power, but sensitive to image rotation. Vice verse. A desirable feature extraction function should have great balance between the discriminative power and transformation invariance. In this paper, for this purpose, we propose to use three rotation invariant encoding methods.

{\bf{Sum Pooling.}}
Uniform LBP depends on the defined start point of the binary sequence. Image rotation will vary one uniform pattern $A$ to another uniform pattern $B$, but $A$ and $B$ are in the same group. Thus, for an image, the cumulative probability of all patterns in a group -accounting for the all patterns- will not change. Thus, we can define the following feature extraction function $\vec{{F1_i}}$ for gorup $G_i$ as:
\begin{equation}
\label{eq:f1}
\vec{{F1_i}} = [\sum_{j = 1}^{N_i} h(G_{ij})],
\end{equation}
where $N_i$ is the numbers of patterns in group $G_i$.

{\bf{Remark:}} Sum pooling is an effective way to capture the cumulative probability in a group. As we have indicated before, regardless of how many degrees the image rotates, the pattern for each point will just change to another pattern in the same group. Thus, the cumulative probability will not vary.
%in ideal condition, when the image rotates a $360*j \div n$ degrees, the binary sequence just circularly rotates. Thus, the probabilities distribution just circularly rotate along with the image rotation. However, the ideal condition
%Like the binary sequence, along with image rotation, the probabilities in a group circularly rotate along with the image rotation. For instance, as shown in Fig. \ref{fig:distribution}, when the image rotates 45 degrees, the probability of $1$th position will turn to the probability in the $2$th position.
%%the relative position of the will change, but the quantity will not change. Thus, there are some

{\bf{Moment Based Pooling.}}
With the image rotating, the probability distributions of all patterns in one group circularly rotate. In such a probability set, there exist some constant properties besides of the cumulative probability.

In this work, we propose a new method to extract the invariant feature from such a group. The function can be defined as:

\begin{equation}
\label{eq:f2}
\vec{F2_i} = [ m, \ {\sigma}_1 \sum_{j=1}^{N_i} ( h(G_{ij}) - \frac{m}{N_i})^{2}],
\end{equation}
where $N_i$ is the numbers of patterns in group $G_i$, $m = \sum_{i = 1}^{N_i} h(G_{ij})$, ${\sigma}_1$  is a parameter.

%
%The second function can be defined as:
%\begin{equation}
%\label{eq:f22}
%\vec{F2_i} = [ m, \ {\sigma}_1 \sum_{j=1}^{N_i} ( h(G_{ij}) - \frac{m}{N_i})^{2}, \ {\sigma}_2 \sum_{j=1}^{N_i} ( h(G_{ij}) - h(G_{i j+1})^{2}],
%\end{equation}
%
%where $N_i$ is the numbers of patterns in group $G_i$. $G_{i N_i+1}$ is equal to $G_1$, ${\sigma}_1$ and ${\sigma}_2$ are two parameters to balance the weights among the three terms.

{\bf{Remark:\ }} The proposed moment based feature extraction method not only captures the cumulative probability in a group, but also reflects the relative variance between each bin and the mean distribution.


{\bf{Fourier Transformation Pooling.}}
In the work \cite{ahonen2009rotation}, Timo et al. firstly proposed to use fourier transformation to achieve a rotation invariant feature from the uniform patterns. Here, we briefly introduce their method. Firstly, we can calculate the Discrete Fourier Transform (DFT) of all patterns' distributions in a group as follows:

\begin{equation}
\label{eq:f3}
HF(k) = \sum_{j=0}^{N_l-1} G_{lj} \exp^{\frac{-i2 \pi jk}{N_l}},\ \ \ \ \  k = 0,.....,N_l-1
\end{equation}
where $N_l$ is the numbers of patterns in group $G_l$.

%
%\begin{figure}
%\begin{center}
% \includegraphics[width=0.80\linewidth]{figures/lbp_and_mslbp.eps}
%\end{center}
%   \caption{An illustration of LBP and Multi-scale LBP. Compared with LBP, LBPs in multiple scales jointly characterize stronger local structures. The LBP patterns in different scales have high correlation. }
%\label{fig:lbp_and_mslbp}
%\end{figure}


After obtaining the DFT of group $G_l$, since it is a complex data, we need to transform it into a real data, thus, we compute its magnitude. The magnitude for $HF(k)$ can be calculated as:
\begin{equation}
\label{eq:f3}
M(k) = \sqrt{HF(k)\overline{HF(k)}}.
\end{equation}


\begin{multicols}{2}
\begin{figure*}[t]
\begin{center}
 \includegraphics[width=0.85\linewidth]{figures/rulbp.eps}
\end{center}
   \caption{A group of single-scale LBPs. }
\label{fig:rlbp}
\end{figure*}
\end{multicols}


Since the last $N_l$-1 terms are sysmatric, the final fourier transformation histogram can be denoted as:
 \begin{equation}
\label{eq:f22}
\vec{F3_l} = [M(0),\  ....,\  M(N_l/2) ].
\end{equation}



{\bf{Remark:}} The DFT is invariant to the circular rotation of the histogram distribution. Thus, in theory, it is invariant to image rotation. However, the natural images are always captured under arbitrary angle. The histogram distribution is circular rotation only when the rotation degree is the times of $360/n$.



The all 3841 co-patterns can be divided into 433 groups. Thus, the dimension of the sum encoding method is 443 dimension for one co-occurrence pair, and the dimension for moment based pooling is 886. According to the DFT encoding strategy, the dimension for DFT based method is 2200.
As CLBP \cite{guo2010completed}, we can also use the center gray level-dividing the image into two parts according to the average gray value. Then, the feature dimension is two times of the original feature. According to \cite{guo2010completed}, the center gray level can provide significant improvement in some datasets.





\subsection{Generalized to Single-scale LBP}
The encoding strategy introduced above can be directly applied to single-scale LBP. As shown in Fig. \ref{fig:rlbp}, we can also construct some groups like above co-occurrence LBPs. For $n = 8$, as shown in Fig. \ref{fig:ulbp},
we can divide the 58 uniform patterns into 9 groups. For the group $G_1$, as shown in Fig. \ref{fig:rlbp}, it contains just one ``1'' in all eight binary bits. In each group, any pattern can rotate to any another pattern in the same group. Thus, in ideal condition, when the image rotates, the uniform pattern of each point will just turn into another pattern in the same group. In practice, we divide all non-uniform patterns into a group which be denoted as $G_{n+1}$.


\subsection{Classification}
To conduct the classification, we use the nearest neighbor (NN) classifier as the compared previous works though the support vector machine (SVM) always shows better performance. ${\chi}^2$ kernel proves to achieve great performance, and is widely used in previous texture classification task. To make the comparison fair, we also use it. The used ${\chi}^2$ kernel similarity between histogram features $X$ and $Y$ can be calculated as follows:
\begin{equation}
\label{eq:kernel}
S(X, Y) = \sum_{i=1}^{N} \frac{2X_iY_i}{X_i+Y_i},
\end{equation}
Where $N$ is the dimension of $X$ and $Y$. The ${\chi}^2$ kernel shows great performance compared with Linear and RBF kernel. It is widely used in texture classification \cite{guo2010completed} \cite{qi2013bmvc1} \cite{zhao2012rotation}. In this paper, to fairly compare with previous works, we will use the NN or SVM according to their used classifier. We use the Vlfeat \cite{vedaldi2010vlfeat} to conduct some fast computation, such similarity computation. For SVM, instead of carefully tuning the parameters, we use the Libsvm \cite{chang2011libsvm} tool with fixed C=100 for all experiments.




\begin{figure}[h]
\begin{center}
 \includegraphics[width=1.0\linewidth]{figures/outex10_curet.eps}
\end{center}
   \caption{Datasets.}
\label{fig:Datasets}
\end{figure}

conclusion.tex
\section{Conclusion}
\label{sec:conclusion}
We have proposed an efficient and effective method for automatic classification of HEp-2 cell via using multiresolution texture and richer shape information.  Specifically, we proposed to capture the multi-resolution
texture information by a novel Pairwise Rotation Invariant Co-occurrence of Local Gabor Binary Pattern (PRICoLGBP) descriptor, depict the richer shape information by using an Improved Fisher Vector (IFV) model with RootSIFT features which are sampled from large image patches in multiple scales, and combine them properly.  We have systematically evaluated the proposed approach on the ICPR 2012, ICIP 2013, and ICPR 2014 contest data sets. The proposed approach significantly outperformed the winners of ICPR 2012 and ICIP 2013 contests, and yileded comparable performance with the winner of the newly released ICPR 2014 contest.



%
%We evaluate the proposed features on the ICPR 2012 and ICIP 2013 contest data sets. The proposed method significantly outperforms the winner of ICPR 2012 and ICIP 2013 contest. Specifically, on ICPR contest, our method outperforms the ICPR winner by 7\%, and on ICIP 2013 contest dataset, our method outperforms the winner by more than 4\%. It should be noted that the difference between the $1$-st and $2$-nd places in ICIP 2013 contest is only 0.1\%.
%%
%%The proposed texture and shape features have many potential applications. They are not limited to the IIF HEp-2 cell classification. We believe that the proposed methods can generalize to many medical applications, such as Tissue Classification \cite{song2013feature} \cite{van1999automated} and Breast
%%Sonographic Tumor Diagnosis \cite{yang2013robust} and etc.

dataset.tex
\section{Datasets and Evaluation Metrics}
\label{sec:dataset}

%
%
%
%In this section, we will firstly introduce the ICPR 2012 contest data set, and then introduce the ICIP 2013 data set, and finally briefly introduce the evaluation metrics used in this paper.
%



\begin{table}[h]
\caption{Details of ICPR 2012 contest data set.}              % title of Table
\centering                                     % used for centering table
%\large
\normalsize
\begin{tabular}{| @{\ }c@{\ } | @{\ }c@{\ } | @{\ }c@{\ } |@{\ }c@{\ } | @{\ }c@{\ } | @{\ }c@{\ } | @{\ }c@{\ } | @{\ }c@{\ } |}                       % centered columns (4 columns)
\hline
                 &   Homo    &    Coar     &    Fine    &    Nucl    &    Cent    &   Cyto   & Total   \\[0.05cm]
\hline
Instances/train  &   3    &    2     &    2    &    2    &    3    &   2        &  14     \\
\hline
  Cells/train          &   150  &    109   &    94  &    102  &    208  &   60      &  723  \\
\hline
Instances/test   &   2    &    3     &    2    &    2    &    3    &   2        &  14     \\
\hline
  Cells/test          &   180  &    101   &    114  &    139  &    149  &   51      &  734  \\
\hline
\end{tabular}
\label{table:icprdataset}                   % is used to refer this table in the text
\end{table}



\subsection{ICPR 2012 Contest Dataset}
ICPR 2012 cell images were acquired by means of a fluorescence microscope (40-fold magnification) coupled with
a 50W mercury vapor lamp and with a digital camera.
The images have a resolution of $1388\times 1038$ pixels, a
color depth of 24 bits and they are stored in an uncompressed format.
Specialists manually segmented and annotated each
cell. In particular, a biomedical engineer manually segmented the cells by the use of a tablet PC. Subsequently,
each image was verified and annotated by a medical
doctor specialized in immunology.
The dataset contains 28 images almost equally distributed with respect to the different patterns.
In the contest, the 28 images are divided into training and testing sets. The information for training and testing sets is shown in Tab. \ref{table:icprdataset}. More detailed information can be found in \cite{icpr2012hep2}.
Some samples are shown in Fig. \ref{fig:icpr}.




Note that a specimen always has dozens of cells. The cells in the same specimen always have higher similarity than that of the cells from different specimens. Thus, to evaluate the generalization ability of the methods, the cells in one specimen can only be used for training or testing, it will be misleading to split them into training and testing. In the ICPR 2012 contest report, several methods used this strategy and directly splits all cell images instead of the specimens into training and validation sets, but their final results reported by the organizers were significantly lower than the authors' reported results.


%
%
%The ICPR dataset consists of six categories:
%
%
%\begin{itemize}
%
% \item {\bf{Centromere:}} characterized by serveral discrete speckles distributed throughout the interphase nucler and characteristically found in the condensed nuclear chromatin during mitosis as a bar of closely associated speckles;
%
% \item {\bf{Nucleolar:}} characterized by clustered large granules in the nucleoli of interphase cells which tend towards homogeneity, with less than six granules per cell;
%
% \item {\bf{Homogeneous:}} characterized by a diffuse staining of the interphase nuclei and staining of the chromatin of mitotic cells;
%
% \item {\bf{Fine Speckled:}} characterized by a fine granular nuclear staining of the interphase cell nuclei;
%
% \item {\bf{Coarse Speckled:}} characterized by a coarse granular nuclear staining of the interphase cell nuclei;
%
% \item {\bf{Cytoplasmic:}} characterized by fine fluorescent fibers running over the length of the cell.
%
%\end{itemize}
%The ICPR 2012 contest data set are color images. Sample images from ICPR 2012 are shown in Fig. \ref{fig:icpr}. For each image in the data set, a mask image is also provided. In Tab. \ref{table:icprdataset}, we present some details of ICPR 2012 contest data set.






\subsection{ICIP 2013 Contest Dataset}
%ICIP 2013 contest dataset was built between 2011 and 2013 at Sullivan Nicolaides Pathlogy laboratory, Austrilia. The dataset utilises 419 patient positive sera which were
%prepared on the 18-well slide of HEP-2000 IIF assay from
%Immuno Concepts N.A. Ltd. with screening dilution 1:80.
%The specimens were then automatically photographed using
%a monochrome high dynamic range cooled microscopy camera which was fitted on a microscope with a plan-Apochromat
%20x/0.8 objective lens and an LED illumination source. Approximately 100-200 cell images were extracted from each
%patient serum.
%%
%%In total there were 68,429 cell images extracted. We divided these into 13,596 images for training and
%%54,833 for testing.






The ICIP 2013 data set uses 419 patients positive sera with screening dilution 1:80. The specimens were automatically captured using a monochrome high dynamic range cooled microscopy camera. For each patient serum, 100-200 cell images were extracted. In total, there were 68429 cell images extracted. The whole 68429 cell images were divided into 13596 training samples and 54833 testing samples.


\begin{table}[h]
\caption{Details of ICIP 2013 contest data set.}              % title of Table
\centering                                     % used for centering table
%\large
\normalsize
\begin{tabular}{| @{\ }c@{\ } | @{\ }c@{\ } | @{\ }c@{\ } |@{\ }c@{\ } | @{\ }c@{\ } | @{\ }c@{\ } | @{\ }c@{\ } | @{\ }c@{\ } |}                       % centered columns (4 columns)
\hline
               &   Ho    &    Sp     &    Nu    &    Ce    &    NM    &   Go   & Total   \\[0.05cm]
\hline
Specimens      &   16    &    16     &    16    &    16    &    15    &   4        &  83     \\
\hline
%Positive       &   16    &    16     &    16    &    16    &    15    &   4        &  83     \\
%\hline
%Intermediate   &   16    &    16     &    16    &    16    &    15    &   4        &  83     \\
%\hline
  Cells        &   2494  &    2831   &    2598  &    2741  &    2208  &   724      &  13596  \\
\hline
\end{tabular}
\label{table:icipdataset}                   % is used to refer this table in the text
\end{table}


The labeling process involved at least two scientists who
read each patient's specimen under a microscope. A third expert's opinion was sought to adjudicate any discrepancy between the two opinions. In this way, a ground-truth mask can be extracted from each cell image.
%We used each specimen label for the
%ground-truth of cells extracted from it. Furthermore, all the
%labels were validated by using secondary tests such as ENA,
%and anti-ds-DNA to confirm the presence and/absence of specific patterns

%
%
%\begin{itemize}
%
% \item {\bf{Homogeneous:}} a uniform diffuse fluorescence covering the entire nucleoplasm sometimes accentuated in the nuclear periphery;
%
% \item {\bf{Speckled:}} these patterns have two categories:
%
%    - coarse speckled: densely distributed, variously sized speckles, generally associated with larger speckles, throughout nucleoplasm of interphase cells; nucleoli are negative;
%
%    - fine speckled: fine speckled staining in a uniform distribution, sometimes very dense so that an almost homogeneous pattern is attained; nucleoli may be positive or negative;
%
% \item {\bf{Nucleolar:}} brightly clustered large granules corresponding to decoration of the fibrillar centers of the nucleoli as well as the coiled bodies;
%
% \item {\bf{Centromere:}} rather uniform discrete speckles located throughout the entire nucleus;
%
% \item {\bf{Golgi:}} staining of a polar organelle adjecent to and partly surrounding the nucleus, composed of irregular large granules. Nuclei and nucleoli are negative. Diffuse staining of the cytoplasm of dividing cells sometimes with accentuation around chromosomal material;
%
% \item {\bf{Nuclear Membrane:}} a smooth homogeneous ring-like fluorescence of the nuclear membrane in interphase cells.
%
%\end{itemize}




The testing images are not released. But the training set is big enough to evaluate different algorithms. Some basic information for the training data in ICIP 2013 contest are shown in Tab. \ref{table:icipdataset}. More detailed information can be found in \cite{icip2013}. Some sample images are shown in Fig. \ref{fig:icip}.

It should be noted that in ICPR 2014 contest, the Task-1 used the same dataset as ICIP 2013 contest.







\subsection{Evaluation Metrics}
In the previous ICPR 2012 and ICIP 2013 contests, accuracy of maximum classification number is used as a performance metric. For specimen, in ICPR 2012 data set, the testing number of images are 734, if the 500 images are classified correctly, then the accuracy is $\frac{500}{734}$. In this paper, we follow the metric of the previous ICPR 2012 and ICIP 2013 contest, and use the maximum classification number as the metric.

When comparing our method with ICPR 2014 winner \cite{manivannan2014hep}, we strictly follow the winner's protocol, and use the leave-one-specimen-out protocol. The averaged Mean Class Accuracy (MCA) is reported.
%Mean class accuracy (MCA) is another widely used performance metric. The difference between them comes from the unbalance of the testing samples in each category.

experiments.tex
\section{Experiments}
\label{sec:experiments}




%
%
%In this experimental section, we will firstly introduce some implementation details, then evaluate some properties of the proposed methods, and finally compared our methods with the state-of-the-art methods on ICPR 2012 and ICIP 2013  contest data sets individually.

\subsection{Implementation Details}
{\bf{PRICoLGBP.}} For multi-resolution PRICoLGBP feature, we use the original image and 7 Gabor-filtered images under 7 different scales $\{1, 2, 3, 4, 5, 6, 7\}$. For each filtered image, we can extract one PRICoLBP feature. In each PRICoLBP feature, we use 10 templates. As we described before, the dimension of PRICoLBP using one template is 590. Thus, the final dimension for PRICoLGBP is $8\times10\times590=47200$.

{\bf{RootSIFT(IFV).}} We densely sample the RootSIFT feature at six scales with grid step 2. The sampled patch size is $41\times 41$. If the image size (height or width) is less 64, we will resize it to the image with minimum size 64 and keep the height/width ratio.
Six scales are achieved by filtering the images with Gaussians with different scales of different standard deviates $\{{1.5}, {1.5}^2, {1.5}^3, {1.5}^4, {1.5}^5, {1.5}^6\}$.
For specimen, for an image with image size $70\times 70$, we can sample 225 points for each scale. Thus, for six scales, we can get 1350 sampled patches. For a larger image, such as $70\times 80$, we will sample more points.
In the IFV, we firstly sample 100000 RootSIFT features from the training samples, then the
100000 RootSIFT features are used to learn the PCA components, and 80 principal components are preserved as the basis for dimension reduction. As pointed out by \cite{sanchez2013image}, the PCA is a key step in the IFV framework. With above-mentioned 100000 after-PCA RootSIFT feature, we learn a Gaussian Mixture Model (GMM) with 256 components. For the PCA, we use the built-in SVD (Singular Value Decomposition). For the GMM, we use Vlfeat to learn the parameters $\theta = \left \{ \pi_k, \mu_k, \Sigma_k, k=1,\cdots,K\right \}$.
The final dimension using the IFV encoding is $2\times80\times256 = 41960$.

{\bf{Experimental Setups.}} Vlfeat toolbox \cite{vedaldi2010vlfeat} is used for fast RootSIFT extraction and IFV encoding, and Liblinear \cite{fan2008liblinear} is used for the linear SVM training and classification. For the parameter C, we cross-validated it in \{0.001, 0.01, 0.1, 1, 100, 1000\}. It should be noted that the first author of this paper provides PRICoLBP feature and classifier for Shen et al. (the ICIP 2013 winner). We share the source code that had been submitted into ICIP 2013 and achieved the 1st place.
All experimental comparisons are conducted in the same framework. Take ICIP 2013 contest data set as example, first, we create 10 splits for 10 repeated experiments. For each split, the whole ICIP contest 2013 data set are randomly divided into the training and testing sets. Meanwhile, to truly show the generalization performance of approaches, the images from the same cell are only divided into training or testing set. Thus, All comparisons are fair in this paper. We have provided the matlab code\footnote{\url{https://www.dropbox.com/s/eoifdhqjs1o7vky/HEp2Cell.zip?dl=0}} to repeat the experimental results.
%For the regularization $C$ in the linear SVM, we fix it as 40 for all experiments.
%We also have the source code for ICIP winner's algorithm.












\subsection{Evaluation of Features}
In this subsection, we will mainly evaluate some aspects of the proposed texture and shape features. The ICPR 2012 data set is too small to fully evaluate the properties of the proposed methods. Thus, we will use ICIP 2013 data set in this subsection. To fully evaluate the properties, we use four sets of different experimental setups, as shown in Tab. \ref{table:icipsetup}.
Take the setup ``D'' as an example, in experimental setup ``D'', 42 specimens (including 8 specimens from ``Homogeneous'', 8 specimens from ``Speckled'', 8 specimens from ``Nucleolar'', 8 specimens from ``Centromere'', 8 specimens from ``NuMem'' and 2 specimens from ``Golgi'') in all 83 specimens are used for training, and the rest 41 specimens are used for testing, each specimen includes 100-200 cell images. Using this strategy, the images in one specimen can only be divided into training or testing. This used strategy can truly reflect the generalization ability because the images come from the specimen usually have higher similarity than that between images from different specimen, if part of the images in one specimen are used for training, the rest images that are used for testing are easily correctly classified, but this strategy can not be generalized to other unknown specimen. We pre-create 10 training and testing splits randomly. We repeat the experiments 10 times and average the results.



\begin{table}[h]
\caption{Four experimental setups for ICIP 2013 data set. For example, ``1'' means all images from one specimen are used for training, and the remaining specimens are used for testing. }              % title of Table
\centering                                     % used for centering table
%\large
\normalsize
\begin{tabular}{| c | c | c | c | c | c | c | c |}                       % centered columns (4 columns)
\hline
           &   Ho    &    Sp     &    Nu    &    Ce    &    NM    &   Go   & total   \\[0.05cm]
\hline
  Setup A        &   1     &    1      &    1     &    1     &    1     &   1        &  6     \\
\hline
  Setup B        &   2     &    2      &    2     &    2     &    2     &   2        &  12  \\
\hline
  Setup C        &   4     &    4      &    4     &    4     &    4     &   2        &  22  \\
\hline
  Setup D        &   8     &    8      &    8     &    8     &    8     &   2        &  42  \\
\hline
\end{tabular}
\label{table:icipsetup}                   % is used to refer this table in the text
\end{table}

%In fact, in ICPR 2012 and ICIP 2013 contest, there are some methods they reported extremely high performance but got bad ranking because of their evaluation strategy. For instance, some submissions tested their algorithms by randomly dividing all images into half training images and half testing images, where the cell images from the same category can appear in both training and testing sets, then they can report high performance, but this is biased. Thus, for each experimental setup,





%
%only six instances (each instance has about 160 cell images.) are used for training, the rest 83-6=77 instances are used for testing. It should be noted that the images in one instance can only be divided into training or testing. Thus, for each setup, we pre-create 10 training and testing splits randomly. It can promise the fairness of all comparisons. When we compare two methods, we use the same training and testing sets to evaluate them. For example, in experimental setup ``D'', 42 instances (including 8 instances from ``Homogeneous'', 8 instances from ``Speckled'', 8 instances from ``Nucleolar'', 8 instances from ``Centromere'', 8 instances from ``NuMem'' and 2 instances from ``Golgi'') in all 83 instances are used for training, and the rest 41 instances are used for testing.  In fact, in ICPR 2012 and ICIP 2013 contest, there are some methods they reported extremely high performance but got bad ranking because of their evaluation strategy. For instance, some submissions tested their algorithms by randomly dividing all images into half training images and half testing images, where the cell images from the same category can appear in both training and testing sets, then they can report high performance, but this is biased. We are confident that The used strategy can truely reflect the generalization ability. We repeat the experiments 10 times and average the results.}


%Or the generalization ability is hard to be evaluated.


{\bf{Evaluation of Multi-Resolution Texture Extraction Strategy.}} Here, we conduct experiments to compare the PRICoLBP and PRICoLGBP on above-mentioned four experimental setups. The results are shown in Tab. \ref{table:MultiResolution}.






\begin{table}[h]
\caption{Comparison between the single-resolution and multi-resolution texture feature on ICIP 2013 data set.}              % title of Table
\centering                                     % used for centering table
%\large
%\normalsize
\small
\begin{tabular}{| @{}c@{} | @{\ }c@{\ } | @{\ }c@{\ } | @{\ }c@{\ } | @{\ }c@{\ } | }
\hline
                 &   Setup A       &   Setup B   &  Setup C     &  Setup D         \\[0.1cm]
\hline
PRICoLBP         &   $43.1 \pm{6.8}$  &    $55.2 \pm{5.4}$ &    $67.6 \pm{4.0}$  &    $73.9 \pm{2.9}$ \\ [0.1cm]
\hline
PRICoLGBP   &   $50.4 \pm{6.2}$  &    $61.4 \pm{5.1}$ &    $72.2 \pm{3.8}$  &    $78.4 \pm{2.7}$ \\ [0.1cm]
\hline
\end{tabular}
\label{table:MultiResolution}                   % is used to refer this table in the text
\end{table}




We can observe that from Tab. \ref{table:MultiResolution}, multi-resolution texture feature significantly improves the single-resolution texture feature. For specimen, the multi-resolution PRICoLGBP improves the PRICoLBP by 7.3\% and 4.5\% for the experimental setup ``A'' and ``D''.
%It should be noted that in Shen, the combination between PRICoLBP (73.9\%) and Bag-of-SIFT with VQ (73.6\%) improves the performance to 75.1\%.




















{\bf{Evaluation of Improved Fisher Vector Encoding.}} To evaluate the effectiveness of the Improved Fisher Vector, we compare it with the traditional Vector Quantization (VQ). For both VQ and IFV, the feature is normalized according to Eq. \ref{eq:chiB}. A linear SVM is used for training and classification. The results averaged on 10 random repeats are shown in Tab. \ref{table:IFV}.












\begin{table}[h]
\caption{Comparison between Vector Quantization and Improved Fisher Vector on ICIP 2013 data set.}              % title of Table
\centering                                     % used for centering table
%\large
%\normalsize
\small
\begin{tabular}{| @{\ }c@{\ } | @{\ }c@{\ } | @{\ }c@{\ } | @{\ }c@{\ } | @{\ }c@{\ } | }
\hline
                 &   Setup A       &   Setup B   &  Setup C     &  Setup D        \\[0.1cm]
\hline
RootSIFT(VQ)        &   $54.9 \pm{6.8}$  &    $62.2 \pm{4.5}$ &    $68.3\pm{2.4}$  &    $71.2 \pm{2.3}$ \\ [0.1cm]
\hline
RootSIFT(IFV)         &   $58.7 \pm{6.7}$  &    $66.7 \pm{5.1}$ &    $74.7 \pm{2.7}$ &    $78.4 \pm{2.7}$ \\ [0.1cm]
\hline
\end{tabular}
\label{table:IFV}                   % is used to refer this table in the text
\end{table}





 From Tab. \ref{table:IFV}, we can find that the IFV encoding method sharply improves the performance of the VQ encoding method. For specimen, under the experimental configuration ``D'', the IFV improves the VQ from 71.2\% to 78.4\%. In conclusion, the IFV is an effective way to preserve the discriminative power of the features under the BoW framework.




%
%{\bf{Evaluation of the Patch Size.}}
%
%
%\begin{table}[h]
%\caption{Performance evaluation of FV encoding with SIFT under different patch size .}              % title of Table
%\centering                                     % used for centering table
%\large
%%\normalsize
%\begin{tabular}{| c | c | c | c | c |  }
%\hline
%                 &   A       &    B      &    C     &    D     \\[0.1cm]
%\hline
%$17 \times 17$   &   60.10   &    60.10  &    60.10 &    60.10 \\[0.1cm]
%\hline
%$21 \times 21$   &   70.10   &    70.10  &    70.10 &    70.10 \\[0.1cm]
%\hline
%$27 \times 27$   &   70.10   &    70.10  &    70.10 &    70.10 \\[0.1cm]
%\hline
%$35 \times 35$   &   70.10   &    70.10  &    70.10 &    70.10 \\[0.1cm]
%\hline
%$41 \times 41$   &   70.10   &    70.10  &    70.10 &    70.10 \\[0.1cm]
%\hline
%\end{tabular}
%\label{table:MultiResolution}                   % is used to refer this table in the text
%\end{table}
%
%We could find that large patch always show better performance than small patch. This may reflect that the shape information is effective on IIF cell classification. Large patch can better capture shape information.








{\bf{Evaluation of Normalization Method.}} Here, we evaluate the importance of the normalization method. For both PRICoLGBP feature and RootSIFT(IFV), we normalized the histograms according to Eq. \ref{eq:chiB}. We compare them with the direct $L_2$ normalized histograms(without using Eq. \ref{eq:chiB}) under the linear SVM framework.
The results averaged on 10 random repeats are shown in Tab. \ref{table:Normalization}.



%
%\begin{multicols}{2}
%\begin{table*}[h]
%\caption{ICIP 2013.}              % title of Table
%\centering                                     % used for centering table
%%\large
%\normalsize
%\begin{tabular}{| c | c | c | c |}                       % centered columns (4 columns)
%\hline
%           &   PRICoLBP   &    BoW of SIFT &    PRICoLBP + BoW of SIFT     \\[0.05cm]
%\hline
%Accuracy  &    20.0       &    20.0       &          20.0        \\[0.05cm]
%\hline
%           &   Gabor PRICoLBP   &    FV with RootSIFT &    Gabor PRICoLBP   +    FV with RootSIFT    \\[0.05cm]
%\hline
%Accuracy  &    20.0       &    20.0       &          20.0        \\[0.05cm]
%\hline
%\end{tabular}
%\label{table:icipstate}                   % is used to refer this table in the text
%\end{table*}
%\end{multicols}







\begin{table}[h]
\caption{Evaluation of Normalization Methods on RootSIFT(IFV) and PRICoLGBP on ICIP 2013 data set. In the table, ``PRICoLGBP'' and ``RootSIFT(IFV)'' denotes the feature under the direct $L_2$ normalization, ``PRICoLGBP*'' and ``RootSIFT(IFV*)'' denotes the features normalized by Eq. \ref{eq:chiB} and then by $L_2$ normalization.}              % title of Table
\centering                                     % used for centering table
%\large
%\normalsize
\small
\begin{tabular}{| @{\ }c@{\ } | @{\ }c@{\ } | @{\ }c@{\ } | @{\ }c@{\ } | @{\ }c@{\ } |  }
\hline
                 &   Setup A       &   Setup B   &  Setup C     &  Setup D         \\[0.1cm]
\hline
PRICoLGBP           &   $48.4 \pm{6.2}$   &    $59.4 \pm{4.6}$  &   $70.3 \pm{3.2}$ &    $77.5 \pm{3.2}$ \\[0.1cm]
\hline
PRICoLGBP*          &   $50.4 \pm{6.2}$   &    $61.4 \pm{5.1}$ &    $72.2 \pm{3.8}$ &    $78.4 \pm{2.7}$ \\ [0.1cm]
\hline
RootSIFT(IFV)             &   $57.8 \pm{7.0}$   &    $65.7 \pm{5.4}$ &    $73.6 \pm{2.9}$ &    $77.4 \pm{2.7}$ \\[0.1cm]
\hline
RootSIFT(IFV*)            &   $58.7 \pm{6.7}$   &    $66.7 \pm{5.1}$ &    $74.7 \pm{2.7}$ &    $78.4 \pm{2.7}$ \\[0.1cm]
\hline
\end{tabular}
\label{table:Normalization}                   % is used to refer this table in the text
\end{table}


From Tab. \ref{table:Normalization}, it is easy to find that the PRICoLGBP with normalization according to Eq. \ref{eq:chiB} consistently outperforms the PRICoLGBP without normalization, and the RootSIFT(IFV) using normalization also consistently outperforms the non-normalized feature.
In conclusion, the normalization always improves the classification accuracy.
%we can find that the used normalization method consistently boosts the classification performance.



\begin{multicols}{2}
\begin{figure*}
\begin{center}
\small
 \includegraphics[width=0.92\linewidth]{icpricip.eps}%{figures/examples.eps}
\end{center}
   \caption{Classification confusion matrixs and averaged accuracies on (a). ICPR 2012 and (b) ICIP 2013 contest data sets.}
\label{fig:icprconfusion}
\end{figure*}
\end{multicols}



%
%\begin{multicols}{2}
%\begin{figure*}[t]
%\begin{center}
%\small
% \includegraphics[width=1.1\linewidth]{figures/icipnew.eps}%{figures/examples.eps}
%\end{center}
%   \caption{Classification confusion matrix and averaged accuracies on ICIP 2013 contest data set.}
%\label{fig:icipconfusion}
%\end{figure*}
%\end{multicols}





\subsection{Comparison with the State-of-the-art Methods}
%For all experiments, we set \textcolor{blue}{the soft margin parameter ``C = 1''} without tuning any parameter.

{\bf{Experiments on ICPR 2012 contest.}} For this dataset, we evaluate seven methods, including PRICoLBP, PRICoLGBP, RootSIFT(IFV), the combination of PRICoLGBP and RootSIFT(IFV), and the top three methods in ICPR 2012 contest. For PRICoLBP and PRICoLGBP, we use the green channel. For RootSIFT(IFV), we use the gray image. In IFV, since the sampled patch is $41\times 41$, when the minimal size of the image is less than 50, we will resize the image to the minimal size 64 while keeping the ratio between the height and width. Since the dataset is very small, for the PRICoLBP and PRICoLGBP, we directly use SVM with ${\chi}^2$ kernel. For the RootSIFT(IFV), and the combination of PRICoLGBP and RootSIFT(IFV), we use linear SVM. The classification confusion matrix and averaged accuracies using the provided experimental setup by the ICPR 2012 contest organizers are shown in Fig. \ref{fig:icprconfusion}(a).




We have the following observations from Fig. \ref{fig:icprconfusion}(a):
\begin{itemize}
\item Texture based methods works better than the shape based methods. For specimen, PRICoLBP achieves 73.7\% which is higher than RootSIFT(IFV) (69.6\%).

\item IFV encoding with RootSIFT works well on this dataset, and slightly outperforms winner of ICPR 2012 contest.

\item The combination of our PRICoLGBP and RootSIFT(IFV) significantly outperforms the winner of ICPR 2012, and performs better than the latter on four categories including ``Homogeneous'', ``Coarse Speckled'', ``Centromere'' and ``Cytoplasmic'', and worse on the categories ``Nucleolar'' and ``Fine Speckled''.


\end{itemize}


%%%
%%%\begin{table}[h]
%%%\caption{Comparison with State-of-the-art methods on ICPR 2012 contest dataset.}              % title of Table
%%%\centering                                     % used for centering table
%%%%\large
%%%\normalsize
%%%%\small
%%%\begin{tabular}{|c | c |  }
%%%\hline
%%%
%%%Methods               &   Accuracy          \\[0.1cm]
%%%\hline
%%%Kuan et al.           &     64.2            \\[0.1cm]
%%%\hline
%%%Xiangfei et al.       &     65.8            \\[0.1cm]
%%%\hline
%%%Nokasa et al.         &     68.7            \\[0.1cm]
%%%\hline
%%%RootSIFT(IFV)             &     69.6            \\[0.1cm]
%%%\hline
%%%%PRICoLBP (Gray)       &     72.5            \\[0.1cm]
%%%%\hline
%%%PRICoLBP           &     73.7            \\[0.1cm]
%%%\hline
%%%%Gabor PRICoLBP (Gray) &     73.7            \\[0.1cm]
%%%%\hline
%%%Gabor-PRICoLBP     &     73.3            \\[0.1cm]
%%%%\hline
%%%%SIFT(FV)+Gabor PRICoLBP (R)             &                 \\[0.1cm]
%%%\hline
%%%Gabor-PRICoLBP + RootSIFT(IFV)  &  \bf{75.6} \\[0.1cm]
%%%\hline
%%%\end{tabular}
%%%\label{table:icprstate}                   % is used to refer this table in the text
%%%\end{table}
%%%
%%%
%
%The confusion matrix in Fig. \ref{fig:icprconfusion} shows the combination of our Gabor-PRICoLBP and RootSIFT(IFV) performs better than the winner of ICPR 2012 contst on five categories including ``Homogeneous'', ``Coarse Speckled'', ``Nucleolar'', ``Centromere'' and ``Cytoplasmic'', and worse on the category ``Fine Speckled''.




It should be noted that the experimental results on ICPR contest dataset are sensitive to the classifier's parameter C. We used the training set to conduct cross-validation to get a good C. Since the number of all specimens in ICPR 2012 contest is limited, thus, we use leave-one-out strategy to make cross-validation.
%
%
%To avoid overfitting, we just set the C = 1 for all experiments although we also found that when C = 1.5, PRICoLBP on Green channels can achieve 77.1\%.
%According to our experiments, we find that the scale of ICPR 2012 contest data set is small and is not enough to evaluate different algorithms. The scale of ICIP 2013 data is about 10 times of ICPR 2012 data set. We found that the classification performance on ICIP 2013 data set are not sensitive to classifier's parameter C.




{\bf{Experiments on ICIP 2013 contest.}} We evaluate and compare six methods including RootSIFT(VQ), PRICoLBP, the combination of RootSIFT with VQ and PRICoLBP, PRICoLGBP, RootSIFT(IFV), and the combination of PRICoLGBP and RootSIFT(IFV).
Here, we use the experimental setup ``D''. The features are all normalized, and a linear SVM. The classification confusion matrix and averaged accuracies based on 10 random repeats are shown in Fig. \ref{fig:icprconfusion}(b).



%with the soft margin parameter C = 40 is used






The confusion matrix in Fig. \ref{fig:icprconfusion}(b) indicates that:
\begin{itemize}

\item Multi-resolution PRICoLGBP texture feature significantly outperforms the single-resolution PRICoLBP, and improves the performance from 73.9\% to 78.4\%. PRICoLGBP significantly improves the PRICoLBP on several categories such as ``Speckled'' and ``Golgi'', and has high performance on other categories. Compared with RootSIFT(VQ), RootSIFT(IFV) significantly outperforms the former on all categories. This fully demonstrates the effectiveness of IFV encoding methods.

%\item IFV encoding sharply outperforms the VQ encoding, and improves the latter from 71.2\% to 78.4\%.

\item The combination between texture and shape features outperforms each of them. For specimen, the combination of RootSIFT(VQ) and PRICoLBP improves the PRICoLBP (73.9\%) and RootSIFT(VQ) (71.2\%) to 75.3\%. And, the combination of PRICoLGBP and RootSIFT(IFV) greatly improves the Shen's method (the winner of ICIP 2013) from 75.3\% to 79.5\%.

%\item The combination of PRICoLGBP and Bag of RootSIFT (IFV) improves the combination of PRICoLBP and RootSIFT (VQ) on all categories especially on the categories ``Homogeneous'' (6.8\% improvement) and ``Golgi'' (12.6\% improvement).

\item The category ``Golgi'' obtains the lowest performance, this is due to the less training sampling in this category. The most confusing pairs are ``Golgi'' and ``Nucleolar'', and ``Speckled'' and ``Homogeneous''. It is easy to find that from Fig. \ref{fig:icip}, the shape and texture structures in ``Homogeneous'' and ``Speckled'' look similar.



\end{itemize}









{\bf{Comparision with the Winner of ICPR 2014 contest.}}
Recently, the results of ICPR 2014 HEp-2 Cell classification have been released. In this part, we compare our method with the winner of ICPR 2014 contest \cite{manivannan2014hep}. We strictly follow the experimental protocol of the winner, and use the leave-one-specimen-out strategy. According to the specimen IDs, we can split the data into training and validation sets. Since we have 83 different specimens, in each test, we use 82 specimens for training and the left one for testing. The result of the ICPR 2014 winner is shown in Table \ref{table:icpr2014winner}(a), and our result based on PRIGCoLBP and RootSIFT(IFV) is shown in Table \ref{table:icpr2014winner}(b).








%
%\begin{table}[h]
%\caption{Confusion matrix of our method for leave-one-specimen-out experiments on I3A Task-1 dataset.}       % title of Table
%\centering                                     % used for centering table
%%\large
%\normalsize
%%\small
%\begin{tabular}{ c  c c c c c c  }
%\hline
%
%               &   Homo.  &  Spec.  &  Nucl.  &  Cent.  &  NuMe.  & Golgi \\[0.1cm]
%\hline
%Homo.          &   \bf{81.32}   &  15.56  &  0.80   &  0.12   &  1.96   &  0.24 \\[0.1cm]
%Spec.          &   12.19   &  \bf{73.86}  &  3.74   &  9.29   &  0.78   &  0.14 \\[0.1cm]
%Nucl.          &   1.58   &  2.23   &  \bf{92.49}  &  2.19   &  0.92   &  0.58 \\[0.1cm]
%Cent.          &   0.15   &  10.98  &  3.43   &  \bf{85.41}  &  0   &  0.04 \\[0.1cm]
%NuMe.          &   6.52   &  1.54   &  1.27   &  0.05   &  \bf{88.04}  &  2.58 \\[0.1cm]
%Colgi.         &   8.56   &  4.70   &  19.34  &  3.31   &  4.97   &  \bf{59.12}\\[0.1cm]
%\hline
%
%
%\hline
%\end{tabular}
%\label{table:icpr2014our}                   % is used to refer this table in the text
%\end{table}



According to Table \ref{table:icpr2014winner}, the Mean Class Accuracy (MCA) for the winner of ICPR 2014 contest is 80.25\%, and our method achieves a comparable performance 80.04\%. It should be noted that Manivannan et al. used four types of features, and for each feature, they trained 4 models, and the final classification result is based on summation of probability of 16 classifier. However, we only trained one model using the combination of the RootSIFT (IFV) and PRICoLGBP features. We can find that our method performs better on ``Nucleolar'' and ``Golgi'', and performs worse on ``Homogeneous'', ``Speckled'' and ``Centromere''.






\begin{table}[h]
\caption{Confusion matrixs of (a). the ICPR 2014 Contest Winner, and (b). Our method for leave-one-specimen-out experiments on I3A Task-1 dataset.}       % title of Table
\centering                                     % used for centering table
%\large
\normalsize
%\small
\begin{tabular}{ c  c c c c c c  }

\multicolumn{7}{c}{(a). The winner of ICPR 2014 contest.} \\
\hline
               &   Homo.  &  Spec.  &  Nucl.  &  Cent.  &  NuMe.  & Golgi \\[0.1cm]
\hline
Homo.          &   \bf{81.8}   &  15.00  &  0.76   &  0.20   &  2.04   &  0.20 \\[0.1cm]
Spec.          &   8.87   &  \bf{77.36}  &  3.67   &  9.18   &  0.74   &  0.18 \\[0.1cm]
Nucl.          &   1.12   &  3.89   &  \bf{90.65}  &  2.08   &  1.27   &  1.00 \\[0.1cm]
Cent.          &   0.47   &  10.87  &  2.85   &  \bf{85.66}  &  0.04   &  0.11 \\[0.1cm]
NuMe.          &   6.30   &  2.04   &  1.40   &  0.27   &  \bf{88.04}  &  1.95 \\[0.1cm]
Colgi.         &   5.66   &  3.73   &  20.72  &  2.35   &  9.53   &  \bf{58.01}\\[0.1cm]
\hline

\\
\multicolumn{7}{c}{(b). Our method.} \\
\hline
               &   Homo.  &  Spec.  &  Nucl.  &  Cent.  &  NuMe.  & Golgi \\[0.1cm]
\hline
Homo.          &   \bf{81.32}   &  15.56  &  0.80   &  0.12   &  1.96   &  0.24 \\[0.1cm]
Spec.          &   12.19   &  \bf{73.86}  &  3.74   &  9.29   &  0.78   &  0.14 \\[0.1cm]
Nucl.          &   1.58   &  2.23   &  \bf{92.49}  &  2.19   &  0.92   &  0.58 \\[0.1cm]
Cent.          &   0.15   &  10.98  &  3.43   &  \bf{85.41}  &  0   &  0.04 \\[0.1cm]
NuMe.          &   6.52   &  1.54   &  1.27   &  0.05   &  \bf{88.04}  &  2.58 \\[0.1cm]
Colgi.         &   8.56   &  4.70   &  19.34  &  3.31   &  4.97   &  \bf{59.12}\\[0.1cm]


\hline
\end{tabular}
\label{table:icpr2014winner}                   % is used to refer this table in the text
\end{table}






%\begin{table}[h]
%    \begin{subtable}[h]{0.45\textwidth}
%        \centering
%        \begin{tabular}{l | l | l}
%        Day & Max Temp & Min Temp \\
%        \hline \hline
%        Mon & 20 & 13\\
%        Tue & 22 & 14\\
%        Wed & 23 & 12\\
%        Thurs & 25 & 13\\
%        Fri & 18 & 7\\
%        Sat & 15 & 13\\
%        Sun & 20 & 13
%        \end{tabular}
%        \caption{First Week}
%        \label{tab:week1}
%    \end{subtable}
%    \hfill
%    \begin{subtable}[h]{0.45\textwidth}
%        \centering
%        \begin{tabular}{l | l | l}
%        Day & Max Temp & Min Temp \\
%        \hline \hline
%        Mon & 17 & 11\\
%        Tue & 16 & 10\\
%        Wed & 14 & 8\\
%        Thurs & 12 & 5\\
%        Fri & 15 & 7\\
%        Sat & 16 & 12\\
%        Sun & 15 & 9
%        \end{tabular}
%        \caption{Second Week}
%        \label{tab:week2}
%    \end{subtable}
%    \caption{Max and min temps recorded in the first two weeks of July}
%    \label{tab:temps}
%\end{table}


%%%
%%%\begin{table}[h]
%%%\caption{Comparison with State-of-the-art methods on ICIP 2013 contest dataset.}              % title of Table
%%%\centering                                     % used for centering table
%%%%\large
%%%\normalsize
%%%%\small
%%%\begin{tabular}{|c | c |  }
%%%\hline
%%%
%%%Methods               &   Acc.       \\[0.1cm]
%%%
%%%\hline
%%%RootSIFT(VQ)             &    71.2             \\[0.1cm]
%%%\hline
%%%PRICoLBP             &   73.9 \\[0.1cm]
%%%\hline
%%%RootSIFT(VQ)+PRICoLBP (ICIP winner's method)   &   75.3 \\[0.1cm]
%%%\hline
%%%PRICoLGBP       &   78.4 \\[0.1cm]
%%%\hline
%%%RootSIFT(IFV)            &   78.4 \\[0.1cm]
%%%\hline
%%%PRICoLGBP + RootSIFT(IFV)  & \bf{79.5}  \\[0.1cm]
%%%\hline
%%%\end{tabular}
%%%\label{table:icipstate}                   % is used to refer this table in the text
%%%\end{table}















%
%
%To intuitively compared the proposed method (Gabor PRICoLBP and RootSIFT with IFV) and the winner method (PRICoLBP and RootSIFT with VQ), we show a category-wise classification accuracy in Fig. \ref{fig:icipcategory}. From Fig. \ref{fig:icipcategory}, we can find that our proposed method consistently outperforms the winner of ICIP 2013 contest. Specifically, on categories 'Homogeneous' and 'Golgi', the improvement is huge, up to 6.7\% and 12.4\% individually.
%
%
%
%
%\begin{figure}[h]
%\begin{center}
%\small
% \includegraphics[width=0.9\linewidth]{figures/icip2.eps}%{figures/examples.eps}
%\end{center}
%   \caption{Category-wise performance comparison between the ICIP bese result and the proposed method in this paper. }
%\label{fig:icipcategory}
%\end{figure}




%
%
%\begin{multicols}{2}
%\begin{table*}[h]
%\caption{ICPR 2012.}              % title of Table
%\centering                                     % used for centering table
%%\large
%\normalsize
%\begin{tabular}{| c | c | c | c | c |}                       % centered columns (4 columns)
%\hline
%Methods           &   Xiangfei et al.  &    PRICoLBP        &    BoW of SIFT      &    PRICoLBP + BoW of SIFT     \\[0.05cm]
%\hline
%Accuracy   &          20.0      &     20.0           &    20.0             &          20.0        \\[0.05cm]
%\hline
%Methods           &   Nokasa et al.    &   Gabor PRICoLBP   &    FV with RootSIFT &    Gabor PRICoLBP   +    FV with RootSIFT    \\[0.05cm]
%\hline
%Accuracy   &    20.0            &     20.0           &    20.0             &          20.0        \\[0.05cm]
%\hline
%\end{tabular}
%\label{table:icprstate}                   % is used to refer this table in the text
%\end{table*}
%\end{multicols}
%



HEp2Cell.tex

%% bare_jrnl.tex
%% V1.3
%% 2007/01/11
%% by Michael Shell
%% see http://www.michaelshell.org/
%% for current contact information.
%%
%% This is a skeleton file demonstrating the use of IEEEtran.cls
%% (requires IEEEtran.cls version 1.7 or later) with an IEEE journal paper.
%%
%% Support sites:
%% http://www.michaelshell.org/tex/ieeetran/
%% http://www.ctan.org/tex-archive/macros/latex/contrib/IEEEtran/
%% and
%% http://www.ieee.org/



% *** Authors should verify (and, if needed, correct) their LaTeX system  ***
% *** with the testflow diagnostic prior to trusting their LaTeX platform ***
% *** with production work. IEEE's font choices can trigger bugs that do  ***
% *** not appear when using other class files.                            ***
% The testflow support page is at:
% http://www.michaelshell.org/tex/testflow/


%%*************************************************************************
%% Legal Notice:
%% This code is offered as-is without any warranty either expressed or
%% implied; without even the implied warranty of MERCHANTABILITY or
%% FITNESS FOR A PARTICULAR PURPOSE!
%% User assumes all risk.
%% In no event shall IEEE or any contributor to this code be liable for
%% any damages or losses, including, but not limited to, incidental,
%% consequential, or any other damages, resulting from the use or misuse
%% of any information contained here.
%%
%% All comments are the opinions of their respective authors and are not
%% necessarily endorsed by the IEEE.
%%
%% This work is distributed under the LaTeX Project Public License (LPPL)
%% ( http://www.latex-project.org/ ) version 1.3, and may be freely used,
%% distributed and modified. A copy of the LPPL, version 1.3, is included
%% in the base LaTeX documentation of all distributions of LaTeX released
%% 2003/12/01 or later.
%% Retain all contribution notices and credits.
%% ** Modified files should be clearly indicated as such, including  **
%% ** renaming them and changing author support contact information. **
%%
%% File list of work: IEEEtran.cls, IEEEtran_HOWTO.pdf, bare_adv.tex,
%%                    bare_conf.tex, bare_jrnl.tex, bare_jrnl_compsoc.tex
%%*************************************************************************

% Note that the a4paper option is mainly intended so that authors in
% countries using A4 can easily print to A4 and see how their papers will
% look in print - the typesetting of the document will not typically be
% affected with changes in paper size (but the bottom and side margins will).
% Use the testflow package mentioned above to verify correct handling of
% both paper sizes by the user's LaTeX system.
%
% Also note that the "draftcls" or "draftclsnofoot", not "draft", option
% should be used if it is desired that the figures are to be displayed in
% draft mode.
%
\documentclass[journal]{IEEEtran}
%\documentclass[11pt,draftclsnofoot,onecolumn]{IEEEtran}
%\documentclass[draftclsnofoot,onecolumn]{IEEEtran}
%\documentclass[20pt,draftclsnofoot, onecolumn]{IEEEtran}
%\documentclass[12pt,journal,draftcls,letterpaper,onecolumn]{IEEEtran}



\usepackage{multirow}
\usepackage{enumerate}


% If IEEEtran.cls has not been installed into the LaTeX system files,
% manually specify the path to it like:
% \documentclass[journal]{../sty/IEEEtran}





% Some very useful LaTeX packages include:
% (uncomment the ones you want to load)


% *** MISC UTILITY PACKAGES ***
%
%\usepackage{ifpdf}
% Heiko Oberdiek's ifpdf.sty is very useful if you need conditional
% compilation based on whether the output is pdf or dvi.
% usage:
% \ifpdf
%   % pdf code
% \else
%   % dvi code
% \fi
% The latest version of ifpdf.sty can be obtained from:
% http://www.ctan.org/tex-archive/macros/latex/contrib/oberdiek/
% Also, note that IEEEtran.cls V1.7 and later provides a builtin
% \ifCLASSINFOpdf conditional that works the same way.
% When switching from latex to pdflatex and vice-versa, the compiler may
% have to be run twice to clear warning/error messages.






% *** CITATION PACKAGES ***
%
\usepackage{cite}
% cite.sty was written by Donald Arseneau
% V1.6 and later of IEEEtran pre-defines the format of the cite.sty package
% \cite{} output to follow that of IEEE. Loading the cite package will
% result in citation numbers being automatically sorted and properly
% "compressed/ranged". e.g., [1], [9], [2], [7], [5], [6] without using
% cite.sty will become [1], [2], [5]--[7], [9] using cite.sty. cite.sty's
% \cite will automatically add leading space, if needed. Use cite.sty's
% noadjust option (cite.sty V3.8 and later) if you want to turn this off.
% cite.sty is already installed on most LaTeX systems. Be sure and use
% version 4.0 (2003-05-27) and later if using hyperref.sty. cite.sty does
% not currently provide for hyperlinked citations.
% The latest version can be obtained at:
% http://www.ctan.org/tex-archive/macros/latex/contrib/cite/
% The documentation is contained in the cite.sty file itself.






% *** GRAPHICS RELATED PACKAGES ***
%
\ifCLASSINFOpdf
  \usepackage[pdftex]{graphicx}
  % declare the path(s) where your graphic files are
  % \graphicspath{{../pdf/}{../jpeg/}}
  % and their extensions so you won't have to specify these with
  % every instance of \includegraphics
  % \DeclareGraphicsExtensions{.pdf,.jpeg,.png}
\else
  % or other class option (dvipsone, dvipdf, if not using dvips). graphicx
  % will default to the driver specified in the system graphics.cfg if no
  % driver is specified.
  \usepackage[dvips]{graphicx}
  % declare the path(s) where your graphic files are
  % \graphicspath{{../eps/}}
  % and their extensions so you won't have to specify these with
  % every instance of \includegraphics
  \DeclareGraphicsExtensions{.eps}
\fi
% graphicx was written by David Carlisle and Sebastian Rahtz. It is
% required if you want graphics, photos, etc. graphicx.sty is already
% installed on most LaTeX systems. The latest version and documentation can
% be obtained at:
% http://www.ctan.org/tex-archive/macros/latex/required/graphics/
% Another good source of documentation is "Using Imported Graphics in
% LaTeX2e" by Keith Reckdahl which can be found as epslatex.ps or
% epslatex.pdf at: http://www.ctan.org/tex-archive/info/
%
% latex, and pdflatex in dvi mode, support graphics in encapsulated
% postscript (.eps) format. pdflatex in pdf mode supports graphics
% in .pdf, .jpeg, .png and .mps (metapost) formats. Users should ensure
% that all non-photo figures use a vector format (.eps, .pdf, .mps) and
% not a bitmapped formats (.jpeg, .png). IEEE frowns on bitmapped formats
% which can result in "jaggedy"/blurry rendering of lines and letters as
% well as large increases in file sizes.
%
% You can find documentation about the pdfTeX application at:
% http://www.tug.org/applications/pdftex





% *** MATH PACKAGES ***
%
\usepackage[cmex10]{amsmath}
% A popular package from the American Mathematical Society that provides
% many useful and powerful commands for dealing with mathematics. If using
% it, be sure to load this package with the cmex10 option to ensure that
% only type 1 fonts will utilized at all point sizes. Without this option,
% it is possible that some math symbols, particularly those within
% footnotes, will be rendered in bitmap form which will result in a
% document that can not be IEEE Xplore compliant!
%
% Also, note that the amsmath package sets \interdisplaylinepenalty to 10000
% thus preventing page breaks from occurring within multiline equations. Use:
%\interdisplaylinepenalty=2500
% after loading amsmath to restore such page breaks as IEEEtran.cls normally
% does. amsmath.sty is already installed on most LaTeX systems. The latest
% version and documentation can be obtained at:
% http://www.ctan.org/tex-archive/macros/latex/required/amslatex/math/





% *** SPECIALIZED LIST PACKAGES ***
%
%\usepackage{algorithmic}
% algorithmic.sty was written by Peter Williams and Rogerio Brito.
% This package provides an algorithmic environment fo describing algorithms.
% You can use the algorithmic environment in-text or within a figure
% environment to provide for a floating algorithm. Do NOT use the algorithm
% floating environment provided by algorithm.sty (by the same authors) or
% algorithm2e.sty (by Christophe Fiorio) as IEEE does not use dedicated
% algorithm float types and packages that provide these will not provide
% correct IEEE style captions. The latest version and documentation of
% algorithmic.sty can be obtained at:
% http://www.ctan.org/tex-archive/macros/latex/contrib/algorithms/
% There is also a support site at:
% http://algorithms.berlios.de/index.html
% Also of interest may be the (relatively newer and more customizable)
% algorithmicx.sty package by Szasz Janos:
% http://www.ctan.org/tex-archive/macros/latex/contrib/algorithmicx/




% *** ALIGNMENT PACKAGES ***
%
\usepackage{array}
% Frank Mittelbach's and David Carlisle's array.sty patches and improves
% the standard LaTeX2e array and tabular environments to provide better
% appearance and additional user controls. As the default LaTeX2e table
% generation code is lacking to the point of almost being broken with
% respect to the quality of the end results, all users are strongly
% advised to use an enhanced (at the very least that provided by array.sty)
% set of table tools. array.sty is already installed on most systems. The
% latest version and documentation can be obtained at:
% http://www.ctan.org/tex-archive/macros/latex/required/tools/


\usepackage{mdwmath}
\usepackage{mdwtab}
% Also highly recommended is Mark Wooding's extremely powerful MDW tools,
% especially mdwmath.sty and mdwtab.sty which are used to format equations
% and tables, respectively. The MDWtools set is already installed on most
% LaTeX systems. The lastest version and documentation is available at:
% http://www.ctan.org/tex-archive/macros/latex/contrib/mdwtools/


% IEEEtran contains the IEEEeqnarray family of commands that can be used to
% generate multiline equations as well as matrices, tables, etc., of high
% quality.


\usepackage{eqparbox}
% Also of notable interest is Scott Pakin's eqparbox package for creating
% (automatically sized) equal width boxes - aka "natural width parboxes".
% Available at:
% http://www.ctan.org/tex-archive/macros/latex/contrib/eqparbox/





% *** SUBFIGURE PACKAGES ***
\usepackage[tight,footnotesize]{subfigure}
% subfigure.sty was written by Steven Douglas Cochran. This package makes it
% easy to put subfigures in your figures. e.g., "Figure 1a and 1b". For IEEE
% work, it is a good idea to load it with the tight package option to reduce
% the amount of white space around the subfigures. subfigure.sty is already
% installed on most LaTeX systems. The latest version and documentation can
% be obtained at:
% http://www.ctan.org/tex-archive/obsolete/macros/latex/contrib/subfigure/
% subfigure.sty has been superceeded by subfig.sty.



%\usepackage[caption=false]{caption}
\usepackage[caption=true]{caption}
\usepackage[font=footnotesize]{subfig}
% subfig.sty, also written by Steven Douglas Cochran, is the modern
% replacement for subfigure.sty. However, subfig.sty requires and
% automatically loads Axel Sommerfeldt's caption.sty which will override
% IEEEtran.cls handling of captions and this will result in nonIEEE style
% figure/table captions. To prevent this problem, be sure and preload
% caption.sty with its "caption=false" package option. This is will preserve
% IEEEtran.cls handing of captions. Version 1.3 (2005/06/28) and later
% (recommended due to many improvements over 1.2) of subfig.sty supports
% the caption=false option directly:
%\usepackage[caption=false,font=footnotesize]{subfig}
%
% The latest version and documentation can be obtained at:
% http://www.ctan.org/tex-archive/macros/latex/contrib/subfig/
% The latest version and documentation of caption.sty can be obtained at:
% http://www.ctan.org/tex-archive/macros/latex/contrib/caption/




% *** FLOAT PACKAGES ***
%
\usepackage{fixltx2e}
% fixltx2e, the successor to the earlier fix2col.sty, was written by
% Frank Mittelbach and David Carlisle. This package corrects a few problems
% in the LaTeX2e kernel, the most notable of which is that in current
% LaTeX2e releases, the ordering of single and double column floats is not
% guaranteed to be preserved. Thus, an unpatched LaTeX2e can allow a
% single column figure to be placed prior to an earlier double column
% figure. The latest version and documentation can be found at:
% http://www.ctan.org/tex-archive/macros/latex/base/



\usepackage{stfloats}
% stfloats.sty was written by Sigitas Tolusis. This package gives LaTeX2e
% the ability to do double column floats at the bottom of the page as well
% as the top. (e.g., "\begin{figure*}[!b]" is not normally possible in
% LaTeX2e). It also provides a command:
%\fnbelowfloat
% to enable the placement of footnotes below bottom floats (the standard
% LaTeX2e kernel puts them above bottom floats). This is an invasive package
% which rewrites many portions of the LaTeX2e float routines. It may not work
% with other packages that modify the LaTeX2e float routines. The latest
% version and documentation can be obtained at:
% http://www.ctan.org/tex-archive/macros/latex/contrib/sttools/
% Documentation is contained in the stfloats.sty comments as well as in the
% presfull.pdf file. Do not use the stfloats baselinefloat ability as IEEE
% does not allow \baselineskip to stretch. Authors submitting work to the
% IEEE should note that IEEE rarely uses double column equations and
% that authors should try to avoid such use. Do not be tempted to use the
% cuted.sty or midfloat.sty packages (also by Sigitas Tolusis) as IEEE does
% not format its papers in such ways.


%\ifCLASSOPTIONcaptionsoff
%  \usepackage[nomarkers]{endfloat}
% \let\MYoriglatexcaption\caption
% \renewcommand{\caption}[2][\relax]{\MYoriglatexcaption[#2]{#2}}
%\fi
% endfloat.sty was written by James Darrell McCauley and Jeff Goldberg.
% This package may be useful when used in conjunction with IEEEtran.cls'
% captionsoff option. Some IEEE journals/societies require that submissions
% have lists of figures/tables at the end of the paper and that
% figures/tables without any captions are placed on a page by themselves at
% the end of the document. If needed, the draftcls IEEEtran class option or
% \CLASSINPUTbaselinestretch interface can be used to increase the line
% spacing as well. Be sure and use the nomarkers option of endfloat to
% prevent endfloat from "marking" where the figures would have been placed
% in the text. The two hack lines of code above are a slight modification of
% that suggested by in the endfloat docs (section 8.3.1) to ensure that
% the full captions always appear in the list of figures/tables - even if
% the user used the short optional argument of \caption[]{}.
% IEEE papers do not typically make use of \caption[]'s optional argument,
% so this should not be an issue. A similar trick can be used to disable
% captions of packages such as subfig.sty that lack options to turn off
% the subcaptions:
% For subfig.sty:
% \let\MYorigsubfloat\subfloat
% \renewcommand{\subfloat}[2][\relax]{\MYorigsubfloat[]{#2}}
% For subfigure.sty:
% \let\MYorigsubfigure\subfigure
% \renewcommand{\subfigure}[2][\relax]{\MYorigsubfigure[]{#2}}
% However, the above trick will not work if both optional arguments of
% the \subfloat/subfig command are used. Furthermore, there needs to be a
% description of each subfigure *somewhere* and endfloat does not add
% subfigure captions to its list of figures. Thus, the best approach is to
% avoid the use of subfigure captions (many IEEE journals avoid them anyway)
% and instead reference/explain all the subfigures within the main caption.
% The latest version of endfloat.sty and its documentation can obtained at:
% http://www.ctan.org/tex-archive/macros/latex/contrib/endfloat/
%
% The IEEEtran \ifCLASSOPTIONcaptionsoff conditional can also be used
% later in the document, say, to conditionally put the References on a
% page by themselves.





% *** PDF, URL AND HYPERLINK PACKAGES ***
%
\usepackage{url}
% url.sty was written by Donald Arseneau. It provides better support for
% handling and breaking URLs. url.sty is already installed on most LaTeX
% systems. The latest version can be obtained at:
% http://www.ctan.org/tex-archive/macros/latex/contrib/misc/
% Read the url.sty source comments for usage information. Basically,
% \url{my_url_here}.


%add by me
\usepackage{multirow}
\usepackage{enumerate}
\usepackage{stfloats}
\usepackage{multicol}
\usepackage{algorithm}
\usepackage{amsmath}



%\usepackage[ruled]{algorithm2e}
%\usepackage{algorithmic}


%\usepackage{algorithmic,algorithm}

\usepackage{algpseudocode}
\usepackage{amsmath}
\usepackage{graphics}
\usepackage{epsfig}
\usepackage{multirow}
\usepackage{multicol}


\usepackage{subfigure}
\usepackage{multirow}
\usepackage{enumerate}
\usepackage{stfloats}

\usepackage{color}

\usepackage{float}


%
%\usepackage[T1]{fontenc}
%\usepackage[latin1]{inputenc}
%\usepackage[english]{babel}
%\usepackage[font=small,labelfont=bf]{caption}
\usepackage[font=footnotesize]{subfig}

%\usepackage{booktabs}
%\usepackage{subcaption}
%\usepackage{lipsum}



%\documentclass[draftclsnofoot,onecolumn]{IEEEtran}
%\documentclass[12pt,draftclsnofoot, onecolumn]{IEEEtran}
%\documentclass[12pt,journal,draftcls,letterpaper,onecolumn]{IEEEtran}

%\usepackage[linesnumbered,boxed]{algorithm2e}
\renewcommand{\algorithmicrequire}{\textbf{Input:}}  % Use Input in the format of Algorithm
\renewcommand{\algorithmicensure}{\textbf{Output:}} % Use Output in the format of Algorithm


% *** Do not adjust lengths that control margins, column widths, etc. ***
% *** Do not use packages that alter fonts (such as pslatex).         ***
% There should be no need to do such things with IEEEtran.cls V1.6 and later.
% (Unless specifically asked to do so by the journal or conference you plan
% to submit to, of course. )


% correct bad hyphenation here
\hyphenation{op-tical net-works semi-conduc-tor}


\begin{document}

%
% paper title
% can use linebreaks \\ within to get better formatting as desired
%\title{HEp-2 Cell Classification using Discriminative Texture and Shape Features}
%\title{HEp-2 Cell Classification by Fusing Multiple Sources of Information}
\title{HEp-2 Cell Classification via Fusing Texture and Shape Information}


%\title{HEp-2 Cell Classification using Multi-Resolution Texture and Discriminative Shape Features}
%
%
% author names and IEEE memberships
% note positions of commas and nonbreaking spaces ( ~ ) LaTeX will not break
% a structure at a ~ so this keeps an author's name from being broken across
% two lines.
% use \thanks{} to gain access to the first footnote area
% a separate \thanks must be used for each paragraph as LaTeX2e's \thanks
% was not built to handle multiple paragraphs
%

\author{Xianbiao Qi, Guoying Zhao, ~\IEEEmembership{Senior Member,~IEEE}, Chun-Guang Li, ~\IEEEmembership{Member,~IEEE}, Jun Guo, \\ Matti Pietik{\"a}inen,~\IEEEmembership{Fellow,~IEEE}


%Xianbiao Qi, Guoying Zhao, ~\IEEEmembership{Senior Member,~IEEE,}, Chun-Guang Li, ~\IEEEmembership{Member,~IEEE,}, and \\Matti Pietik{\"a}inen,~\IEEEmembership{Fellow,~IEEE



%        Rong Xiao,
%        Yu Qiao, ~\IEEEmembership{Member,~IEEE}
%        Jun Guo,
%        %Lei Zhang, ~\IEEEmembership{Senior Member,~IEEE}
%        and Xiaoou Tang,~\IEEEmembership{Fellow,~IEEE}% <-this % stops a space


\thanks{Xianbiao Qi, Chun-Guang Li and Jun Guo are with the School of Information and Communication Engineering, Beijing University of Posts and Telecommunications, Beijing 100876, China. Xianbiao Qi is also with the Center of Machine Vision Research, University of Oulu, PO Box 4500, FIN-90014. E-mails:
qixianbiao@gmail.com, lichunguang@bupt.edu.cn, guojun@bupt.edu.cn.}
\thanks{Guoying Zhao and Matti Pietik{\"a}inen are with the Center of Machine Vision Research, University of Oulu, PO Box 4500, FIN-90014, Finland. E-mails: gyzhao@ee.oulu.fi, mkp@ee.oulu.fi.} }



%\thanks{Rong Xiao is with the Microsoft Corporation, USA. E-mail: rxiao@microsoft.com.}
%\thanks{Yu Qiao and Xiaoou Tang are with the Shenzhen Key Laboratory for Computer Vision and Pattern Recognition, Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences, Shenzhen
%518055, China. E-mail: yu.qiao@siat.ac.cn, xtang@ie.cuhk.edu.hk.}}
%\thanks{Lei Zhang is with Microsoft Research Asia, Beijing, 100080, P.R. China. E-mail: leizhang@microsoft.com.}}
%\thanks{M. Shell is with the Department
%of Electrical and Computer Engineering, Georgia Institute of Technology, Atlanta,
%GA, 30332 USA e-mail: (see http://www.michaelshell.org/contact.html).}% <-this % stops a space
%\thanks{J. Doe and J. Doe are with Anonymous University.}% <-this % stops a space
%\thanks{Manuscript received April 19, 2005; revised January 11, 2007.}}

% note the % following the last \IEEEmembership and also \thanks -
% these prevent an unwanted space from occurring between the last author name
% and the end of the author line. i.e., if you had this:
%
% \author{....lastname \thanks{...} \thanks{...} }
%                     ^------------^------------^----Do not want these spaces!
%
% a space would be appended to the last name and could cause every name on that
% line to be shifted left slightly. This is one of those "LaTeX things". For
% instance, "\textbf{A} \textbf{B}" will typeset as "A B" not "AB". To get
% "AB" then you have to do: "\textbf{A}\textbf{B}"
% \thanks is no different in this regard, so shield the last } of each \thanks
% that ends a line with a % and do not let a space in before the next \thanks.
% Spaces after \IEEEmembership other than the last one are OK (and needed) as
% you are supposed to have spaces between the names. For what it is worth,
% this is a minor point as most people would not even notice if the said evil
% space somehow managed to creep in.



% The paper headers
%\markboth{Under Review}%
%{Shell \MakeLowercase{\textit{et al.}}: Bare Demo of IEEEtran.cls for Journals}
% The only time the second header will appear is for the odd numbered pages
% after the title page when using the twoside option.
%
% *** Note that you probably will NOT want to include the author's ***
% *** name in the headers of peer review papers.                   ***
% You can use \ifCLASSOPTIONpeerreview for conditional compilation here if
% you desire.




% If you want to put a publisher's ID mark on the page you can do it like
% this:
%\IEEEpubid{0000--0000/00\$00.00~\copyright~2007 IEEE}
% Remember, if you use this you must call \IEEEpubidadjcol in the second
% column for its text to clear the IEEEpubid mark.



% use for special paper notices
%\IEEEspecialpapernotice{(Invited Paper)}




% make the title area
\maketitle
%\boldmath
\begin{abstract}

Indirect Immunofluorescence (IIF) HEp-2 cell image is an effective evidence for diagnosis of autoimmune diseases. Recently computer-aided diagnosis of autoimmune diseases by IIF HEp-2 cell classification has attracted great attention. However the HEp-2 cell classification task is quite challenging due to large intra-class variation and small between-class variation.
In this paper we propose an effective and efficient approach for the automatic classification of IIF HEp-2 cell image by fusing multi-resolution texture information and richer shape information. To be specific, we propose to: a) capture the multi-resolution texture information by a novel Pairwise Rotation Invariant Co-occurrence of Local Gabor Binary Pattern (PRICoLGBP) descriptor, b) depict the richer shape information by using an Improved Fisher Vector (IFV) model with RootSIFT features which are sampled from large image patches in multiple scales, and c) combine them properly. We evaluate systematically the proposed approach on the IEEE International Conference on Pattern Recognition (ICPR) 2012, IEEE International Conference on Image Processing (ICIP) 2013 and ICPR 2014 contest data sets. The experimental results for the proposed methods significantly outperform the winners of
ICPR 2012 and ICIP 2013 contest, and achieve comparable performance with the winner of the newly released ICPR 2014 contest.


%
%
%This paper proposes an effective method for automatic classification of Indirect Immunofluorescence (IIF) HEp-2 cell images via using discriminative
%texture and shape features. \textcolor{red}{Our contributions to HEp-2 cell classification are three folds: (1) we propose a novel Pairwise Rotation Invariant
%Co-occurrence of Local Gabor Binary Pattern (PRICoLGBP) feature to represent discriminative texture information. Compared to its counterpart (PRICoLBP),
%the proposed PRICoLGBP can capture multi-resolution texture information more effectively.}
%(2) we propose an effective method to depict discriminative shape information. We fully investigate three key steps in this framework, including patch sampling, feature description and encoding method. We propose to sample large patches that covers about $\frac{1}{3}$ area of the whole image in multiple Gaussian scales. Then, we use RootSIFT feature to depict the sampled patches. Finally, motivated by the success of recent
%computer vision technology, we propose to use Improved Fisher Vector (IFV) to encode the features. The IFV can preserve the strong discriminative power of the used features. \textcolor{red}{(3) The proposed discriminative
%texture and shape features prove to be complementary in the experiments.}
%
%We have evaluated the proposed methods on IEEE International Conference on Pattern Recognition (ICPR) 2012, IEEE International Conference on Image Processing (ICIP) 2013 and \textcolor{red}{ICPR 2014 contest data sets. The experimental results for
%the proposed methods significantly outperform the winners of
%ICPR 2012 and ICIP 2013 contest, and achieve comparable performance with the winner of the newly released ICPR 2014 contest.}










%Specifically, on ICPR
%contest, our methods improves the ICPR winner by about 7\%, and
%on ICIP 2013 contest \textcolor{red}{data set}, our method improves the accuracy of the winner's method
%by 4.1\%. It should be noted that the difference between the
%$1$-st and $2$-nd places in ICIP 2013 \textcolor{red}{contest} is only about 0.1\%.



%In this paper, we firstly introduce an effective HEp-2 cell classification system that ranks the first in HEp-2 Cells Classification contest in the international conference of image processing 2013 (ICIP 2013).
%Then, we further introduce some new methods for HEp-2 cell classification motivated by some new development of the computer vision area.
%indirect immunofluorescence (IIF) image analysis


\end{abstract}



% IEEEtran.cls defaults to using nonbold math in the Abstract.
% This preserves the distinction between vectors and scalars. However,
% if the journal you are submitting to favors bold math in the abstract,
% then you can use LaTeX's standard command \boldmath at the very start
% of the abstract to achieve this. Many IEEE journals frown on math
% in the abstract anyway.

% Note that keywords are not normally used for peerreview papers.
\begin{IEEEkeywords}
HEp-2 Cell Classification, PRICoLGBP, Improved Fisher Vector, Multi-resolution Texture Descriptor, Discriminative Shape Feature.
\end{IEEEkeywords}



\input{introduction.tex}

\input{mainbody.tex}

\input{dataset.tex}

\input{experiments.tex}

\input{conclusion.tex}




\section{Acknowledgment}
The authors would like to thank Andrea Vedaldi for sharing the Vlfeat toolbox, and Rong-En Fan et al. for sharing the Liblinear toolbox. We also want to thank the organizer of ICPR 2012 and ICIP 2013 HEp-2 Cell classification contests for releasing the HEp-2 cell data sets. These two contests substantially put forward the research on HEp-2 cell classification. This work was supported by the Academy of Finland and Infotech Oulu.





%
%\cite{qi2012pairwise} \cite{shen2006review} \cite{ojala2002multiresolution} \cite{guo2010completed}, \cite{zhao2012rotation}, \cite{ahonen2009rotation}
%\cite{nosaka2012feature},  \cite{guo2010rotation}, \cite{foggia2013benchmarking} \cite{nosaka2013hep} \cite{shen2013hep} \cite{theodorakopoulos2013hep} \cite{kong2013hep}, \cite{sanchez2013image},
%\cite{lowe2004distinctive}, \cite{percannella2012classification}, \cite{foggia2010early}, \cite{vedaldi2010vlfeat}, \cite{fan2008liblinear} \cite{vedaldi11efficient} \cite{varma2005statistical} \cite{varma2009statistical} \cite{agrawal2013hep} \cite{mivia2010} \cite{icpr2012hep2} \cite{soda2009multiple} \cite{han2007rotation} \cite{haralick1973textural} \cite{rigon2007indirect}
%\cite{zhang2005local} \cite{shen20083d} \cite{csurka2004visual} \cite{sorensen2008texture} \cite{srensen2010quantitative} \cite{pietikainen2011computer} \cite{qi2013bmvc1} \cite{sorensen2012texture}
%\cite{rigon2007indirect} \cite{hiemann2007automatic} \cite{nanni2010local} \cite{gabor1946theory} \cite{daugman1985uncertainty} \cite{wimmer2013scale} \cite{hafner2012color} \cite{hegenbart2013customised}
%
%\cite{simonyan2013fisher} \cite{arandjelovic2012three}, \cite{han2007rotation}, \cite{movellan2002tutorial}
%
%


% For peer review papers, you can put extra information on the cover
% page as needed:
% \ifCLASSOPTIONpeerreview
% \begin{center} \bfseries EDICS Category: 3-BBND \end{center}
% \fi
%
% For peerreview papers, this IEEEtran command inserts a page break and
% creates the second title. It will be ignored for other modes.
%\IEEEpeerreviewmaketitle
%
%{
%%\scriptsize
%%\large
%\bibliographystyle{IEEEtran}
%\bibliography{egbib}
%}


{
\large
\bibliographystyle{IEEEtran}
\bibliography{egbib}
}



%
%
%
%\begin{IEEEbiography}[{\includegraphics[width=1.05in,height=1.35in,clip,keepaspectratio]{figures/xianbiao.eps}}]%
%{Xianbiao Qi} received his B.E. degree in information engineering from Beijing University of Posts and Telecommunications (BUPT) in 2008. He is currently pursuing his Ph.D. degree at BUPT. He visited the Web Search and Mining Group in Microsoft Research Asia (MSRA) from Jan 2011 to May 2012. Now, he is visiting the Center of Machine Vision group in Oulu university of Finland. His main research interests lie on computer vision, pattern recognition and medical image analysis. Specifically, he focus on local feature design, texture and material classification, object recognition and medical image classification.
%\end{IEEEbiography}
%
%
%
%\begin{IEEEbiography}[{\includegraphics[width=1.05in,height=1.35in,clip,keepaspectratio]{figures/guoying.eps}}]%
%{Guoying Zhao} received the Ph.D. degree in computer science from the Chinese Academy of Sciences, Beijing, China, in 2005. From July 2005 to August 2010, she was a Senior Researcher with the Center for Machine Vision Research, University of Oulu, Oulu, Finland, where she has been an Associate Professor since September 2010. She has authored over 70 papers in journals and conferences, and has served as a reviewer for many journals and conferences. Her research interests include gait analysis, dynamic-texture recognition, facial-expression recognition, human motion analysis, and person identification. Dr. Zhao was a co-chair of the European Conference on Computer Vision 2008 Workshop on Machine Learning for Vision-based Motion Analysis (MLVMA) and the MLVMA workshop at the ICCV 2009 and the IEEE Conference on Computer Vision and Pattern Recognition 2011.
%\end{IEEEbiography}
%
%
%\begin{IEEEbiography}[{\includegraphics[width=1in,height=1.25in,clip,keepaspectratio]{figures/Chun-GuangLi2012.eps}}]%
%{Chun-Guang Li} received the B.E. degree in telecommunication engineering from the Jilin University in 2002 and the Ph.D. degree in signal processing from the Beijing University of Posts and Telecommunications (BUPT) in 2007. Currently, he is an Assistant Professor with the School of Information and Communication Engineering, BUPT. From Jul. 2011 to Apr. 2012, he visited the Visual Computing group, Microsoft Research Asia. From Dec. 2012 to Nov. 2013, he visited the Vision, Dynamics, and Learning lab, the Johns Hopkins University. His research interests are statistical machine learning, compressive sensing, and pattern recognition. He is a member of the IEEE, ACM, and CCF.
%
%\end{IEEEbiography}
%
%
%%
%%\begin{IEEEbiography}[{\includegraphics[width=1in,height=1.25in,clip,keepaspectratio]{figures/Chun-GuangLi2012_greyscale.eps}}]%
%%{Chun-Guang Li} received his B.E. degree from Jilin University in 2002 and PhD degree from Beijing University of Posts and Telecommunications (BUPT) in 2007. Currently, he is a lecturer with the School of Information and Communication Engineering, BUPT. From July 2011 to April 2012, he visited the Visual Computing group, Microsoft Research Asia. From Dec. 2012 to Nov. 2013, he visited the Vision, Dynamics and Learning lab, Johns Hopkins University. His research interests are statistical machine learning, compressive sensing, and pattern recognition. He is a member of the IEEE.
%%\end{IEEEbiography}
%
%\begin{IEEEbiography}[{\includegraphics[width=1in,height=1.35in,clip,keepaspectratio]{figures/junguo.eps}}]%
%{Jun Guo} received B.E. and M.E. degrees from Beijing University of Posts and Telecommunications (BUPT), China, in 1982 and 1985, respectively, and a Ph.D. degree from the Tohuku-Gakuin University, Japan, in 1993. At present he is a professor and vice president of BUPT. His publications cover IEEE Transaction on Pattern Analysis Machine Intelligence, Pattern Recognition, CVPR, ICCV, ECCV and SIGIR. His research interests include pattern recognition theory and application, information retrieval, content-based information security, and network management.
%\end{IEEEbiography}
%
%
%
%
%
%\begin{IEEEbiography}[{\includegraphics[width=1.05in,height=1.35in,clip,keepaspectratio]{figures/matti.eps}}]%
%{Matti Pietik�inen} (SM�95�F�12) received the Doctor of Science in Technology degree from the University of Oulu, Oulu, Finland, in 1982. In 1981, he established the Machine Vision Group, University of Oulu. Currently, he is a Professor of information engineering, the Scientific Director of the Infotech Oulu Research Center, and the Director of the Center for Machine Vision Research with the University of Oulu. From 1980 to 1981 and from 1984 to 1985, he was with the Computer Vision Laboratory, University of Maryland, College Park. He has made pioneering contributions, e.g., to local binary pattern (LBP) methodology, texture-based image and video analysis, and facial-image analysis. He has authored over 260 refereed papers in international journals, books, and conference proceedings, and about 100 other publications or reports. He has authored the book entitled �Computer Vision Using Local Binary Patterns� published by Springer in 2011. His research is frequently cited, and its results are used in various applications around the world.
%
%Dr. Pietik�inen was an associate editor of the IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE and of Pattern Recognition journals, and currently serves as an associate editor of the Image and Vision Computing journal. From 1989 to 1992, he was the president of the Pattern Recognition Society of Finland. From 1989 to 2007, he served as a member of the Governing Board of the International Association for Pattern Recognition (IAPR) and became one of the founding fellows of the IAPR in 1994. He regularly serves on program committees of the top conferences and workshops of his field. He has lectured tutorials on LBP-based image and video descriptors at the Scandinavian Conference on Image Analysis 2005, International Conference on Pattern Recognition 2006, International Conference on Computer Vision 2009, Computer Vision and Pattern Recognition 2011, and International Conference on Image Processing 2011 conferences.
%\end{IEEEbiography}
%
%




\end{document}




introduction.tex
\section{Introduction}
\label{sec:introduction}
Indirect immunofluorescence image (IIF) is an image analysis based diagnostic methodology to determine the existence of autoimmune diseases. Recently, it has attracted great attention due to its effectiveness.
More and more pattern recognition techniques \cite{foggia2013benchmarking, foggia2014PR, foggia2010early, percannella2012classification, theodorakopoulos2012hep, nosaka2013hep, theodorakopoulos2013hep, kong2013hep, shen2013hep, wiliem2013classification, han2014high, faraki2014fisher}  have been developed to make computer-aided diagnosis (CAD) of autoimmune diseases.
Before, manual labeling is the main approach for classifying the fluorescence patterns. However, the process of human labeling requires high expert knowledge, and meanwhile, it is also time consuming. Thus, to design a discriminative and robust HEp-2 cell classification system is extremely important.


The HEp-2 cell classification task is challenging due to large intra-class and small between-class variations regardless of its importance. As shown in Fig. \ref{fig:icip}, the ``Intermediate'' and ``Positive'' cells from same categories have large variations, the ``Positive'' cells in raw images can be seen clearly, but the ``Intermediate'' cells can not be seen clearly. Meanwhile, some categories share similar shapes, such as the categories ``Homogeneous'' and ``Speckled'', and some categories show similar textures, such as the categories ``Nucleolar'' and ``Golgi''.





\begin{figure}[t]
\begin{center}
\small
 \includegraphics[width=0.95\linewidth]{icip.eps}%{figures/examples.eps}
\end{center}
   \caption{Sample images from all six categories in ICIP 2013 contest data set with each column from one category. The six categories are ``Homogeneous'', ``Speckled'', ``Nucleolar'', ``Centromere'', ``NuMem'' and ``Golgi''. The first four rows show the ``intermediate'' samples, and the last two rows show the ``positive'' samples.}
\label{fig:icip}
\end{figure}

Recent ICPR 2012, ICIP 2013 and ICPR 2014 HEp-2 cell classification contests \cite{icpr2012hep2, icip2013, icpr2014hep2} have greatly put forward the development of HEp-2 cell analysis. Many features, image representation, classification methods were proposed or applied to this task. Currently, texture-based methods are the most widely used in this area. Local Binary Pattern (LBP) \cite{ojala1996comparative, ojala2002multiresolution, pietikainen2011computer} is widely recognized as a discriminative texture descriptor, and widely used in face recognition \cite{ahonen2006face}, static and dynamic texture classification \cite{ojala2002multiresolution, zhao2007dynamic}. Co-occurrence of adjacent LBP (CoALBP) \cite{nosaka2013hep}, Gradient-oriented Co-occurrence of LBPs (GoC-LBPs) \cite{theodorakopoulos2013hep} and pairwise rotation invariant co-occurrence of LBP (PRICoLBP) \cite{qi2014pairwise} are three of the best performing LBP variants in HEp-2 cell classification.  Besides of these three LBP variants, original LBP \cite{ojala2002multiresolution}, Completed LBP (CLBP) \cite{guo2010completed} were also used in the contests. Besides of LBP based texture features, some other famous texture features, such as Maximum Response Filter Banks (e.g. MR8) \cite{varma2005statistical}, Gray-Level Co-occurrence Matrices (GLCM) \cite{haralick1973textural}, Wavelet \cite{gabor1946theory}, were also used in this task. We also observed that Bag of Word \cite{csurka2004visual} model had been applied this task. %In BoW framework, several works focused on dictionary learning.





However, regardless of big improvement of classification accuracy in the past few years, previous works on HEp-2 cell classification task still have some limitations. Three key limitations are shown as follows:
\begin{itemize}

\item  Previous methods pay less attention to the multi-resolution texture information. Although texture information is widely studied, the influence of multi-resolution texture analysis to HEp-2 cell classification task is unknown.
\item Few works focus on capturing discriminative shape information. As far as we known, Vestergaard et al. \cite{Vestergaard2014} was the only work that explicitly explored the shape information in HEp-2 cell classification. Their work is different from the widely used Bag of Words (BOW) framework that our work is built on.
\item  The texture and shape information were considered individually, but they may be complementary to each other in practice. Thus, it will be interesting to investigate their complementary properties between them.

\end{itemize}



In this work, we attempt to address the pending issues mentioned above and hence our contributions are highlighted as follows:
\begin{itemize}
 \item  We explore the effect of multi-resolution texture for HEp-2 cell classification. To be specific, we capture the multi-resolution texture information by a novel Pairwise Rotation Invariant Co-occurrence of Local Gabor Binary Pattern (PRICoLGBP) descriptor, which is able to capture multi-resolution texture information effectively.
 \item We propose an effective method to depict the richer shape information by using an Improved Fisher Vector (IFV) model with RootSIFT features. Different from previous work, we extract local features from large image patches in multiple scales.
 \item We investigate the complementary effect of texture and shape information. By combining the multi-resolution texture and richer shape information, we yield superior classification accuracy. Compared with the winner of ICPR 2012 contest, our methods improves the accuracy of the winner by about 7\%. Compared with the winner of ICIP 2013 contest, our method obtains 4\% higher accuracy. Our method also achieves comparable performance to the winner of the newly release ICPR 2014 contest.
\end{itemize}


%To solve these limitations, \textcolor{blue}{we make the following three contributions to the HEp-2 cell classification:}
%\begin{itemize}
% \item  \textcolor{blue}{We propose a novel Pairwise Rotation Invariant Co-occurrence of Local Gabor Binary Pattern (PRICoLGBP) feature to capture multi-resolution texture information.} The PRICoLBP is good at depicting texture in small scales, such as $LBP(8, 1)$ and $LBP(8, 2)$, but has some limitations in depicting larger scales. The fusion of Gabor filters and PRICoLBP can facilitate the latter to capture multi-resolution texture information more effectively.
%
% \item We propose an effective method to depict discriminative shape information. We fully investigate three key steps in the framework, including patch sampling, feature description and encoding methods. We firstly propose to sample large patches that cover about $\frac{1}{3}$ of the whole image, and then use RootSIFT to depict the sampled patches and finally apply Improved Fisher Vector (IFV) \cite{perronnin2010improving} to encode the features. The IFV can well preserve the discriminative power of the used features.
%
%% We propose to use the RootSIFT \cite{arandjelovic2012three} on large patches to capture discriminative shape information and use Improved Fisher Vector (IFV) \cite{perronnin2010improving} \cite{vedaldi2010vlfeat} encoding to preserve the strong discrimination of the proposed feature.  We believe that, for IIF cell classification, texture information dominates the local regions (such as 3 and 5 regions), and shape information becomes stronger along with the increase of the patch size. Thus, we propose to sample the RankSIFT feature on large patches. Meanwhile, recent research in computer vision area proves that the IFV encoding method can effectively preserve the discrimination of the features.
%
% \item  \textcolor{blue}{We achieve the state-of-the-art performance by using the discriminative and complementary texture and shape features. Compared with the winner of ICPR 2012 contest, our methods improves the accuracy of the winner by about 7\%. Compared with the winner of ICIP 2013 contest, our method obtains 4\% higher accuracy. Our method also achieves comparable performance to the winner of the newly release ICPR 2014 contest.}
%
%\end{itemize}





















%
%This paper proposes an effective method for automatic clas-
%sification of HEp-2 cell classification via using discriminative tex-
%ture and shape features. Firstly, we propose a novel Gabor-based
%Co-occurrence of LBPs (Gabor-CoLBP) feature to represent
%texture information. Compared with traditional co-occurrence of
%LBPs, the proposed Gabor-CoLBP can capture multi-resolution
%textyure information. Secondly, inspired by the success of recent
%computer vision technology, we propose to use RootSIFT and
%Fisher Vector to capture the shape information. We extract the
%RootSIFT feature on some large patches. The patch size is about
%50% of the whole image. In the way, the RootSIFT can well
%capture the shape information.
%
%We have evaluated the proposed methods on the ICPR 2012
%and ICIP 2013 contest datasets. The experimental results for
%the proposed methods significantly outperform the winner of
%ICPR 2012 and ICIP 2013 contest. Specifically, on ICPR contest,
%our methods improves the ICPR winner for 10%, and on ICIP
%2013 contest dataset, our method improves the winner for 5%.
%It should be noted that the difference between the first place and
%the second place is 0.5%


The rest of the paper is organized as follows. We firstly review the state-of-the-art methods in the HEp-2 cell classification area in Sec. \ref{sec:relatedworks}. Then, we present the proposed texture and shape features in detail in Sec. \ref{sec:mainbody}. The used data sets are introduced in Sec. \ref{sec:dataset}. In Sec. \ref{sec:experiments}, we firstly
 give a comprehensive experimental evaluations of properties of the proposed discriminative texture and shape methods, and then compare it with some state-of-the-art methods. Finally, we give a conclusion in Sec. \ref{sec:conclusion}.


\section{Related Works}
\label{sec:relatedworks}
%\textcolor{blue}{In this section, we will firstly introduce the methods of the $1$-st and $2$-nd places in ICPR 2012 contest, then describe the approaches of $1$-st and $2$-nd places in ICIP 2013 contest, and then introduce two well-performing works that are highly relevant to this paper, and finally give some remarks about existing HEp-2 cell classification methods.}

\subsection{Best Performing Methods in ICPR 2012 Contest}

{\bf{Nosaka}} et al. \cite{nosaka2013hep}-the winner of ICPR 2012-only used the green channel in their method. The image was filtered by a Gaussian function to remove the noise. To improve the robustness to image rotation, they manually rotated the image to 9 orientations. Then, they extracted co-occurrence of adjacent LBP (CoALBP) features for all images (including the original images and the manually created images). Finally, they trained a linear Support Vector Machine (SVM) classifier.

  The success of Nosaka's methods is due to the following three aspects:
\begin{itemize}
 \item  Strong discriminative of CoALBP: the CoALBP was built on LBP that proves to be a powerful texture descriptor. Moreover, to capture strong spatial layout information, the CoALBP used 10 templates.

 \item  Green channel used: Among all the three channels, green channel was much stronger than the red and blue channels. Using gray-scale image would weaken the texture information in the green channel.

 \item  Manually creating many rotated training samples:  To improve the robustness of CoALBP to image rotation, they manually rotate the imaged to 9 orientations, and created 9 new rotated training samples.

\end{itemize}
  Regardless of its success on ICPR 2012 contest, this method also has some limitations. Firstly, since the CoALBP itself is not rotation invariant, thus, the CoALBP is not robust to image rotation although Nosaka et al. try to improve the CoALBP's robustness to rotation by manually creating more rotated training samples. Secondly, the discriminative power of CoALBP is limited due to that the CoALBP is built on the co-occurrence of two LBPs with four neighbors. The LBP(4, 1) is usually considered to be less discriminative than the LBP(8, 1).

{\bf{Kong}} et al. \cite{kong2013hep}-the second place of ICPR 2012- adopt Varma's MR8 method to extract the texture feature. The local regions were normalized before the filter responses are applied. After feature extraction, they trained a global texton dictionary using K-means clustering. Thus, each image could be represented as a frequency histogram of textons. They also used a pyramid histogram of oriented gradients (PHOG) \cite{bosch2007representing} feature to depict the shape information. The texture and shape histogram were concatenated with different weights. Finally, they used a K-Nearest Neighbor (KNN) classifier with ${\chi}^2$ distance.






%We believe the following two aspects may greatly affect the final classification accuracy. First, the used shape and texture feature may not be discriminative enough. Second, it has been proved in many applications, KNN classifier always shows worse performance compared with Support Vector Machine (SVM) classifier.

\subsection{Best Performing Methods in ICIP 2013 Contest}

{\bf{Shen}} et al. \cite{icip2013}-the winner of ICIP 2013- combined the the original PRICoLBP and the Bag of SIFT feature. For the PRICoLBP feature, they used 10 templates. The dimension of the PRICoLBP for each template is 590. Thus, the total dimension of their used PRICoLBP\footnote{http://qixianbiao.github.io/} feature is 5900. For the Bag of SIFT feature, following the traditional bag of words model, they created 1024 words using K-means clustering. Finally, they concatenated these two features and used linear SVM (Support Vector Machine) with square root features.

The success of this method is due to the following three aspects. Firstly, the PRICoLBP is good at capturing the texture information, meanwhile, as argued in \cite{qi2014pairwise}, when the shape structures are
strong in the data set, the utilization of 10 templates significantly improves the performance of 2 templates. Secondly, the bag of SIFT is good at capturing the global texture and shape information. Finally, the square root normalization of the feature is an
effective method for linear SVM. The square root normalization has proved to be effective in many computer vision works \cite{vedaldi11efficient}.

{\bf{Vestergaard}} et al. \cite{Vestergaard2014}-the merit winner of ICIP 2013- adopted a standard pipeline for the supervised image classification: preprocessing of the images, feature extraction and classification. A two-stage preprocessing method was exploited. First, each image $I$ was augmented with its logarithmic representation $loge(I)$. Then, the logarithmic representation was mapped linearly to [0,1]. For the feature extraction, Vestergaard et al. extracted three kinds of features including: 1) the ``Intersity'' of each image (Negative/Intermediate/Positive) as an integer flat, 2) morphological features extracted from the provided mask (containing the area of the mask region, eccentricity, major and minor axis length, perimeter); and 3) the donut-like shape index histogram feature (for both image representations). For the classification, Vestergaard et al. used a RBF kernel SVM.

\subsection{Best Performing Methods in ICPR 2014 Contest}
{\bf{Manivannan}} et al. \cite{manivannan2014hep} ranked 1st in the newly released ICPR 2014 HEp-2 cell classification contest \cite{icpr2014hep2}. Their method can be summarized into the following steps:
\begin{itemize}
\item[a)] Rotating the images to four orientations (0, 90, 180, 270) respectively;
\item[b)] Dense sampling of multi-scale patches ($12 \times 12, 16 \times 16, 20 \times 20$);
\item[c)] Extraction of four types of features (Multi-resolution local patterns (mLP), Root-SIFT (rSIFT), Random projections (RP), Intensity histogram (IH));
\item[d)] Feature encoding with Locality-constrained Linear Coding (LLC) for four types of features and four orientations individually. Thus, $4\times 4 = 16$ histograms can be obtained;
\item[e)] Training 16 classifiers with linear SVM and Classification based on 16 classifiers.

\end{itemize}

\subsection{Other Relevant and Well-Performing Methods}
%\textcolor{blue}{In this subsection, we introduce several approaches that are highly relevant with our methods. These mentioned methods below did not show in above-mentioned contest, but demonstrates good performance on HEp-2 cell classification task.}

{\bf{Theodorakopoulos}} et al. \cite{theodorakopoulos2013hep} proposed a sparse representation of textural features which were fused into dissimilarity space. Along with a multivariate distribution of SIFT feature, Theodorakopoulos et al. \cite{theodorakopoulos2013hep} proposed a Gradient-oriented Co-occurrence of LBPs which is considered in \cite{theodorakopoulos2013hep} as a relaxed variation of the PRICoLBP. The descriptors were fused while creating a dissimilarity representation of an image. Finally, a sparse representation-based classification scheme was used for the classification.

In \cite{theodorakopoulos2013hep}, the usage of SIFT feature was in a simple manner. Simple multivariate distribution of SIFT feature was used. Meanwhile, the used GoC-LBP was not robust to image rotation. Since the GoC-LBP was built on the co-occurrence of two uniform LBPs,
its dimension ($59\times 59 = 3481$) was higher than PRICoLBP (590).

{\bf{Faraki}} et al. \cite{faraki2014fisher} extended the traditional bag-of-word (BOW) from Euclidean space to non-Euclidean Riemanian manifolds that is an intrinsic bag of Riemannian words (BoRW). The BOW model has been applied to HEp-2 cell in \cite{wiliem2013classification} before.
Faraki et al. also proposed Fisher Tensor to encode higher statistics information when building the histogram for the images. The Fisher Tensor can be seen as a Riemannian version of Fisher Vector \cite{perronnin2010improving}. Their proposed BoRW and its extension with Fisher Tensor in \cite{faraki2014fisher} demonstrate great performance on both HEp-2 cell classification and texture classification tasks.




%
%
%%It is unclear of how much the 'Intersity' of each image and morphological features affect the final performance, but it should be pointed out that the morphological features may be associated with the captured devices and image resolution.
%
%
%\subsection{Remarks}
%The above-mentioned works on HEp-2 cell classification have significantly put forward the research on this area. We can find that, among the above-mentioned methods, the texture based methods dominate the HEp-2 cell classification task. For instance, Nosaka et al., Theodorakopoulos et al. and Shen et al. used the LBP based methods, and Kong et al. used the MR8 texture textons. On the other hand, the method of Vestergaard et al. can be seen as a great attempt to depict shape information.
%% Regardless the success of the up-mentioned methods, they still have some drawbacks. Multi-resolution information has been ignored. Secondly, the shape information. Thirdly, the texture and shape information is individually considered although Shen's method has used texture information and some potential shape information.
%
%  However, regardless of the success of the above-mentioned methods, there still exists some limitations in the current HEp-2 Cell classification methods.  Firstly, previous methods pay less attention to the multi-resolution texture information although texture information proves to be effective in this task. Secondly, few works focus on capturing the shape information, but actually, the shape information is discriminative. Thirdly, the texture and shape information is usually considered individually. Although Shen's method implicitly used texture information and some potential shape information, their method is directly based on the existing methods, and didn't explore the texture and shape information deeply.
%
%  In this paper, we improve the HEp-2 Cell classification methods by mining the above-mentioned three aspects.

mainbody.tex
\section{Hep-2 Cell Classification Using Discriminative Texture and Shape Features}
\label{sec:mainbody}
This section consists of three subsections. In the first part, we introduce one novel multi-resolution texture feature. In the second part, we present our approach for depicting discriminative shape information. Finally, we describe the normalization and classification methods.


\subsection{Discriminative Texture Feature}


\subsubsection{\bf{Local Binary Pattern}}



Local Binary Pattern (LBP) that was firstly proposed by Ojala et al. \cite{ojala2002multiresolution} is considered as a simple and effective texture descriptor.
For any pixel $A$ in an image, we can compute its LBP pattern by thresholding the pixel values of its circularly symmetric
$P$ neighbors with the pixel value of the central point $A$. The LBP of pixel $A$ can be defined as
follows:
\begin{equation}\nonumber
    \text{LBP}_{P,R}(A) = \sum_{i=0}^{P-1} s(V_i - V_c)2^i, \ \
    s(x) = \begin{cases}
    1, &  x \ge 0 \\
    0, &  x < 0, \\
    \end{cases}
\end{equation}
where $P$ is the number of the neighbors, $R$ is the radius, $V_c$ is the pixel value of point $A$, and $V_i$ is the pixel value of point $A$'s $i$th neighbor. Since the $s(V_i-V_c)$ is invariant to monotonic change of illumination, thus the LBP is gray-scale invariant.








The patterns with very few spatial transitions is considered to depict the fundamental image micro-structures. Such patterns were called as ``uniform patterns''.
Ojala et al. \cite{ojala2002multiresolution} defined a uniformity measure for the uniform patterns, which is $\mathcal{U}(\text{LBP}_{P,R}(A)) \le N$ ($N$ is usually set to 2). The uniformity measure $\mathcal{U}(\cdot)$ can be calculated as follows:
\begin{equation}\nonumber
\mathcal{U}(\text{LBP}_{P,R}(A)) = \sum_{i=0}^{P-1}|s(V_i - V_c) - s(V_{i-1}-V_c)|,
\end{equation}
where the pixel value of $V_{-1}$ is equivalent to the pixel value of $V_{P-1}$. For example, ``11000000" and ``10000001" are uniform patterns, and ``10000100" and ``10101100" are non-uniform patterns.







Rotation invariant LBP ({${\text{LBP}^{ri}}$}) and rotation invariant uniform LBP ({${\text{LBP}^{ru}}$})
are also introduced in \cite{ojala2002multiresolution}. The ${\text{LBP}^{ri}}$ can be defined as:

\begin{equation}\nonumber
    \text{LBP}_{P, R}^{ri}(A) = \text{min}\{\text{ROR}(\text{LBP}_{P, R}(A), i) \ \  | \ \  \forall i\in[0,P-1]\},
    \label{eq:riLBPCode}
\end{equation}
where $ROR(x , i)$  performs a circularly bit-wise right shift for $i$ times. The $\text{LBP}_{n, r}^{ru}(A)$ is defined as
\begin{equation}\nonumber
\label{eq:rui}
    \text{LBP}_{P, R}^{ru}(A) = \begin{cases}
    \sum_{i=0}^{P-1} s(V_i-V_c), &  \mathcal{U}(\text{LBP}(A)) \le 2 \\
    P+1, &  \text{otherwise}, \\
    \end{cases}
\end{equation}

The $\text{LBP}_{8,1}$ has 256 patterns in total, in which 58 patterns are uniform and the rest 198 patterns are non-uniform. Usually, the 198 non-uniform patterns are summarized to one pattern. Thus, 59 patterns are usually used for uniform LBP. The rotation invariant uniform $\text{LBP}_{8,1}$ includes 10 patterns.

%Above, we have introduced the original LBP and its uniform and rotation invariant variants.



\begin{figure}[h]
\begin{center}
\small
 \includegraphics[width=1.0\linewidth]{pricolbp.eps}%{figures/examples.eps}
\end{center}
   \caption{Pairwise Rotation Invariant Co-occurrence of Local Binary Pattern. The left of the figure shows 10 templates based on the reference point $A$. $G_A$ is the gradient orientation of point $A$, $N_A$ is the normal direction of point $A$. The right point shows one co-occurrence pair of LBPs. PRICoLBP captures stronger structures than the single LBP.}
\label{fig:pricolbp}
\end{figure}





\subsubsection{\bf{Single-Resolution Texture Information}}
Pairwise rotation invariant co-occurrence LBPs (PRICoLBP) is recently introduced by Qi et al. \cite{qi2014pairwise} for texture related tasks. As shown in Fig. \ref{fig:pricolbp}(a), the PRICoLBP is built on the two adjacent LBP points. Given a point $A$, the PRICoLBP contains the following two key steps to calculate its rotation invariant pattern:

\begin{itemize}
\item According to the gradient and normal orientation (Normal orientation is the direction that is orthogonal to the gradient orientation.) of point $A$ and pre-defined templates as shown in Fig. \ref{fig:pricolbp}(a), the position of point $B_i$ can be uniquely determined. The gradient orientation can be calculated as $\theta(A) = \arctan(\frac{dy(A)}{dx(A)})$.
\item With a pair $A$ and $B_i$, pairwise rotation invariant encoding was used to encode the co-occurrence of two LBPs.
\end{itemize}
In practice, we used the gradient magnitudes of point $A$ and $B_i$ to weight their co-pattern.





For the first step, given a point $A$, the PRICoLBP uses the following equation to determine the position of point $B_i$:
\begin{equation}
\label{eq:MSMOCoLBP}
\setlength{\abovedisplayskip}{0pt}
\setlength{\belowdisplayskip}{1pt}
B_i  = a_i*G(A) + b_i*N(A) + A,
\end{equation}
where $a_i$ and $b_i$ are pre-defined coefficients for template $i$, and $G(A)$ and $N(A)$ are the gradient and normal directions of point $A$. In practice, we can choose 10 pairs for [$a_i$, $b_i$] as shown in Fig. \ref{fig:pricolbp}(a), one pair corresponds to one template.





\begin{figure}[t]
\begin{center}
\small
 \includegraphics[width=1.0\linewidth]{gaborcolbp.eps}%{figures/examples.eps}
\end{center}
   \caption{The framework of PRICoLGBP. First, the image is filtered by Gabor filters with different scales. Then, the PRICoLBP features are extracted from each filtered images. Finally, the features extracted from each scale will be concatenated into the final feature.}
\label{fig:gaborcolbp}
\end{figure}




When the point pair $A$ and $B_i$ are determined, a pairwise rotation invariant encoding strategy is used to encode the pair. Denote $\text{LBP}^{u}(A,i)$ as the uniform LBP of point $A$ by using $i$-th index as the start point of the binary sequence. The PRICoLBP can be defined as follows:
\begin{equation}
\label{eq:PriCoLBP}
\text{PRICoLBP}_{P, R}(A,B_i)=[\text{LBP}_{P, R}^{ru}(A),\text{LBP}_{P, R}^{u}(B_i, i(A))]_{co},
\end{equation}
where $i(A)\in \{0,P-1\}$ is an index, which can be determined by minimizing the binary sequence of point $A$. $[\ ,\ ]_{co}$ is a co-occurrence operator firstly introduced in \cite{haralick1973textural}. Suppose $\text{LBP}_{P, R}^{ru}(A)$ has $M$ patterns, and $\text{LBP}_{P, R}^{u}(B_i, i(A))$ has $N$ patterns, then their co-occurrence has $M\times N$ patterns.

For one pair $A$ and $B_i$ with $P = 8$ and $R = 1$, $\text{LBP}_{8, 1}^{ru}(A)$ has 10 patterns, $\text{LBP}_{8, 1}^{u}$ has 59 patterns,
thus, the dimension of $\text{PRICoLBP}_{8, 1}$ is $10\times 59 = 590$. If 10 templates are used as shown in Fig. \ref{fig:pricolbp}(a), the dimension for PRICoLBP is $10\times 590 = 5900$.





















\subsubsection{\bf{Multi-Resolution Texture Information}}




\begin{multicols}{2}
\begin{figure*}[t]
\begin{center}
\small
 \includegraphics[width=1.0\linewidth]{FVF.eps}%{figures/examples.eps}
\end{center}
   \caption{The framework of discriminative shape information description. First, we densely sample thousands of large patches in multiple scales with a preset step (such as 2), then the RootSIFT feature is extracted for each patch. PCA is applied to the RootSIFT. Finally, IFV encoding is applied to the PCA-after features with pre-learned GMM parameters. The obtained histogram from IFV encoding can be input into a linear SVM for the final classification.}
\label{fig:Framework}
\end{figure*}
\end{multicols}



 The PRICoLBP is effective to capture the structures in the small scales
  (such as co-occurrence of $\text{LBP}_{8,1}$ and  co-occurrence of $\text{LBP}_(8,2)$), but texture information in large scales is ignored. However, multi-resolution texture
  information is always effective for many vision applications.


To capture multi-resolution texture information, we propose a novel pairwise rotation invariant co-occurrence of local Gabor binary pattern (PRICoLGBP) descriptor. Gabor wavelet \cite{gabor1946theory} \cite{movellan2002tutorial} is an effective filter to
capture multi-resolution and multi-orientation information. The PRICoLGBP is built on the Gabor filter and PRICoLBP descriptor. The framework of our PRICoLGBP can be seen in Fig. \ref{fig:gaborcolbp}. We convolute the original
image with different Gabor filters, and then extract the PRICoLBP from each filtered image, and finally
concatenate all PRICoLBPs into the final feature.
In experiments, we found that the PRICoLGBP is not sensitive to rotation variation for the Gabor filtered
images, thus, we only use one pre-fixed orientation for all scales.


%
%Gabor wavelet \cite{gabor1946theory} \cite{movellan2002tutorial} is an effective filter to
%capture multi-resolution and multi-orientation information. The Gabor function in space domain
%can be defined as follows:
%\begin{equation}\nonumber
%\label{eq:gabor}
%g(x, y) = s(x, y)w_r(x, y),
%\end{equation}
%where $s(x, y)$ is a complex sinusoid, known as the carrier, and $w_r(x, y)$ is a 2-D Gaussian-shaped function, known as the envelope.
%



%The complex sinusoid function is defined as follows:
%\begin{equation}\nonumber
%\label{eq:sx}
%s(x, y) = \exp(-j(2\pi (u_0 x + v_0 y)) + \theta),
%\end{equation}
%where ($u_0$, $v_0$) and $\theta$ define the spatial frequency and the phase of the sinusoid
%respectively.
%
%The Gaussian envelope is defined as:
%\begin{equation}\nonumber
%\label{eq:sx}
%w_r(x, y) = \frac{1}{ 2 \pi {\sigma}_x {\sigma}_y } \exp( -\pi ( \frac{x^2}{ {{\sigma}_x}^2 } + \frac{y^2}{ {{\sigma}_y}^2 } )),
%\end{equation}
%where ${\sigma}_x$ and ${\sigma}_y$ are scaling parameters of the Gaussian.
%
%
%








%g(x, y, s, \theta) = \frac{1}{ 2 \pi {\sigma}_x {\sigma}_y } \exp( -\pi ( \frac{x^2}{ {{\sigma}_x}^2 } + \frac{y^2}{ {{\sigma}_y}^2 } )) \exp(-j(2\pi (u_0 x + v_0 y)) + \theta),

%\frac{x^2}{ {\sigma}_x}^2 } + \frac{y^2}{ {\sigma}_y}^2 }

%-\pi ( \frac{x^2}{ {\sigma}_x}^2 } + \frac{y^2}{ {\sigma}_y}^2 })} ,



%One method to preserve the rotation invariance is to average the filtered by multiple orientation. However, in practice, we found that the PRICoLBP is not sensitive to rotation variation for the Gabor filtered images. Thus, one simple method is to filter the image with a fixed orientation. Then, the PRICoLBP features for each Gabor filtered images and the original images. Finally, the PRICoLBP is concatenated into the final feature.




The PRICoLGBP shares some similar properties with Local Gabor Binary
Pattern (LGBP) \cite{zhang2005local} that is seen as a powerful LBP variants in face recognition, but
different from the LGBP, our PRICoLGBP is built on a more discriminative co-occurrence of LBPs features.
Thus, we can expect that PRICoLGBP can capture stronger multi-resolution texture information.


We believe two strong properties of the PRICoLGBP makes it effective for
IIF HEp-2 cell classification.



\begin{itemize}

\item PRICoLGBP has strong texture discrimination. In IIF HEp-2 cell classification, texture-based
methods proves to be effective.

\item Gabor and PRICoLBP both are robust to image illumination variation. PRICoLGBP inherited the
properties from both Gabor and PRICoLBP. In IIF HEp-2 cells, the ``Positive'' and ``Intermediate'' cells
from the same categories show extremely varying illumination.
\end{itemize}














\subsection{Effective Shape Feature}
In this subsection, we present an effective method to depict the richer shape information by using an Improved Fisher Vector (IFV) model with RootSIFT features extracted from large image patches in multiple scales. Our approach consists of three steps: a) patch sampling, b) feature description with RootSIFT, and c) encoding by IFV. The flowchart to illustrate our approach is displayed in Fig.~\ref{fig:Framework}.

%\begin{itemize}
%\item Patch Sampling to obtain the most representative and discriminative regions.
%\item Feature Description to effectively capture the structures of the sampled regions.
%\item Encoding Methods to preserve the strong discrimination of the features.
%\end{itemize}

%In this section, we will mainly introduce our approach to depict discriminative shape information.
%Fig. \ref{fig:Framework} has illustrated the framework of our approach. In the following, we will introduce three key steps in the framework: patch sampling, feature description and encoding methods. To fully capture the shape information, these three steps are all indispensable:
%\begin{itemize}
%\item Patch Sampling to obtain the most representative and discriminative regions.
%\item Feature Description to effectively capture the structures of the sampled regions.
%\item Encoding Methods to preserve the strong discrimination of the features.
%\end{itemize}



% \noindent
%\subsubsection{\bf{Discriminative Patch Sampling for depicting shape information.}}
\subsubsection{\bf{Patch Sampling for Depicting Shape Information}}
To increase the discriminativeness in shape information, we propose to sample large patches, since that the large patches preserve stronger shape structures. To be specific, instead of sampling patches of small size, e.g., $16\times 16$, $17\times 17$, or $19\times 19$ as in object categorization tasks, we sample much larger patches, e.g., $41\times 41$. We can observe in Fig. \ref{fig:Framework} that the sampled patches cover more than $\frac{1}{3}$ of the whole image.\footnote{In general, a HEp-2 cell image is of $65\times 70$ and hence preserve stronger shape structure from the sampled patches. % texture information dominates the local regions (such as $3\times 3$ and $5\times 5$ regions), and shape information becomes stronger along with the increase of the patch size.
}
%Our motivation to focus on capturing shape information is based on the observation that shape information is rich and discriminative for both ``positive'' and ``intermediate'' HEp-2 cells.



\begin{figure}[h]
\begin{center}
\small
 \includegraphics[width=1.0\linewidth]{enhancement1.eps}%{figures/examples.eps}
\end{center}
   \caption{Illustration of discrimination of shape information in ICIP 2013 contest data set. Each column comes from one category. It is easy to find that the shapes vary a lot among different categories. (A) shows the ``Intermediate'' cell images from each category, and (B) shows (A)'s corresponding enhanced images from a logarithmic operator $loge(I)$. (C) shows the ``Positive'' cell images, and (D) is (C)'s corresponding enhanced images.}
\label{fig:enhancement}
\end{figure}

In Fig. \ref{fig:enhancement}, we show some samples from all six categories in ICIP 2013 contest
data set.\footnote{To visualize the shape structures clearly, we enhance the images at first by using a logarithmic operator $loge(I)$ on the image and then normalize the image to the range of $[0,255]$. This preprocessing method was proposed in \cite{Vestergaard2014}.
Notice that:}
\begin{itemize}
\item The shape structures from different categories change a lot. Each category has its own basic characteristics. For instance, the category ``NuMum'' has bright and thick boundary, the category ``Centermere'' has many bright spots, and the category ``Golgi'' does not have well-formed boundary. Considering local texture structures, the shape difference between some categories is large. For instance, the categories ``Nucleolar'' and ``Centromere'' are easy to differentiate when jointly considering the shape and texture.
\item The ``positive'' and ``intermediate'' HEp-2 cells from same category share similar shape structure, %although sometimes,    it is not \textcolor{blue}{easy for humans}
    although we cannot see the shape structure of the ``intermediate'' HEp-2 cells clearly.
\end{itemize}
These observations are the rationales to explore the shape information for HEp-2 cell image classification.
%Based on the above-mentioned two observations, we can conclude that shape information is discriminative for HEp-2 cell images.

%To capture the discriminative shape information, we propose to sample large patches because we
%believe that large patches can preserve strong shape structures. Instead of sampling
%patches like $17\times 17$ pixels or $19\times 19$ pixels as usual object recognition tasks, we sample
%larger patches (e.g. $41\times 41$).
%The sampled patches can be seen in Fig. \ref{fig:Framework}. They cover more than $\frac{1}{3}$ of
%the whole image (e.g. $65\times 70$). We can clearly see its shape structure from the sampled patches, which means that the sampled patches preserves the strong shape information. We believe that texture
%information dominates the local regions (such as $3\times 3$ and $5\times 5$ regions), and
%shape information becomes stronger along with the increase of the patch size.






\subsubsection{\bf{RootSIFT Feature Extraction on Large Patches}}
We extract 128-dimensional SIFT features \cite{lowe2004distinctive} from the sampled large patches. For each SIFT feature $\bf f$, we normalize it with $L_1$-norm and then take the componentwise square root operation, i.e.,
\begin{align}
\begin{split}
\bar {\bf f} &= {\bf f \over \| \bf f \|_1}, \\
\tilde f_j   &= \sqrt{\bar f_j},~~~~j=1,\cdots, 128.
\end{split}
\end{align}
The obtained $\tilde f$ is termed as ``RootSIFT'' \cite{arandjelovic2012three}, which was proposed by Relja et al. to enhance the discriminative power of SIFT.

%\footnote{Geometrically, each RootSIFT vector lies on a 127-dimensional sphere, i.e., $S^{127}$.}
%which is After the normalization and componentwise square-root operation,
%}
%SIFT \cite{lowe2004distinctive} is a widely used feature in computer vision, image processing and medical image analysis, but it was originally designed to be used with Euclidean distance. It has been shown that, in texture classification, object recognition, image retrieval and etc., Euclidean kernel performs poorer than Hellinger and ${\chi}^2$ kernel.




%Relja et al. \cite{arandjelovic2012three} proposed to use Hellinger distance to enhance the discriminative power of SIFT, and they called it RootSIFT. RootSIFT doesn't change
%the feature extraction process of SIFT. It just does some post-processing to the SIFT feature. When
%SIFT is obtained, it is normalized with $L_1$ norm, and then the square root operator is conducted on
%all bins of 128 dimensions.
%
%RootSIFT can be thought as a normalization method to SIFT. It can also be considered
%as an explicit feature map from the original SIFT space to the RootSIFT space. RootSIFT
%can be thought as computing Hellinger kernel in the original SIFT space. Vedaldi et al. deeply
%investigate the kernel mapping problem in \cite{vedaldi11efficient}. According to their
%experiments, we could see significant improvement provided by kernel mapping.
%%
%Spatial division of the described patch ($4\times 4$ grids) is an effective way to capture
% shape layout information. Gradient orientation is also widely recognized as an effective shape
% information description method in many vision tasks. The usages of spatial division and
% gradient orientation endow the RootSIFT strong ability to describe shape structures.


%
%
%IIF HEp-2 Cells always have strong shape structures although for the intermediate cells, the shape structures could be directly seen. Under some simple image enhancement technology. As shown in Fig. , all cells including intermediate and positive cells have strong shape structures.
%
%
%
%Usually, texture information dominates the local regions (such as $3\time 3$ and $5\time 5$ regions), and shape information becomes stronger along with the increase of the patch size. Thus, instead of sampling patches with $16\time 16$ or $18\time 18$ for feature extraction, we sample larger patches ($41\time 41$). For usual HEp-2 cell, their cell size is about $65\time 65$. Patches with size $41\time 41$ almost covers 40\% of whole image. Thus, the larger patch can reflect the shape structure better. For instance, in Fig. \ref{fig:Framework}, we can find that the sampled patches can effectively reflect the structure of the cell.


%In local region texture information is more than $3\time 3$,
%\subsubsection{\bf{Improved Fisher Vector (IFV) Encoding}}
\subsubsection{\bf{Improved Fisher Vector (IFV) for Encoding the RootSIFTs}}
We encode the RootSIFT features by Improved Fisher Vector (IFV) approach \cite{perronnin2010improving} \cite{sanchez2013image}, which consists of three steps:
\begin{itemize}
\item Data decorrelation by Principal Component Analysis (PCA).
\item Training a Gaussian Model of Mixture (GMM).
\item Forming the IFV by using the first and second order statistics in GMM.
\end{itemize}


Denote the parameters in GMM as $\{ \pi_k, \mu_k, \Sigma_k\}_{k=1}^K$ where $\pi_k$ is the membership probability, $\mu_k$ is the mean of $k$-th component Gaussian, and $\Sigma_k$ is the covariance matrix which is enforced to be diagonal.
Let $X=[\textbf{x}_1, \textbf{x}_2, ..., \textbf{x}_N]\in \mathcal{R}^{d\times{N}}$ be a set of feature vectors of an image after decorrelation, where $d$ is reduced feature dimension of using PCA and $N$ is the number of RootSIFT features in the image. IFV captures the deviation of the features in an image from the first and second statistics of the GMM. %IFV measures the first and second order variation between the features and the GMM components. Given a set of features $X = [x_1,...,x_N]$, we can calculate $s_{ki}$, i.e. the probability of feature $x_i$ belonging to $k$-th Gaussian component.
To be specific, IFV is defined as follows:
\begin{equation}
{\bf{F}} = [{{\bf{u}}_1}^{\rm{T}},{\bf{v}}_1^{\rm{T}},..., {\bf{u}_K}^{\rm{T}},{\bf{v}_K}^{\rm{T}}]^{\rm{T}},
\end{equation}
where
\begin{equation}
 {\bf{u}}_k = \frac{1}{N \sqrt{{\pi}_k}} \sum_{i=1}^{N}s_{ki} {{\Sigma}_k}^{-\frac{1}{2}} (x_i - \mu_k),
\end{equation}
\begin{equation}
 {\bf{v}}_k = \frac{1}{N \sqrt{2{\pi}_k}} \sum_{i=1}^{N}s_{ki} [(x_i - \mu_k) \cdot ({{\Sigma}_k}^{-1} (x_i - \mu_k) ) - {\bf{1}}],
\end{equation}
in which $s_{ki}$ is defined as
\begin{equation}\nonumber
 s_{ki} =  \frac{ p(x_i|\mu_k,{\Sigma}_k) {\pi}_k}{ \sum_{i=1}^{K} p(x_i|\mu_i,{\Sigma}_k)){\pi}_i }, \ \ \ \ \ \ k=1,...,K .
\label{eq:gmmS}
\end{equation}
The parameter $s_{ki}$ is the responsibility of feature $\textbf{x}_i$ belonging to the $k$-th component in the GMM.

%IFV \cite{perronnin2010improving} \cite{sanchez2013image} encoding proves to be extremely effective, compared to its counterparts (Vector Quantization (VQ)), on many computer vision tasks, including image classification \cite{chatfield2011devil}, face verification \cite{simonyan2013fisher} and texture and material classification \cite{mircea2014}. This paper is the first time that IFV has been used in HEp-2 cell classification.

%In the BoW model, a \emph{visual dictionary} must be created before feature encoding. A visual dictionary partitions the whole feature space into some regions that we call \emph{visual words}.

%The IFV consists of three steps. First, hundreds of thousands of local features from the training images are used to learn a Principal Component Analysis (PCA) model. Then, the after-PCA features are used to learn a Gaussian Mixture Model (GMM) model with parameters $\theta = \left \{ \pi_k, \mu_k, \Sigma_k, k=1,\cdots,K\right \}$, \textcolor{blue}{where $\pi_k$ ($k \in [1, K]$) are the prior probability, $\mu_k$ ($k \in [1, K]$) are the means of each gaussian, and $\Sigma_k$ ($k \in [1, K]$) are the positive definite covariance matrix, $\Sigma_k$ are forced to be diagonal in IFV framework}. Finally, given an images, the features in this images can be encoded by IFV.
%Let $\mathbf{X}=[x_1, x_2, ..., x_N]\in \mathcal{R}^{d_{pca}\times{N}}$ be a set of feature descriptors for an image after PCA, where $d_{pca}$ is feature dimension after PCA and $N$ is the number of features. For instance, if we sampled 1000 patches in an image and extracted one descriptor from each patch, then $N = 1000$.
%The IFV measures the first and second order variation between the features and the GMM components. Given a set of features $X = [x_1,...,x_N]$, we can calculate $s_{ki}$, i.e. the probability of feature $x_i$ belonging to $k$-th Gaussian component.
%For each component $k = 1, ..., K$, the vector can be defined as:
%\begin{equation}
% {\bf{u}}_k = \frac{1}{N \sqrt{{\pi}_k}} \sum_{i=1}^{N}s_{ki} {{\Sigma}_k}^{-\frac{1}{2}} (x_i - \mu_k),
%\end{equation}
%\begin{equation}
% {\bf{v}}_k = \frac{1}{N \sqrt{2{\pi}_k}} \sum_{i=1}^{N}s_{ki} [(x_i - \mu_k)^{\rm{T}} {{\Sigma}_k}^{-1} (x_i - \mu_k)],
%\end{equation}
%where $s_{ki}$ denotes the probability of feature $x_i$ belonging to the $k$-th component. The $s_{ki}$ can be calculated as:
%\begin{equation}\nonumber
% s_{ki} =  \frac{ p(x_i|\mu_k,{\Sigma}_k) {\pi}_k}{ \sum_{i=1}^{K} p(x_i|\mu_i,{\Sigma}_k)){\pi}_i }, \ \ \ \ \ \ k=1,...,K .
%\label{eq:gmmS}
%\end{equation}
%The computation of ${\bf{u}}_k$ and ${\bf{v}}_k$ is fast because ${\Sigma}_k$ is diagonal. Given ${\bf{u}}_k$ and ${\bf{v}}_k$ for all $K$ components, the IFV can be defined as follows:
%\begin{equation}
%{\bf{F}}_{IFV} = [{{\bf{u}}_1}^{\rm{T}},{\bf{v}}_1^{\rm{T}},..., {\bf{u}_K}^{\rm{T}},{\bf{v}_K}^{\rm{T}}] .
%\end{equation}

Note that the dimension of ${\bf{F}}$ is $2\times{d}\times{K}$.% $d$ is the dimension of our RootSIFT after PCA.
In our experiments, we set $d$ as 80, and $K$ as 256, the number of mixture components in GMM. The final dimension of IFV feature is  $2\times 80 \times 256 = 41960$. Note also that this is the first time that IFV is used in HEp-2 cell classification task.


%\subsubsection{\bf{Summary:}} To capture discriminative shape information, we have the following three key contributions  in this subsection,
%\begin{itemize}
%\item We propose to sample large patches to capture stronger shape information. The sampled
%patches cover more than $\frac{1}{3}$ of the whole image.
%\item We propose to use the RootSIFT feature to depict the sampled patches. We believe
%this spatial division in RootSIFT can depict shape information well.
%\item We propose to use IFV to encode the RootSIFT to preserve the discrimination of RootSIFT feature.
%
%%This is motivated by the recent success of the IFV in object recognition.
%
%\end{itemize}

\subsection{\bf{Histogram Normalization and Classification}}
Histogram normalization is a key step before training a SVM model.
We normalize the histogram componentwisely as follows:
\begin{equation}
\label{eq:chiB}
    %H_i = sign(H_i) \sqrt{\mid{H_i}\mid}, \forall \ i \in [1, D],
\hat H_i = sign(H_i) \sqrt{\mid{H_i}\mid}, \forall i=1,\cdots,D,
\end{equation}
where $D$ is the dimension of $H$, $sign(\cdot)$ is a sign function. And then we further normalize the histogram  with $L_2$ norm.

For classification we use \textit{linear} SVM since it is widely used in large scale problems. For linear SVM, the training is fast and the speed of classification in test phase is also fast, compared to kernel SVM. We use the \textit{one-vs-the-rest} strategy to handle the multi-class classification problem.


















































%
%
%
%
%\subsection{Discriminative Shape Feature}
%In this section, we will mainly introduce our approach to depict discriminative shape
%information. Fig. \ref{fig:Framework} has illustrated the framework of our approach. In the
%following, we will introduce three key steps in the framework: patch sampling, feature description
%and encoding methods. To fully capture the shape information, these three steps are all indispensable:
%\begin{itemize}
%\item Patch Sampling to obtain the most representative and discriminative regions.
%\item Feature Description to effectively capture the structures of the sampled regions.
%\item Encoding Methods to preserve the strong discrimination of the features.
%\end{itemize}
%
%
%
%
%
%
%
%\subsubsection{\bf{Discriminative Patch Sampling for Depicting Shape Information.}} Our
%motivation to focus on capturing shape information is based on the observation that
%shape information is rich and discriminative for both ``positive'' and ``intermediate''
%HEp-2 cells. Fig. \ref{fig:enhancement} shows samples from all six categories in ICIP 2013 contest
%dataset. To visualize the shape structures clearly, we use one preprocessing method as \cite{Vestergaard2014} to
%enhance the images: we use a logarithmic operator $loge(I)$ to
%the image and then normalize the image to [0,255].
%From Fig. \ref{fig:enhancement}, we have the following two observations:
%
%
%
%\begin{itemize}
%\item The shape structures from different categories vary a lot. Each category has its own characteristics. For instance, the catergory ``NuMum'' has bright and thick boundary, the category ``Centermere'' has many bright spots and the category ``Golgi'' does not have \textcolor{blue}{well-formed boundary. Considering local texture structures, the shape difference between some categories is large. For instance, the categories ``Nucleolar'' and ``Centromere'' are easy to differentiate when jointly considering the shape and texture.}
%
%
%\item The ``positive'' and ``intermediate'' HEp-2 cells from same category share similar shape structure although sometimes, it is not so \textcolor{blue}{easy for humans} to see the shape structure of the ``intermediate'' HEp-2 cells clearly.
%
%\end{itemize}
%
%%Based on the above-mentioned two observations, we can conclude that shape information is discriminative for HEp-2 cell images.
%
%
%
%
%
%\begin{figure}[h]
%\begin{center}
%\small
% \includegraphics[width=1.0\linewidth]{figures/enhancement1.eps}%{figures/examples.eps}
%\end{center}
%   \caption{Illustration of discrimination of shape information in ICIP 2013 contest data set. Each column comes from one category. It is easy to find that the shapes vary a lot among different categories. (A) shows the ``Intermediate'' cell images from each category, and (B) shows (A)'s corresponding enhanced images from a logarithmic operator $loge(I)$. (C) shows the ``Positive'' cell images, and (D) is (C)'s corresponding enhanced images.}
%\label{fig:enhancement}
%\end{figure}
%
%
%
%
%To capture the discriminative shape information, we propose to sample large patches because we
%believe that large patches can preserve strong shape structures. Instead of sampling
%patches like $17\times 17$ pixels or $19\times 19$ pixels as usual object recognition tasks, we sample
%larger patches (e.g. $41\times 41$).
%The sampled patches can be seen in Fig. \ref{fig:Framework}. They cover more than $\frac{1}{3}$ of
%the whole image (e.g. $65\times 70$). We can clearly see its shape structure from the sampled patches, which means
%that the sampled patches preserves the strong shape information. We believe that texture
%information dominates the local regions (such as $3\times 3$ and $5\times 5$ regions), and
%shape information becomes stronger along with the increase of the patch size.
%
%\textcolor{blue}{Texture information dominants in small regions, but shape information dominants in large regions. The proposed sampling strategy can affiliate the features stronger power in capturing joint texture and shape information.}
%
%
%\begin{multicols}{2}
%\begin{figure*}[t]
%\begin{center}
%\small
% \includegraphics[width=1.0\linewidth]{figures/FVF.eps}%{figures/examples.eps}
%\end{center}
%   \caption{The framework of discriminative shape information description. First, we densely sample thousands of large patches in multiple scales with a preset step (such as 2), then the RootSIFT feature is extracted for each patch. PCA is applied to the RootSIFT. Finally, IFV encoding is applied to the PCA-after features with pre-learned GMM parameters. The obtained histogram from IFV encoding can be input into a linear SVM for the final classification.}
%\label{fig:Framework}
%\end{figure*}
%\end{multicols}
%
%
%\subsubsection{\bf{RootSIFT Feature Extraction on Large Patches}}
%SIFT \cite{lowe2004distinctive} is a widely used feature in computer vision, image
%processing and medical image analysis, but it was originally designed to be used with
%Euclidean distance. It has been shown that, in texture classification, object recognition, image
%retrieval and etc., Euclidean kernel performs poorer than Hellinger and ${\chi}^2$ kernel.
%
%
%
%
%
%Relja et al. \cite{arandjelovic2012three} proposed to use Hellinger distance to
%enhance the discriminative power of SIFT, and they called it RootSIFT. RootSIFT doesn't change
%the feature extraction process of SIFT. It just does some post-processing to the SIFT feature. When
%SIFT is obtained, it is normalized with $L_1$ norm, and then the square root operator is conducted on
%all bins of 128 dimensions. RootSIFT can be thought as a normalization method to SIFT. It can also be considered as an explicit feature map from the original SIFT space to the RootSIFT space. RootSIFT
%can be thought as computing Hellinger kernel in the original SIFT space. \textcolor{blue}{Vedaldi et al. made a deep
%investigation of the kernel mapping in \cite{vedaldi11efficient}.}
%%According to their
%%experiments, we could see significant improvement provided by kernel mapping.
%
%Spatial division of the described patch ($4\times 4$ grids) is an effective way to capture
% shape layout information. Gradient orientation is also widely recognized as an effective shape
% information description method in many vision tasks. The usages of spatial division and
% gradient orientation endow the RootSIFT strong ability to describe shape structures.
%
%
%%
%%
%%IIF HEp-2 Cells always have strong shape structures although for the intermediate cells, the shape structures could be directly seen. Under some simple image enhancement technology. As shown in Fig. , all cells including intermediate and positive cells have strong shape structures.
%%
%%
%%
%%Usually, texture information dominates the local regions (such as $3\time 3$ and $5\time 5$ regions), and shape information becomes stronger along with the increase of the patch size. Thus, instead of sampling patches with $16\time 16$ or $18\time 18$ for feature extraction, we sample larger patches ($41\time 41$). For usual HEp-2 cell, their cell size is about $65\time 65$. Patches with size $41\time 41$ almost covers 40\% of whole image. Thus, the larger patch can reflect the shape structure better. For instance, in Fig. \ref{fig:Framework}, we can find that the sampled patches can effectively reflect the structure of the cell.
%
%
%%In local region texture information is more than $3\time 3$,
%\subsubsection{\bf{Improved Fisher Vector (IFV) Encoding}} IFV \cite{perronnin2010improving} \cite{sanchez2013image} encoding proves to be extremely effective, compared to its counterparts (Vector Quantization (VQ)), on many computer vision tasks, including image classification \cite{chatfield2011devil}, face verification \cite{simonyan2013fisher} and texture and material classification \cite{mircea2014}. This paper is the first time that the IFV has been used in HEp-2 cell classification.
%
%
%
%%In the BoW model, a \emph{visual dictionary} must be created before feature encoding. A visual dictionary partitions the whole feature space into some regions that we call \emph{visual words}.
%
%The IFV consists of three steps. First, hundreds of thousands of local features from the training images are used to learn a Principal Component Analysis (PCA) model. Then, the after-PCA features are used to learn a Gaussian Mixture Model (GMM) model with parameters $\theta = \left \{ \pi_k, \mu_k, \Sigma_k, k=1,\cdots,K\right \}$, \textcolor{blue}{where $\pi_k$ ($k \in [1, K]$) are the prior probability, $\mu_k$ ($k \in [1, K]$) are the means of each gaussian, and $\Sigma_k$ ($k \in [1, K]$) are the positive definite covariance matrix, $\Sigma_k$ are forced to be diagonal in IFV framework. Finally, given an images, the features in this images can be encoded by IFV.}
%
%Let $\mathbf{X}=[x_1, x_2, ..., x_N]\in \mathcal{R}^{d_{pca}\times{N}}$ be a set of feature descriptors for an image after PCA, where $d_{pca}$ is feature dimension after PCA and $N$ is the number of features. For instance, if we sampled 1000 patches in an image and extracted one descriptor from each patch, then $N = 1000$.
%The IFV measures the first and second order variation between the features and the GMM components. Given a set of features $X = [x_1,...,x_N]$, we can calculate $s_{ki}$, i.e. the probability of feature $x_i$ belonging to $k$-th Gaussian component.
%For each component $k = 1, ..., K$, the vector can be defined as:
%\begin{equation}
% {\bf{u}}_k = \frac{1}{N \sqrt{{\pi}_k}} \sum_{i=1}^{N}s_{ki} {{\Sigma}_k}^{-\frac{1}{2}} (x_i - \mu_k),
%\end{equation}
%
%\begin{equation}
%\textcolor{blue}{ {\bf{v}}_k = \frac{1}{N \sqrt{2{\pi}_k}} \sum_{i=1}^{N}s_{ki} [(x_i - \mu_k) \cdot ({{\Sigma}_k}^{-1} (x_i - \mu_k) ) - {\bf{1}}],}
%\end{equation}
%
%\textcolor{blue}{where $\cdot$ means dot product, $s_{ki}$ denotes the probability of feature $x_i$ belonging to the $k$-th component.} The $s_{ki}$ can be calculated as:
%\begin{equation}\nonumber
% s_{ki} =  \frac{ p(x_i|\mu_k,{\Sigma}_k) {\pi}_k}{ \sum_{i=1}^{K} p(x_i|\mu_i,{\Sigma}_k)){\pi}_i }, \ \ \ \ \ \ k=1,...,K .
%\label{eq:gmmS}
%\end{equation}
%
%
%
%
%The computation of ${\bf{u}}_k$ and ${\bf{v}}_k$ is fast because ${\Sigma}_k$ is diagonal. Given ${\bf{u}}_k$ and ${\bf{v}}_k$ for all $K$ components, the IFV can be defined as follows:
%\begin{equation}
%\textcolor{blue}{
%{\bf{F}}_{IFV} = [{{\bf{u}}_1}^{\rm{T}},{\bf{v}}_1^{\rm{T}},..., {\bf{u}_K}^{\rm{T}},{\bf{v}_K}^{\rm{T}}]^{\rm{T}}.
%}
%\end{equation}
%
%The dimension of ${\bf{F}}_{IFV}$ is $2\times{d_{pca}}\times{K}$. $d_{pca}$ is the dimension of our RootSIFT after PCA. In our experiments, we can set $d_{pca}$ to be 80. $K$ is the number of GMM clusters. In this paper,
%we set $K$ to be 256. For $d_{pca}=80$ and $K=256$, the final dimension for the IFV feature is  $2\times 80 \times 256 = 41960$.
%
%
%\subsubsection{\bf{Summary:}} To capture discriminative shape information, we have the following three key contributions  in this subsection,
%\begin{itemize}
%\item We propose to sample large patches to capture stronger shape information. The sampled
%patches cover more than $\frac{1}{3}$ of the whole image.
%\item We propose to use the RootSIFT feature to depict the sampled patches. We believe
%this spatial division in RootSIFT can depict shape information well.
%\item We propose to use the IFV to encode the RootSIFT to preserve the discrimination of RootSIFT feature.
%
%%This is motivated by the recent success of the IFV in object recognition.
%
%\end{itemize}
%
%\subsection{\bf{Histogram Normalization and Classification}}
%
%
%Histogram normalization is a key step before training a SVM model. The Hellinger's kernel proves
%to be an effective normalization method in \cite{vedaldi11efficient}. It doesn't increase the
%feature dimension but always improves the classification performance of linear SVM.
%
%%When values of the histogram are all positive, first, the histogram $H$ is normalized with $L_1$ norm. It means
%%the summation of all values in the histogram is 1. Then, for each dimension in $H$,
%%
%%\begin{equation}
%%\label{eq:chiA}
%%    H_i = \sqrt{H_i}, \forall \ i \in [1, N],
%%\end{equation}
%%where $N$ is the dimension of $H$.
%
%We first normalize the histogram as follows:
%\begin{equation}
%\label{eq:chiB}
%    H_i = sign(H_i) \sqrt{\mid{H_i}\mid}, \forall \ i \in [1, D],
%\end{equation}
%where $D$ is the dimension of $H$, $sign()$ is a sign function. After this operator, the
%histogram is further normalized with $L_2$ norm.
%
%
%Linear SVM is widely used in large scale problems because of its two advantages. First, the
%training of linear SVM is fast. Second, the speed of classification for linear SVM is
%extremely fast compared with kernel SVM. One-vs-the-rest training strategy is usually used in SVM.
%When the model for the linear SVM is obtained, we classify the testing samples
%using the following equation:
%\begin{equation}
%\label{eq:chi}
%    F_c = \sum_{i=1}^{D} w_{ci}{H_i} + b_c,
%\end{equation}
%where $D$ is the dimension of the feature vector $H$, $(w_c, b_c)$ are the learned parameters from linear SVM. $C$ is the number of categories.
%When $F_c$ ($\forall \ c \in [1, C]$) is achieved, the sample will be classified to the category with the maximun $F_c$.



\begin{figure}[h]
\begin{center}
\small
 \includegraphics[width=1.0\linewidth]{icprsample.eps}%{figures/examples.eps}
\end{center}
   \caption{Sample images from ICPR 2012 contest dataset with each column from one category. The first two rows show the intermediate samples, and the last two rows show the positive samples. All the images are shown in their original image size.}
\label{fig:icpr}
\end{figure}

HEp2Cell.bbl
% Generated by IEEEtran.bst, version: 1.13 (2008/09/30)
\begin{thebibliography}{10}
\providecommand{\url}[1]{#1}
\csname url@samestyle\endcsname
\providecommand{\newblock}{\relax}
\providecommand{\bibinfo}[2]{#2}
\providecommand{\BIBentrySTDinterwordspacing}{\spaceskip=0pt\relax}
\providecommand{\BIBentryALTinterwordstretchfactor}{4}
\providecommand{\BIBentryALTinterwordspacing}{\spaceskip=\fontdimen2\font plus
\BIBentryALTinterwordstretchfactor\fontdimen3\font minus
  \fontdimen4\font\relax}
\providecommand{\BIBforeignlanguage}[2]{{%
\expandafter\ifx\csname l@#1\endcsname\relax
\typeout{** WARNING: IEEEtran.bst: No hyphenation pattern has been}%
\typeout{** loaded for the language `#1'. Using the pattern for}%
\typeout{** the default language instead.}%
\else
\language=\csname l@#1\endcsname
\fi
#2}}
\providecommand{\BIBdecl}{\relax}
\BIBdecl

\bibitem{foggia2013benchmarking}
P.~Foggia, G.~Percannella, P.~Soda, and M.~Vento, ``Benchmarking hep-2 cells
  classification methods.'' \emph{IEEE Transactions on Medical Imaging},
  vol.~32, no.~10, p. 1878, 2013.

\bibitem{foggia2014PR}
P.~Foggia, G.~Percannella, A.~Saggese, and M.~Vento, ``Pattern recognition in
  stained hep-2 cells: Where are we now?'' \emph{Pattern Recognition}, vol.~47,
  no.~7, pp. 2305--2314, 2014.

\bibitem{foggia2010early}
P.~Foggia, G.~Percannella, P.~Soda, and M.~Vento, ``Early experiences in
  mitotic cells recognition on hep-2 slides,'' in \emph{IEEE 23rd International
  Symposium on Computer-Based Medical Systems (CBMS)}, 2010, pp. 38--43.

\bibitem{percannella2012classification}
G.~Percannella, P.~Soda, and M.~Vento, ``A classification-based approach to
  segment hep-2 cells,'' in \emph{IEEE 25th International Symposium on
  Computer-Based Medical Systems (CBMS)}.\hskip 1em plus 0.5em minus
  0.4em\relax IEEE, 2012, pp. 1--5.

\bibitem{theodorakopoulos2012hep}
I.~Theodorakopoulos, D.~Kastaniotis, G.~Economou, and S.~Fotopoulos, ``Hep-2
  cells classification via fusion of morphological and textural features,'' in
  \emph{IEEE 12th International Conference on Bioinformatics \& Bioengineering
  (BIBE)}.\hskip 1em plus 0.5em minus 0.4em\relax IEEE, 2012, pp. 689--694.

\bibitem{nosaka2013hep}
R.~Nosaka and K.~Fukui, ``Hep-2 cell classification using rotation invariant
  co-occurrence among local binary patterns,'' \emph{Pattern Recognition},
  vol.~47, no.~7, pp. 2428--2436, 2014.

\bibitem{theodorakopoulos2013hep}
I.~Theodorakopoulos, D.~Kastaniotis, G.~Economou, and S.~Fotopoulos, ``Hep-2
  cells classification via sparse representation of textural features fused
  into dissimilarity space,'' \emph{Pattern Recognition}, vol.~47, no.~7, pp.
  2367--2378, 2014.

\bibitem{kong2013hep}
X.~Kong, K.~Li, J.~Cao, Q.~Yang, and L.~Wenyin, ``Hep-2 cell pattern
  classification with discriminative dictionary learning,'' \emph{Pattern
  Recognition}, vol.~47, no.~7, pp. 2379--2388, 2014.

\bibitem{shen2013hep}
L.~Shen, J.~Lin, S.~Wu, and S.~Yu, ``Hep-2 image classification using intensity
  order pooling based features and bag of words,'' \emph{Pattern Recognition},
  vol.~47, no.~7, pp. 2419--2427, 2014.

\bibitem{wiliem2013classification}
A.~Wiliem, Y.~Wong, C.~Sanderson, P.~Hobson, S.~Chen, and B.~C. Lovell,
  ``Classification of human epithelial type 2 cell indirect immunofluoresence
  images via codebook based descriptors,'' in \emph{2013 IEEE Workshop on
  Applications of Computer Vision (WACV)}.\hskip 1em plus 0.5em minus
  0.4em\relax IEEE, 2013, pp. 95--102.

\bibitem{han2014high}
X.-H. Han, J.~Wang, G.~Xu, and Y.-W. Chen, ``High-order statistics of
  micro-texton for hep-2 staining pattern classification,'' \emph{IEEE
  Transactions on Biomedical Engineering}, 2014.

\bibitem{faraki2014fisher}
M.~Faraki, M.~T. Harandi, A.~Wiliem, and B.~C. Lovell, ``Fisher tensors for
  classifying human epithelial cells,'' \emph{Pattern Recognition}, vol.~47,
  no.~7, pp. 2348--2359, 2014.

\bibitem{icpr2012hep2}
P.~Foggia, G.~Percannella, P.~Soda, and M.~Vento, ``Hep-2 cells classification
  contest website [online],'' in \emph{Available: http://
  mivia.unisa.it/hep2contest/}.

\bibitem{icip2013}
P.~Hobson, G.~Percannella, M.~Vento, and A.~Wiliem,
  ``http://nerone.diiie.unisa.it/contest-icip-2013-test/index.shtml.''

\bibitem{icpr2014hep2}
B.~Lovell, G.~Percannella, M.~Vento, and A.~Wiliem, ``Icpr 2014 hep-2 cells
  classification workshop and contest,'' in \emph{Available:
  http://i3a2014.unisa.it/}.

\bibitem{ojala1996comparative}
T.~Ojala, M.~Pietik{\"a}inen, and D.~Harwood, ``A comparative study of texture
  measures with classification based on featured distributions,'' \emph{Pattern
  Recognition}, vol.~29, no.~1, pp. 51--59, 1996.

\bibitem{ojala2002multiresolution}
T.~Ojala, M.~Pietikainen, and T.~Maenpaa, ``Multiresolution gray-scale and
  rotation invariant texture classification with local binary patterns,''
  \emph{IEEE Transactions on Pattern Analysis and Machine Intelligence},
  vol.~24, no.~7, pp. 971--987, 2002.

\bibitem{pietikainen2011computer}
M.~Pietik{\"a}inen, A.~Hadid, G.~Zhao, and T.~Ahonen, \emph{Computer vision
  using local binary patterns}.\hskip 1em plus 0.5em minus 0.4em\relax
  Springer, 2011, vol.~40.

\bibitem{ahonen2006face}
T.~Ahonen, A.~Hadid, and M.~Pietikainen, ``Face description with local binary
  patterns: Application to face recognition,'' \emph{IEEE Transactions on
  Pattern Analysis and Machine Intelligence}, vol.~28, no.~12, pp. 2037--2041,
  2006.

\bibitem{zhao2007dynamic}
G.~Zhao and M.~Pietikainen, ``Dynamic texture recognition using local binary
  patterns with an application to facial expressions,'' \emph{IEEE Transactions
  on Pattern Analysis and Machine Intelligence}, vol.~29, no.~6, pp. 915--928,
  2007.

\bibitem{qi2014pairwise}
X.~Qi, R.~Xiao, C.-G. Li, Y.~Qiao, J.~Guo, and X.~Tang, ``Pairwise rotation
  invariant co-occurrence local binary pattern,'' \emph{IEEE Transactions on
  Pattern Analysis and Machine Intelligence}, 2014.

\bibitem{guo2010completed}
Z.~Guo, L.~Zhang, and D.~Zhang, ``A completed modeling of local binary pattern
  operator for texture classification,'' \emph{IEEE Transactions on Image
  Processing}, vol.~19, no.~6, pp. 1657--1663, 2010.

\bibitem{varma2005statistical}
M.~Varma and A.~Zisserman, ``A statistical approach to texture classification
  from single images,'' \emph{International Journal of Computer Vision},
  vol.~62, no. 1-2, pp. 61--81, 2005.

\bibitem{haralick1973textural}
R.~M. Haralick, K.~Shanmugam, and I.~H. Dinstein, ``Textural features for image
  classification,'' \emph{IEEE Transactions on Systems, Man and Cybernetics},
  no.~6, pp. 610--621, 1973.

\bibitem{gabor1946theory}
D.~Gabor, ``Theory of communication. part 1: The analysis of information,''
  \emph{Journal of the Institution of Electrical Engineers-Part III: Radio and
  Communication Engineering}, vol.~93, no.~26, pp. 429--441, 1946.

\bibitem{csurka2004visual}
G.~Csurka, C.~Dance, L.~Fan, J.~Willamowski, and C.~Bray, ``Visual
  categorization with bags of keypoints,'' in \emph{European Conference on
  Computer Vision Workshop on Statistical Learning in Computer Vision}, vol.~1,
  2004, p.~22.

\bibitem{Vestergaard2014}
B.~L.~L. Anders, S.~V. Jacob, and L.~Rasmus, ``Hep-2 cell classification using
  shape index histograms with donut-shaped spatial pooling,'' \emph{IEEE
  Transactions on Medical Imaging}, 2014.

\bibitem{bosch2007representing}
A.~Bosch, A.~Zisserman, and X.~Munoz, ``Representing shape with a spatial
  pyramid kernel,'' in \emph{ACM International Conference on Image and Video
  Retrieval}.\hskip 1em plus 0.5em minus 0.4em\relax ACM, 2007, pp. 401--408.

\bibitem{vedaldi11efficient}
A.~Vedaldi and A.~Zisserman, ``Efficient additive kernels via explicit feature
  maps,'' \emph{IEEE Transactions on Pattern Analysis and Machine
  Intellingence}, vol.~34, no.~3, 2011.

\bibitem{manivannan2014hep}
S.~Manivannan, W.~Li, S.~Akbar, R.~Wang, J.~Zhang, and S.~J. McKenna, ``Hep-2
  cell classification using multi-resolution local patterns and ensemble
  svms,'' in \emph{IEEE International Conference of Pattern Recognition
  Workshop on Pattern Recognition Techniques for Indirect Immunofluorescence
  Images (I3A)}.

\bibitem{perronnin2010improving}
F.~Perronnin, J.~S{\'a}nchez, and T.~Mensink, ``Improving the fisher kernel for
  large-scale image classification,'' in \emph{European Conference on Computer
  Vision 2010}.\hskip 1em plus 0.5em minus 0.4em\relax Springer, 2010, pp.
  143--156.

\bibitem{movellan2002tutorial}
J.~R. Movellan, ``Tutorial on gabor filters.''

\bibitem{zhang2005local}
W.~Zhang, S.~Shan, W.~Gao, X.~Chen, and H.~Zhang, ``Local gabor binary pattern
  histogram sequence (lgbphs): A novel non-statistical model for face
  representation and recognition,'' in \emph{IEEE International Conference on
  Computer Vision (ICCV)}, vol.~1.\hskip 1em plus 0.5em minus 0.4em\relax IEEE,
  2005, pp. 786--791.

\bibitem{lowe2004distinctive}
D.~G. Lowe, ``Distinctive image features from scale-invariant keypoints,''
  \emph{International Journal of Computer Vision}, vol.~60, no.~2, pp. 91--110,
  2004.

\bibitem{arandjelovic2012three}
R.~Arandjelovic and A.~Zisserman, ``Three things everyone should know to
  improve object retrieval,'' in \emph{IEEE Conference on Computer Vision and
  Pattern Recognition (CVPR)}.\hskip 1em plus 0.5em minus 0.4em\relax IEEE,
  2012, pp. 2911--2918.

\bibitem{sanchez2013image}
J.~S{\'a}nchez, F.~Perronnin, T.~Mensink, and J.~Verbeek, ``Image
  classification with the fisher vector: Theory and practice,''
  \emph{International Journal of Computer Vision}, pp. 1--24, 2013.

\bibitem{vedaldi2010vlfeat}
A.~Vedaldi and B.~Fulkerson, ``Vlfeat: An open and portable library of computer
  vision algorithms,'' in \emph{ACM Multimedia}, 2010, pp. 1469--1472.

\bibitem{fan2008liblinear}
R.~Fan, K.~Chang, C.~Hsieh, X.~Wang, and C.~Lin, ``Liblinear: A library for
  large linear classification,'' Journal of Machine Learning Research, 2008.

\end{thebibliography}
