Registration of Images with Outliers Using Joint Saliency Map
Binjie~Qin
~\IEEEmembership{Member,~IEEE,} Zhijun~Gu
~Xianjun~Sun
%\boldmath Mutual information (MI) is a popular similarity measure for image registration, whereby good registration can be achieved by maximizing the compactness of the clusters in the joint histogram. However, MI is sensitive to the ``outlier'' objects that appear in one image but not the other, and also suffers from local and biased maxima. We propose a novel joint saliency map (JSM) to highlight the corresponding salient structures in the two images, and emphatically group those salient structures into the smoothed compact clusters in the weighted joint histogram. This strategy could solve both the outlier and the local maxima problems. Experimental results show that the JSM-MI based algorithm is not only accurate but also robust for registration of challenging image pairs with outliers.
bare_jrnl_two_column.tex

%% bare_jrnl.tex
%% V1.3
%% 2007/01/11
%% by Michael Shell
%% see http://www.michaelshell.org/
%% for current contact information.
%%
%% This is a skeleton file demonstrating the use of IEEEtran.cls
%% (requires IEEEtran.cls version 1.7 or later) with an IEEE journal paper.
%%
%% Support sites:
%% http://www.michaelshell.org/tex/ieeetran/
%% http://www.ctan.org/tex-archive/macros/latex/contrib/IEEEtran/
%% and
%% http://www.ieee.org/



% *** Authors should verify (and, if needed, correct) their LaTeX system  ***
% *** with the testflow diagnostic prior to trusting their LaTeX platform ***
% *** with production work. IEEE's font choices can trigger bugs that do  ***
% *** not appear when using other class files.                            ***
% The testflow support page is at:
% http://www.michaelshell.org/tex/testflow/


%%*************************************************************************
%% Legal Notice:
%% This code is offered as-is without any warranty either expressed or
%% implied; without even the implied warranty of MERCHANTABILITY or
%% FITNESS FOR A PARTICULAR PURPOSE!
%% User assumes all risk.
%% In no event shall IEEE or any contributor to this code be liable for
%% any damages or losses, including, but not limited to, incidental,
%% consequential, or any other damages, resulting from the use or misuse
%% of any information contained here.
%%
%% All comments are the opinions of their respective authors and are not
%% necessarily endorsed by the IEEE.
%%
%% This work is distributed under the LaTeX Project Public License (LPPL)
%% ( http://www.latex-project.org/ ) version 1.3, and may be freely used,
%% distributed and modified. A copy of the LPPL, version 1.3, is included
%% in the base LaTeX documentation of all distributions of LaTeX released
%% 2003/12/01 or later.
%% Retain all contribution notices and credits.
%% ** Modified files should be clearly indicated as such, including  **
%% ** renaming them and changing author support contact information. **
%%
%% File list of work: IEEEtran.cls, IEEEtran_HOWTO.pdf, bare_adv.tex,
%%                    bare_conf.tex, bare_jrnl.tex, bare_jrnl_compsoc.tex
%%*************************************************************************

% Note that the a4paper option is mainly intended so that authors in
% countries using A4 can easily print to A4 and see how their papers will
% look in print - the typesetting of the document will not typically be
% affected with changes in paper size (but the bottom and side margins will).
% Use the testflow package mentioned above to verify correct handling of
% both paper sizes by the user's LaTeX system.
%
% Also note that the "draftcls" or "draftclsnofoot", not "draft", option
% should be used if it is desired that the figures are to be displayed in
% draft mode.
%
%\newcommand{\CLASSINPUTbaselinestretch}{1}
\documentclass[10pt,twocolumn,final]{IEEEtran}
\usepackage[dvips]{graphicx}
\usepackage[para]{threeparttable}
\usepackage{amsmath}
\usepackage[noadjust]{cite}
% If IEEEtran.cls has not been installed into the LaTeX system files,
% manually specify the path to it like:
% \documentclass[journal]{../sty/IEEEtran}




% Some very useful LaTeX packages include:
% (uncomment the ones you want to load)


% *** MISC UTILITY PACKAGES ***
%
%\usepackage{ifpdf}
% Heiko Oberdiek's ifpdf.sty is very useful if you need conditional
% compilation based on whether the output is pdf or dvi.
% usage:
% \ifpdf
%   % pdf code
% \else
%   % dvi code
% \fi
% The latest version of ifpdf.sty can be obtained from:
% http://www.ctan.org/tex-archive/macros/latex/contrib/oberdiek/
% Also, note that IEEEtran.cls V1.7 and later provides a builtin
% \ifCLASSINFOpdf conditional that works the same way.
% When switching from latex to pdflatex and vice-versa, the compiler may
% have to be run twice to clear warning/error messages.






% *** CITATION PACKAGES ***
%
%\usepackage{cite}
% cite.sty was written by Donald Arseneau
% V1.6 and later of IEEEtran pre-defines the format of the cite.sty package
% \cite{} output to follow that of IEEE. Loading the cite package will
% result in citation numbers being automatically sorted and properly
% "compressed/ranged". e.g., [1], [9], [2], [7], [5], [6] without using
% cite.sty will become [1], [2], [5]--[7], [9] using cite.sty. cite.sty's
% \cite will automatically add leading space, if needed. Use cite.sty's
% noadjust option (cite.sty V3.8 and later) if you want to turn this off.
% cite.sty is already installed on most LaTeX systems. Be sure and use
% version 4.0 (2003-05-27) and later if using hyperref.sty. cite.sty does
% not currently provide for hyperlinked citations.
% The latest version can be obtained at:
% http://www.ctan.org/tex-archive/macros/latex/contrib/cite/
% The documentation is contained in the cite.sty file itself.






% *** GRAPHICS RELATED PACKAGES ***
%
\ifCLASSINFOpdf
  % \usepackage[pdftex]{graphicx}
  % declare the path(s) where your graphic files are
  % \graphicspath{{../pdf/}{../jpeg/}}
  % and their extensions so you won't have to specify these with
  % every instance of \includegraphics
  % \DeclareGraphicsExtensions{.pdf,.jpeg,.png}
\else
  % or other class option (dvipsone, dvipdf, if not using dvips). graphicx
  % will default to the driver specified in the system graphics.cfg if no
  % driver is specified.
  % \usepackage[dvips]{graphicx}
  % declare the path(s) where your graphic files are
  % \graphicspath{{../eps/}}
  % and their extensions so you won't have to specify these with
  % every instance of \includegraphics
  % \DeclareGraphicsExtensions{.eps}
\fi
% graphicx was written by David Carlisle and Sebastian Rahtz. It is
% required if you want graphics, photos, etc. graphicx.sty is already
% installed on most LaTeX systems. The latest version and documentation can
% be obtained at:
% http://www.ctan.org/tex-archive/macros/latex/required/graphics/
% Another good source of documentation is "Using Imported Graphics in
% LaTeX2e" by Keith Reckdahl which can be found as epslatex.ps or
% epslatex.pdf at: http://www.ctan.org/tex-archive/info/
%
% latex, and pdflatex in dvi mode, support graphics in encapsulated
% postscript (.eps) format. pdflatex in pdf mode supports graphics
% in .pdf, .jpeg, .png and .mps (metapost) formats. Users should ensure
% that all non-photo figures use a vector format (.eps, .pdf, .mps) and
% not a bitmapped formats (.jpeg, .png). IEEE frowns on bitmapped formats
% which can result in "jaggedy"/blurry rendering of lines and letters as
% well as large increases in file sizes.
%
% You can find documentation about the pdfTeX application at:
% http://www.tug.org/applications/pdftex





% *** MATH PACKAGES ***
%
%\usepackage[cmex10]{amsmath}
% A popular package from the American Mathematical Society that provides
% many useful and powerful commands for dealing with mathematics. If using
% it, be sure to load this package with the cmex10 option to ensure that
% only type 1 fonts will utilized at all point sizes. Without this option,
% it is possible that some math symbols, particularly those within
% footnotes, will be rendered in bitmap form which will result in a
% document that can not be IEEE Xplore compliant!
%
% Also, note that the amsmath package sets \interdisplaylinepenalty to 10000
% thus preventing page breaks from occurring within multiline equations. Use:
%\interdisplaylinepenalty=2500
% after loading amsmath to restore such page breaks as IEEEtran.cls normally
% does. amsmath.sty is already installed on most LaTeX systems. The latest
% version and documentation can be obtained at:
% http://www.ctan.org/tex-archive/macros/latex/required/amslatex/math/





% *** SPECIALIZED LIST PACKAGES ***
%
%\usepackage{algorithmic}
% algorithmic.sty was written by Peter Williams and Rogerio Brito.
% This package provides an algorithmic environment fo describing algorithms.
% You can use the algorithmic environment in-text or within a figure
% environment to provide for a floating algorithm. Do NOT use the algorithm
% floating environment provided by algorithm.sty (by the same authors) or
% algorithm2e.sty (by Christophe Fiorio) as IEEE does not use dedicated
% algorithm float types and packages that provide these will not provide
% correct IEEE style captions. The latest version and documentation of
% algorithmic.sty can be obtained at:
% http://www.ctan.org/tex-archive/macros/latex/contrib/algorithms/
% There is also a support site at:
% http://algorithms.berlios.de/index.html
% Also of interest may be the (relatively newer and more customizable)
% algorithmicx.sty package by Szasz Janos:
% http://www.ctan.org/tex-archive/macros/latex/contrib/algorithmicx/




% *** ALIGNMENT PACKAGES ***
%
%\usepackage{array}
% Frank Mittelbach's and David Carlisle's array.sty patches and improves
% the standard LaTeX2e array and tabular environments to provide better
% appearance and additional user controls. As the default LaTeX2e table
% generation code is lacking to the point of almost being broken with
% respect to the quality of the end results, all users are strongly
% advised to use an enhanced (at the very least that provided by array.sty)
% set of table tools. array.sty is already installed on most systems. The
% latest version and documentation can be obtained at:
% http://www.ctan.org/tex-archive/macros/latex/required/tools/


%\usepackage{mdwmath}
%\usepackage{mdwtab}
% Also highly recommended is Mark Wooding's extremely powerful MDW tools,
% especially mdwmath.sty and mdwtab.sty which are used to format equations
% and tables, respectively. The MDWtools set is already installed on most
% LaTeX systems. The lastest version and documentation is available at:
% http://www.ctan.org/tex-archive/macros/latex/contrib/mdwtools/


% IEEEtran contains the IEEEeqnarray family of commands that can be used to
% generate multiline equations as well as matrices, tables, etc., of high
% quality.


%\usepackage{eqparbox}
% Also of notable interest is Scott Pakin's eqparbox package for creating
% (automatically sized) equal width boxes - aka "natural width parboxes".
% Available at:
% http://www.ctan.org/tex-archive/macros/latex/contrib/eqparbox/





% *** SUBFIGURE PACKAGES ***
%\usepackage[tight,footnotesize]{subfigure}
% subfigure.sty was written by Steven Douglas Cochran. This package makes it
% easy to put subfigures in your figures. e.g., "Figure 1a and 1b". For IEEE
% work, it is a good idea to load it with the tight package option to reduce
% the amount of white space around the subfigures. subfigure.sty is already
% installed on most LaTeX systems. The latest version and documentation can
% be obtained at:
% http://www.ctan.org/tex-archive/obsolete/macros/latex/contrib/subfigure/
% subfigure.sty has been superceeded by subfig.sty.



%\usepackage[caption=false]{caption}
%\usepackage[font=footnotesize]{subfig}
% subfig.sty, also written by Steven Douglas Cochran, is the modern
% replacement for subfigure.sty. However, subfig.sty requires and
% automatically loads Axel Sommerfeldt's caption.sty which will override
% IEEEtran.cls handling of captions and this will result in nonIEEE style
% figure/table captions. To prevent this problem, be sure and preload
% caption.sty with its "caption=false" package option. This is will preserve
% IEEEtran.cls handing of captions. Version 1.3 (2005/06/28) and later
% (recommended due to many improvements over 1.2) of subfig.sty supports
% the caption=false option directly:
%\usepackage[caption=false,font=footnotesize]{subfig}
%
% The latest version and documentation can be obtained at:
% http://www.ctan.org/tex-archive/macros/latex/contrib/subfig/
% The latest version and documentation of caption.sty can be obtained at:
% http://www.ctan.org/tex-archive/macros/latex/contrib/caption/




% *** FLOAT PACKAGES ***
%
%\usepackage{fixltx2e}
% fixltx2e, the successor to the earlier fix2col.sty, was written by
% Frank Mittelbach and David Carlisle. This package corrects a few problems
% in the LaTeX2e kernel, the most notable of which is that in current
% LaTeX2e releases, the ordering of single and double column floats is not
% guaranteed to be preserved. Thus, an unpatched LaTeX2e can allow a
% single column figure to be placed prior to an earlier double column
% figure. The latest version and documentation can be found at:
% http://www.ctan.org/tex-archive/macros/latex/base/



%\usepackage{stfloats}
% stfloats.sty was written by Sigitas Tolusis. This package gives LaTeX2e
% the ability to do double column floats at the bottom of the page as well
% as the top. (e.g., "\begin{figure*}[!b]" is not normally possible in
% LaTeX2e). It also provides a command:
%\fnbelowfloat
% to enable the placement of footnotes below bottom floats (the standard
% LaTeX2e kernel puts them above bottom floats). This is an invasive package
% which rewrites many portions of the LaTeX2e float routines. It may not work
% with other packages that modify the LaTeX2e float routines. The latest
% version and documentation can be obtained at:
% http://www.ctan.org/tex-archive/macros/latex/contrib/sttools/
% Documentation is contained in the stfloats.sty comments as well as in the
% presfull.pdf file. Do not use the stfloats baselinefloat ability as IEEE
% does not allow \baselineskip to stretch. Authors submitting work to the
% IEEE should note that IEEE rarely uses double column equations and
% that authors should try to avoid such use. Do not be tempted to use the
% cuted.sty or midfloat.sty packages (also by Sigitas Tolusis) as IEEE does
% not format its papers in such ways.


%\ifCLASSOPTIONcaptionsoff
%  \usepackage[nomarkers]{endfloat}
% \let\MYoriglatexcaption\caption
% \renewcommand{\caption}[2][\relax]{\MYoriglatexcaption[#2]{#2}}
%\fi
% endfloat.sty was written by James Darrell McCauley and Jeff Goldberg.
% This package may be useful when used in conjunction with IEEEtran.cls'
% captionsoff option. Some IEEE journals/societies require that submissions
% have lists of figures/tables at the end of the paper and that
% figures/tables without any captions are placed on a page by themselves at
% the end of the document. If needed, the draftcls IEEEtran class option or
% \CLASSINPUTbaselinestretch interface can be used to increase the line
% spacing as well. Be sure and use the nomarkers option of endfloat to
% prevent endfloat from "marking" where the figures would have been placed
% in the text. The two hack lines of code above are a slight modification of
% that suggested by in the endfloat docs (section 8.3.1) to ensure that
% the full captions always appear in the list of figures/tables - even if
% the user used the short optional argument of \caption[]{}.
% IEEE papers do not typically make use of \caption[]'s optional argument,
% so this should not be an issue. A similar trick can be used to disable
% captions of packages such as subfig.sty that lack options to turn off
% the subcaptions:
% For subfig.sty:
% \let\MYorigsubfloat\subfloat
% \renewcommand{\subfloat}[2][\relax]{\MYorigsubfloat[]{#2}}
% For subfigure.sty:
% \let\MYorigsubfigure\subfigure
% \renewcommand{\subfigure}[2][\relax]{\MYorigsubfigure[]{#2}}
% However, the above trick will not work if both optional arguments of
% the \subfloat/subfig command are used. Furthermore, there needs to be a
% description of each subfigure *somewhere* and endfloat does not add
% subfigure captions to its list of figures. Thus, the best approach is to
% avoid the use of subfigure captions (many IEEE journals avoid them anyway)
% and instead reference/explain all the subfigures within the main caption.
% The latest version of endfloat.sty and its documentation can obtained at:
% http://www.ctan.org/tex-archive/macros/latex/contrib/endfloat/
%
% The IEEEtran \ifCLASSOPTIONcaptionsoff conditional can also be used
% later in the document, say, to conditionally put the References on a
% page by themselves.





% *** PDF, URL AND HYPERLINK PACKAGES ***
%
%\usepackage{url}
% url.sty was written by Donald Arseneau. It provides better support for
% handling and breaking URLs. url.sty is already installed on most LaTeX
% systems. The latest version can be obtained at:
% http://www.ctan.org/tex-archive/macros/latex/contrib/misc/
% Read the url.sty source comments for usage information. Basically,
% \url{my_url_here}.





% *** Do not adjust lengths that control margins, column widths, etc. ***
% *** Do not use packages that alter fonts (such as pslatex).         ***
% There should be no need to do such things with IEEEtran.cls V1.6 and later.
% (Unless specifically asked to do so by the journal or conference you plan
% to submit to, of course. )


% correct bad hyphenation here
\hyphenation{op-tical net-works semi-conduc-tor}


\begin{document}
%
% paper title
% can use linebreaks \\ within to get better formatting as desired
\title{Registration of Images with Outliers Using Joint Saliency Map}
%
%
% author names and IEEE memberships
% note positions of commas and nonbreaking spaces ( ~ ) LaTeX will not break
% a structure at a ~ so this keeps an author's name from being broken across
% two lines.
% use \thanks{} to gain access to the first footnote area
% a separate \thanks must be used for each paragraph as LaTeX2e's \thanks
% was not built to handle multiple paragraphs
%

\author{Binjie~Qin,~\IEEEmembership{Member,~IEEE,}
        Zhijun~Gu,~Xianjun~Sun,~and~Yisong Lv% <-this % stops a space
\thanks{This work was supported in part by NSFC (60872102), NBRPC (2010CB834300),
the Science Foundation of Shanghai Municipal Science \& Technology Commission (04JC14060), Shanghai Municipal Health Bureau (2008115)
and the small animal imaging project (06-545).}% <-this % stops a space
\thanks{Binjie Qin, Zhijun Gu and Xianjun Sun are with the Department of Biomedical Engineering, School of Life Sciences \& Biotechnology, Shanghai Jiao Tong University, Shanghai, 200240, China (e-mail: bjqin@sjtu.edu.cn; gzj0126@gmail.com; sxj\_sun@sjtu.edu.cn).}
\thanks{Yisong Lv is with the Department of Mathematics, Shanghai Jiao Tong University (e-mail: yslv@sjtu.edu.cn).}
\thanks{It is preprint version for only personal use.}}


% note the % following the last \IEEEmembership and also \thanks -
% these prevent an unwanted space from occurring between the last author name
% and the end of the author line. i.e., if you had this:
%
% \author{....lastname \thanks{...} \thanks{...} }
%                     ^------------^------------^----Do not want these spaces!
%
% a space would be appended to the last name and could cause every name on that
% line to be shifted left slightly. This is one of those "LaTeX things". For
% instance, "\textbf{A} \textbf{B}" will typeset as "A B" not "AB". To get
% "AB" then you have to do: "\textbf{A}\textbf{B}"
% \thanks is no different in this regard, so shield the last } of each \thanks
% that ends a line with a % and do not let a space in before the next \thanks.
% Spaces after \IEEEmembership other than the last one are OK (and needed) as
% you are supposed to have spaces between the names. For what it is worth,
% this is a minor point as most people would not even notice if the said evil
% space somehow managed to creep in.



% The paper headers
\markboth{Preprint version for IEEE SIGNAL PROCESSING LETTERS}%
{Shell \MakeLowercase{\textit{et al.}}: Bare Demo of IEEEtran.cls for Journals}
% The only time the second header will appear is for the odd numbered pages
% after the title page when using the twoside option.
%
% *** Note that you probably will NOT want to include the author's ***
% *** name in the headers of peer review papers.                   ***
% You can use \ifCLASSOPTIONpeerreview for conditional compilation here if
% you desire.




% If you want to put a publisher's ID mark on the page you can do it like
% this:
%\IEEEpubid{0000--0000/00\$00.00~\copyright~2007 IEEE}
% Remember, if you use this you must call \IEEEpubidadjcol in the second
% column for its text to clear the IEEEpubid mark.



% use for special paper notices
%\IEEEspecialpapernotice{(Invited Paper)}




% make the title area
\maketitle


\begin{abstract}
%\boldmath
Mutual information (MI) is a popular similarity measure for image registration, whereby good registration can be achieved by maximizing the compactness of the clusters in the joint histogram. However, MI is sensitive to the ``outlier'' objects that appear in one image but not the other, and also suffers from local and biased maxima. We propose a novel joint saliency map (JSM) to highlight the corresponding salient structures in the two images, and emphatically group those salient structures into the smoothed compact clusters in the weighted joint histogram. This strategy could solve both the outlier and the local maxima problems. Experimental results show that the JSM-MI based algorithm is not only accurate but also robust for registration of challenging image pairs with outliers.
\end{abstract}
% IEEEtran.cls defaults to using nonbold math in the Abstract.
% This preserves the distinction between vectors and scalars. However,
% if the journal you are submitting to favors bold math in the abstract,
% then you can use LaTeX's standard command \boldmath at the very start
% of the abstract to achieve this. Many IEEE journals frown on math
% in the abstract anyway.

% Note that keywords are not normally used for peerreview papers.
\begin{IEEEkeywords}
image registration, mutual information, outliers, joint saliency map, weighted joint histogram.
\end{IEEEkeywords}






% For peer review papers, you can put extra information on the cover
% page as needed:
% \ifCLASSOPTIONpeerreview
% \begin{center} \bfseries EDICS Category: 3-BBND \end{center}
% \fi
%
% For peerreview papers, this IEEEtran command inserts a page break and
% creates the second title. It will be ignored for other modes.
%\IEEEpeerreviewmaketitle



\section{Introduction}
% The very first letter is a 2 line initial drop letter followed
% by the rest of the first word in caps.
%
% form to use if the first word consists of a single letter:
% \IEEEPARstart{A}{demo} file is ....
%
% form to use if you need the single drop letter followed by
% normal text (unknown if ever used by IEEE):
% \IEEEPARstart{A}{}demo file is ....
%
% Some journals put the first two words in caps:
% \IEEEPARstart{T}{his demo} file is ....
%
% Here we have the typical use of a "T" for an initial drop letter
% and "HIS" in caps to complete the first word.
\IEEEPARstart{I}{mage} registration can be considered as finding the optimal transformation $T$ between the reference image $I_R$ and the floating image $I_F$ to maximize a defined similarity measure such as mutual information (MI). Since 1995 \cite{1}\cite{2}, MI has been proved to be very effective in image registration. The MI between $I_R $ and $I_F $ (with intensity bins $r$ and $f$) is defined as:
\begin{equation}
\label{eq1}
\mbox{MI}=H\left({I_R }\right)+H\left({I_F}\right)-H\left({I_R,I_F}\right)
\end{equation}
where $H\left({I}\right)=-\sum_{i}p\left({i}\right)\log p\left({i}\right)$ and $H\left({I_R,I_F}\right)=-\sum_{r,f}p\left({r,f}\right)\log p\left({r,f}\right)$ are the entropy of the intensities of image $I$ and the entropy of the joint intensities of two
images, $p\left({i}\right)$ is the intensity probabilities with $p\left(r\right)=\sum\nolimits_f{p\left({r,f}\right)}$ and $p\left(f\right)=\sum\nolimits_r {p\left({r,f}\right)}$, $p\left( {r,f}\right)$ is the joint intensity probabilities estimated by the joint histogram $h\left({r,f}\right)$.

MI-based registration methods take advantage of the fact that properly registered images usually correspond to compactly-clustered joint histograms \cite{3}. They measure the joint histogram dispersion by computing the entropy of the joint intensity probabilities. When the images become misregistered, the compact clusters become disperse sets of points in the joint histogram and the entropy of the joint intensity probabilities increases. Making no assumptions about the form of the intensity mapping between the two images, MI is sensitive to the unmatchable outliers, e.g. the tumor resection in the intra- and pre-operative brain images (see Figs. 1a-b). To reject the outliers, some approaches are proposed including consistency test \cite{4}, intensity transformation \cite{5}, gradient-based asymmetric multifeature MI \cite{6} and graph-based multifeature MI \cite{7}. However, all these methods do not emphasize the corresponding salient structures in the two
images to suppress the outliers. Furthermore, MI likely suffers
from local and biased maxima \cite{8} which are caused by the ambiguities in defining
structure correspondence.
\begin{figure}[!t]
\centerline{\includegraphics[width=3.45in,height=0.80in]{fig1}}
\caption{(a)-(b) Intra-operative and pre-operative MR image with a large tumor
resection. (c) Joint histogram dispersion with two clotted clusters (dark
red in pseudo color). (d) Joint saliency map for (a) and (b).}
\label{fig1}
\end{figure}

Spatial information, i.e. the dependence of the intensities of neighboring
pixels, has been included in MI \cite{9}-\cite{12} to improve
registration. Nevertheless, almost all MI-based methods equally
treat each overlapping pixel pair as a separate point in the overlap area to calculate the joint histogram. This could raise three
issues: 1) when we equally consider the outlier pixel pairs,
the noncorresponding structures overlap and the histogram will show certain
clusters for the grey values of the outliers. These clusters easily
introduce the histogram dispersion (see Fig. 1c) with increasing misregistration; 2) while registration can be achieved by maximizing the compactness of the histogram, the undesired clotted clusters (see Fig. 1c) related to many noisy pixel pairs in the structureless
regions, such as background and white matter in the brain image, increase the MI ambiguities and the local maxima \cite{8} (Fig. 5c shows that the normalized MI \cite{1}\cite{20} is in a biased global maximum when the whole background areas in the two endoscopic images
are exactly aligned); 3) when we group the intensity pairs as separate
points into the histogram, the independence of the neighboring bins could increase the MI ambiguities and the
local maxima. To solve this problem, joint histogram smoothing (or blurring) \cite{6}\cite{8} has been used to increase the dependence of the neighboring histogram bins. We address these issues above as follows.

In fact, image registration is to match the corresponding salient structures in both images. To suppress the outliers and the homogeneous pixel pairs, the corresponding pixel pairs in the corresponding salient structures should contribute more to the joint histogram. For example, the corresponding salient pixel pairs in the normal brain tissues should be given more weight in the histogram
than the homogeneous and the tumor resection pixel pairs. To weight each overlapping pixel pair when computing the joint histogram, we propose a novel joint saliency map (JSM) to assign a joint saliency value between 0 and 1 to the pixel pair. The idea of JSM is demonstrated schematically in Fig. 1d, where the high joint saliency values are assigned to the corresponding salient pixel pairs rather than the outlier and the homogeneous pixel pairs.

The JSM is determined by correlating each overlapping pixel pair's respective regional saliency vectors
(RSVs). The RSV characterizes the regional salient structure around each underlying pixel after a principal axis analysis (PAA) of the pixel's regional saliency distribution. In the JSM-weighted joint histogram (WJH), the contributions of the corresponding salient structures are distributed over neighboring histogram bins. This leads to the smoothing of the compact clusters for the grey values of the corresponding salient structures, which can solve both the outlier and the local maxima problems.

The proposed JSM-MI has been applied to the rigid registration of 2D images.
Experimental results show that, compared to other MI-based registration
methods, JSM-MI method achieves better robustness and higher
accuracy for the registration of challenging image pairs with outliers. The
letter is organized as follows. We first introduce the JSM for WJH in MI. Next, we report some experiment results to identify the registration performance on accuracy and robustness. Finally, the conclusions close this letter.


% An example of a floating figure using the graphicx package.
% Note that \label must occur AFTER (or within) \caption.
% For figures, \caption should occur after the \includegraphics.
% Note that IEEEtran v1.7 and later has special internal code that
% is designed to preserve the operation of \label within \caption
% even when the captionsoff option is in effect. However, because
% of issues like this, it may be the safest practice to put all your
% \label just after \caption rather than within \caption{}.
%
% Reminder: the "draftcls" or "draftclsnofoot", not "draft", class
% option should be used if it is desired that the figures are to be
% displayed while in draft mode.
%
%\begin{figure}[!t]
%\centering
%\includegraphics[width=2.5in]{myfigure}
% where an .eps filename suffix will be assumed under latex,
% and a .pdf suffix will be assumed for pdflatex; or what has been declared
% via \DeclareGraphicsExtensions.
%\caption{Simulation Results}
%\label{fig_sim}
%\end{figure}

% Note that IEEE typically puts floats only at the top, even when this
% results in a large percentage of a column being occupied by floats.


% An example of a double column floating figure using two subfigures.
% (The subfig.sty package must be loaded for this to work.)
% The subfigure \label commands are set within each subfloat command, the
% \label for the overall figure must come after \caption.
% \hfil must be used as a separator to get equal spacing.
% The subfigure.sty package works much the same way, except \subfigure is
% used instead of \subfloat.
%
%\begin{figure*}[!t]
%\centerline{\subfloat[Case I]\includegraphics[width=2.5in]{subfigcase1}%
%\label{fig_first_case}}
%\hfil
%\subfloat[Case II]{\includegraphics[width=2.5in]{subfigcase2}%
%\label{fig_second_case}}}
%\caption{Simulation results}
%\label{fig_sim}
%\end{figure*}
%
% Note that often IEEE papers with subfigures do not employ subfigure
% captions (using the optional argument to \subfloat), but instead will
% reference/describe all of them (a), (b), etc., within the main caption.


% An example of a floating table. Note that, for IEEE style tables, the
% \caption command should come BEFORE the table. Table text will default to
% \footnotesize as IEEE normally uses this smaller font for tables.
% The \label must come after \caption as always.
%
%\begin{table}[!t]
%% increase table row spacing, adjust to taste
%\renewcommand{\arraystretch}{1.3}
% if using array.sty, it might be a good idea to tweak the value of
% \extrarowheight as needed to properly center the text within the cells
%\caption{An Example of a Table}
%\label{table_example}
%\centering
%% Some packages, such as MDW tools, offer better commands for making tables
%% than the plain LaTeX2e tabular which is used here.
%\begin{tabular}{|c||c|}
%\hline
%One & Two\\
%\hline
%Three & Four\\
%\hline
%\end{tabular}
%\end{table}


% Note that IEEE does not put floats in the very first column - or typically
% anywhere on the first page for that matter. Also, in-text middle ("here")
% positioning is not used. Most IEEE journals use top floats exclusively.
% Note that, LaTeX2e, unlike IEEE journals, places footnotes above bottom
% floats. This can be corrected via the \fnbelowfloat command of the
% stfloats package.


\section{Methods}
\subsection{Regional Saliency Vector}
We use visual saliency operator to enhance the regional salient structures we are interested in. Many techniques have been developed to define the saliency of image, i.e., using edge gradient, local phase \cite{12}, salient regions \cite{13}, corner and keypoints \cite{14}. Gradient map has been incorporated into the MI-based registration methods \cite{9}-\cite{11}. However, gradient is a local feature and sensitive to noise. Local phase \cite{12} and salient regions \cite{15} suffer from high computational complexity. Corner and keypoint can not be defined for each image pixel. Inspired by the center-surround mechanism \cite{16}\cite{17} which has defined the intensity-contrast-based visual saliency map, we define a two-step scale and rotation invariant saliency operator based on intensity contrast as follows:
\begin{equation}
\label{eq1}
S_l (v)=\sum\nolimits_{u\in N_v } {\left( {I_l \left( v \right)-I_l \left( u
\right)} \right)^2}
\end{equation}
where $N_v $ is the 1-pixel radius circular neighborhood of the pixel position $v=\left( {x,y} \right)$ at scale $l$, $S_l (v)$ is the local saliency computed for the intensity $I_l (v)$ in the Gaussian image pyramid \cite{18} at scale $l$, $I_l (u)$ is the intensity of the pixel in the $I_l (v)$'s neighborhood. The multiscale local saliency map $S(x,y)$ at the finest scale is reconstructed by summing up all the saliency maps at the coarser scales.

In the second step, a PAA of the saliency distribution in a certain region assigns \emph{regional saliency} to each pixel based on the inertia matrix:
\begin{equation}
\label{eq1}
\boldsymbol{M}=\left[ {\begin{array}{l}
 \mu _{20} \;\;\;\;\mu _{11} \\
 \mu _{11} \;\;\;\;\mu _{02} \\
 \end{array}} \right]
\end{equation}
where $\mu_{jk} =\sum\nolimits {(x-g_x )^j(y-g_y )^k} S(x,y)$ , $(g_x,g_y
)=({{m_{10} } \mathord{\left/ {\vphantom {{m_{10} } {m_{00} }}} \right.
\kern-\nulldelimiterspace} {m_{00} },m_{01} } \mathord{\left/ {\vphantom
{{{m_{10} } \mathord{\left/ {\vphantom {{m_{10} } {m_{00} }}} \right.
\kern-\nulldelimiterspace} {m_{00} },m_{01} } {m_{00} }}} \right.
\kern-\nulldelimiterspace} {m_{00} })$ and $m_{jk} =\sum\nolimits {x^jy^kS(x,y)}$ are the central $(j,k)\mbox{-}$ moment, the centroid and the $(j,k)\mbox{-}$ moment of the saliency distribution $S(x,y)$ in the 5.5-pixel radius circular neighborhood around each pixel. This regional saliency distribution describes a 2D regional salient structure. The two eigenvectors of the matrix $\boldsymbol{M}$ represent the orthogonal coordinate system within the regional salient structure, while the corresponding eigenvalues give information about the length of the respective axes. Because the regional information about the orientation of the salient structure is mostly stored along the first eigenvector corresponding to the largest eigenvalue, the first eigenvector referred as the RSV is enough to represent the regional salient structure around a pixel (see Figs. 2a-b).

\begin{figure}[!t]
\centerline{\includegraphics[width=3.45in,height=0.80in]{fig2}}
\caption{(a)-(b) RSVs for the sub-blocks in the reference and the floating images (size: $400\times300$ pixels).}
\label{fig2}
\end{figure}


\subsection{Joint Saliency Map}
Given two RSVs of each overlapping pixel pair, JSM is ready to describe the matching degree between the two RSVs. The inner product of two RSVs is a measure of their co-linearity and naturally can be used as their similarity
measure. The essential idea of JSM is an assumption, which
is always valid in practice according to the empirical experience in image
registration: for two precisely aligned multi-modal (or multi-temporal)
images, the majority of the corresponding pixel locations are very likely to
produce the RSVs with similar orientations (see Figs. 2a-b). This is because the
two images under registration fundamentally depict the same image
structures. As a result, the RSVs of the corresponding pixel locations from
two images could present relatively coincident orientations in general.
Therefore, the angle $\theta$ between the two RSVs (${\rm {\bf x}}_R $, ${\rm {\bf x}}_F)$ is
simply calculated, making $\cos\theta$ the scalar measure of the joint
saliency value $w\left(v \right)$:
\begin{equation}
\label{eq1}
w\left(v \right)\mbox{=}\cos \theta \left( {{\rm {\bf x}}_R ,{\rm {\bf
x}}_F } \right)={\left\langle {{\rm {\bf x}}_R ,{\rm {\bf x}}_F }
\right\rangle } \mathord{\left/ {\vphantom {{\left\langle {{\rm {\bf x}}_R
,{\rm {\bf x}}_F } \right\rangle } {\left\| {{\rm {\bf x}}_R } \right\|\cdot
\left\| {{\rm {\bf x}}_F } \right\|}}} \right. \kern-\nulldelimiterspace}
{\left\| {{\rm {\bf x}}_R } \right\|\cdot \left\| {{\rm {\bf x}}_F }
\right\|}
\end{equation}


A JSM value near one suggests that the underlying pixel pair originates from the corresponding salient
structures. Contrarily, a JSM value near zero indicates that the underlying pixel pair comes from either the outliers or a homogeneous region. To speed up the registration without reducing accuracy, the pixel with a small saliency value below a threshold value (10 percent of the maximum saliency value) is assigned a zero JSM value directly. The JSM would primarily respond to the high-gradient edge pixels if a high threshold value is chosen. However, the JSM does not simply emphasize the common image gradients in the two images. Figs. 3d-f present the image gradient and the JSM profiles of the same line (marked as dashed lines across the tumor areas) at the two registered images (see Figs. 3a-b). As shown in the figures, the image gradient features in Figs. 3d-e are very noisy and do not agree with each other at each overlapping location. The JSM in Fig. 3f can accurately preserve the corresponding salient structures in larger capture range with smaller variability than the image gradients.


\begin{figure}[!t]
\centerline{\includegraphics[width=2.44in,height=1.60in]{fig3}}
\caption{(a)-(b) The reference and the floating images for the gradient magnitude
and the JSM magnitude. (c) Compact JSM-WJH smoothing for (a)-(b). (d)-(e)
Gradient value profiles of the lines in (a)-(b), which are marked as dashed
lines. (f) JSM value profiles of the lines in (a)-(b).}
\label{fig3}
\end{figure}


\subsection{JSM-Weighted Joint Histogram }
The contribution of the interpolated floating intensity $f(v_f)$ to the joint histogram is weighted by a $w(v)$ of the JSM (the pixel positions ($v_r$,$v_f$) are overlapped at the position $v$). For 2D image registration, if using a nearest neighbor or a bilinear interpolation, the value $w(v)$ should be added to the histogram entry $h(r,f)$. In bilinear partial volume distribution (PV) interpolation, the contribution of the $f(v_f)$ to the histogram, distributed over the intensity values of all nearest neighbors of the reference pixel position $v_r$ on the grid of $I_R$, is weighted using the $w(v)$. Similarly, JSM could be easily incorporated into other interpolation schemes and Parzen-based joint histogram.

In the JSM-WJH, the outliers and homogeneous regions have little impact on the histogram distribution.
Furthermore, each histogram entry for the corresponding salient structures is the sum of smoothly varying
fractions of one, such that the histogram changes smoothly in the neighboring bins related to those structures. As a result, the compact histogram smoothing (see Fig. 3c) is introduced by highlighting the grey values of the corresponding salient structures. Computed from the compact and smooth histogram, the MI is then maximized to achieve robust and accurate rigid registration.

\subsection{Computational complexity}
The JSM should be re-calculated with the transformation changing the overlap area at each registration iteration. The RSV orientation for a JSM calculation could be easily re-oriented as it is done in the diffusion tensor image registration \cite{19}. Nevertheless, to ensure the numerical stability and the computation speedup, a new JSM at each iteration can be simply updated from the JSM of the previous iteration through the PV interpolation. The JSM could be re-calculated after $n$ iterations ($n{=}10\sim15$) to reflect the updated correspondence between the salient structures in the two images.

\section{Experimental results}
We evaluated our JSM-MI-based (JMI) algorithm on 11 challenging image pairs including CT-PET tumor images, MR brain tumor resection images, optical images with background/foreground clutter and etc. We implemented the JMI algorithm using the simplex optimization in a multiresolution scheme \cite{18}. The algorithm stops if the current step length is smaller than ${10^{-5}}$ or if it has reached the limit of 200 evaluation numbers. The challenging image pairs include some complex outliers that the normalized MI-based method and four of MI-based adaptations with incorporating spatial information fail to deal with. Due to space restrictions, we only show some typical experimental results in this letter.

\begin{figure}[!t]
\centerline{\includegraphics[width=3.44in,height=2.5in]{fig4}}
\caption{Registration results for the two images in Figs. 2a-b. (a) JMI. The yellow contour overlap of the book validates the registration accuracy owing
to the additive mixing of red and green. (b) NMI. (c) RMI. (d)
HMI. (e) GMI. (f) PMI. (g)-(h) NMI and JMI similarity surfaces plotted as a function of $x$ and $y$ translation (within a range of $\pm10$ pixels around the matching position)}
\label{fig4}
\end{figure}

Fig. 4 shows the various registration results for
the two images at Fig. 2 with a foreground book and the large changes of background appearance. To facilitate the visual assessment of registration accuracy,
the green floating contours and the red reference contours obtained by Canny-Deriche edge detector have been overlaid
over each other. The sub-pixel registration accuracy (see Table I. case 1)
of our JMI algorithm can be validated by the book's yellow contour overlap, which is due to the additive color mixing of
the green and the red contour (see Fig. 4a).

Using particle swarm optimization (PSO) to deal with the local maxima,
the other methods based on normalized MI (NMI)
\cite{1}\cite{20}, regional MI (RMI)
\cite{21}, high-dimensional MI (HMI)
\cite{22}, MI with gradient information (GMI)
\cite{10}, and phase MI (PMI)
\cite{12} show different misregistration results in Figs. 4b-f. The PSO is conducted with 20 particles and allowed to experience 2000 iterations. The algorithm stops if it has reached the limit of 200 evaluation numbers or if the minimum error (${10^{-5}}$) conditions is satisfied. The computation time needed for the different algorithms are listed in Table II.

Figs. 4g-h show that the NMI and JMI similarity surfaces are plotted as a function of $x$ and $y$ translation. In this case, the JSM removes all local maxima and achieves the global maximum at the registration position, while the NMI suffers from the biased maximum at the mismatching position.

Figs. 5a-b show the reference and floating endoscopic images ($720\times572$ pixels) including a surgical instrument with different illuminations. Using a mosaic pattern to fuse the two images, Figs. 5c-d show the NMI-based and PMI-based misregistration results. Fig. 5e shows our accurate JMI-based registration result (see Table I. case 2).

\begin{figure}[!t]
\centerline{\includegraphics[width=2.2in,height=1.5in]{fig5}}
\caption{(a)-(b) Reference and floating endoscopic images (size: $720\times572$ pixels) with a
surgical tool and illumination changes. The two images are fused using a mosaic pattern. (c) NMI. (d)
PMI. (e) JMI.}
\label{fig5}
\end{figure}

\begin{center}
\begin{threeparttable}
%\renewcommand{\arraystretch}{1.3}
%\caption{Registration results for Fig. 4 and Fig. 5~\tnote{1}}
\caption{Registration results for Fig. 4 and Fig. 5 (The translations $X$ and $Y$ are in pixels in the $x$ and $y$ directions,
the rotation $\beta$ is in degrees around the center of the images.).}
%\begin{tabular*}{0.50\textwidth}{rlcc}
\begin{tabular*}{0.48\textwidth}{@{\extracolsep{\fill}} rlcc }
\hline
&Cases&Correct($X$,$Y$,$\beta$)&Computed($X$,$Y$,$\beta$)\\
\hline
&1&$-23.11$, $45.59$, $11.43^\circ$&$-22.34$, $45.30$, $11.03^\circ$\\
&2&$37.91$, $-36.78$, $4.43^\circ$&$37.46$, $-38.18$, $4.68^\circ$\\
\hline
\end{tabular*}
%\begin{tablenotes}
%\item[1]The translations $X$ and $Y$ are in pixels in the $x$ and $y$ directions, the rotation $\beta$ is in degrees around the center of the images.
%\end{tablenotes}
\end{threeparttable}
\end{center}

\begin{center}
\begin{threeparttable}
%\renewcommand{\arraystretch}{1.3}
\caption{Computation iterations and runtime in seconds for Fig. 4. (Matlab 6.5, single core Intel Celeron 2.8GHz, RAM 2GB)}
\begin{tabular*}{0.48\textwidth}{@{\extracolsep{\fill}}rlcccccc}
\hline
&&JMI&NMI&RMI&HMI&GMI&PMI\\
\hline
&Iter.&64&41&45&46&50&29\\
&Time&157.4&296.7&297.1&1060.1&329.1&3049.3\\
\hline
\end{tabular*}
\end{threeparttable}
\end{center}

\section{Conclusion}
We propose an effective JSM to solve the problems of outliers and local maxima in MI-based
image registration. Representing the corresponding salient structures in the two images to be registered, JSM is easily integrated into other intensity-based similarity measures for 3D nonrigid registration. Independent of this work but subsequent to our preliminary
conference papers \cite{23}\cite{24} which this letter elaborates on and extends, Ou et al. \cite{25} developed a similar mutual saliency map for
outlier rejection in 3D nonrigid image registration.

Additionally, our method is an intensity-based method and also sensitive to the initial conditions. It is necessary in principle to set the proper initial conditions close to a correct alignment solution, which can be achieved by coarse alignment techniques such as principal axes based method. Nevertheless, all instances of correct registration in this letter are directly performed by our method without any coarse alignment.


% if have a single appendix:
%\appendix[Proof of the Zonklar Equations]
% or
%\appendix  % for no appendix heading
% do not use \section anymore after \appendix, only \section*
% is possibly needed

% use appendices with more than one appendix
% then use \section to start each appendix
% you must declare a \section before using any
% \subsection or using \label (\appendices by itself
% starts a section numbered zero.)
%


%\appendices
%\section{Proof of the First Zonklar Equation}
%Appendix one text goes here.

% you can choose not to have a title for an appendix
% if you want by leaving the argument blank
%\section{}
%Appendix two text goes here.


% use section* for acknowledgement
\section*{Acknowledgment}


The authors thank Simon K. Warfield, Michal Irani, Robert Barnett and Edward Vrscay for allowing the use of image, Rehan Ali for the phase recovery source code, Shanbao Tong and all reviewers for their useful comments, and Wendy Wang for her help to our algorithm.


% Can use something like this to put references on a page
% by themselves when using endfloat and the captionsoff option.
\ifCLASSOPTIONcaptionsoff
  \newpage
\fi



% trigger a \newpage just before the given reference
% number - used to balance the columns on the last page
% adjust value as needed - may need to be readjusted if
% the document is modified later
%\IEEEtriggeratref{8}
% The "triggered" command can be changed if desired:
%\IEEEtriggercmd{\enlargethispage{-5in}}

% references section

% can use a bibliography generated by BibTeX as a .bbl file
% BibTeX documentation can be easily obtained at:
% http://www.ctan.org/tex-archive/biblio/bibtex/contrib/doc/
% The IEEEtran BibTeX style support page is at:
% http://www.michaelshell.org/tex/ieeetran/bibtex/
%\bibliographystyle{IEEEtran}
% argument is your BibTeX string definitions and bibliography database(s)
%\bibliography{IEEEabrv,../bib/paper}
%
% <OR> manually copy in the resultant .bbl file
% set second argument of \begin to the number of references
% (used to reserve space for the reference number labels box)
\begin{thebibliography}{99}

\bibitem{1} F. Maes, A. Collignon,D. Vandermeulen, G. Marchal,and P. Suetens, Multimodality image registration by maximization of mutual information, \textit{IEEE Trans. Med. Imag.,} 16(2):187-98,1997.
\bibitem{2} P. Viola and W. M. Wells III, Alignment by maximization of mutual information, in \textit{IJCV}, 24(2):137-54, 1997.
\bibitem{3} J. Orchard, Multimodal image registration using floating regressors in the joint intensity scatter plot, \textit{Med Image Anal.,} 12(4):385-96,2008.
\bibitem{4} M. Auer, P. Regitnig and G. A. Holzapfel, An automatic nonrigid registration for stained histological sections, \textit{IEEE Trans. Im. Proc.,} 14(4): 475-86, 2005.
\bibitem{5} P. Likar and F. Pernus, A hierarchical approach to elastic registration based on mutual information, \textit{Image Vis. Comput.,} 19(1-2):33-44, 2001.
\bibitem{6} D. Tomazevic, B. Likar, F. Pernus, 3-D/2-D registration by integrating 2-D information in 3-D, \textit{IEEE Trans. Med. Imag.,} 25(1):17-27,2006.
\bibitem{7} M. Staring, U. A. van der Heide, S. Klein, M. A. Viergever, and J. P. W. Pluim, Registration of Cervical MRI Using Multifeature Mutual Information, \textit{IEEE Trans. Med. Imag.,} 28(9):1412-21,2009.
\bibitem{8} J. Tsao, Interpolation Artifacts in Multimodality Image Registration Based on Maximization of Mutual Information, \textit{IEEE Trans. Med. Imag.,} 22(7):854-64,2003.
\bibitem{9} R. Gan, A. C. S. Chung, S. Liao, Maximum distance-gradient for robust image registration, \textit{Med Image Anal.,} 12(4):452-68, 2008.
\bibitem{10} J. P. W. Pluim, J. B. Maintz, M. A. Viergever, Image registration by maximization of combined mutual information and gradient information, \textit{IEEE Trans. Med. Imag.,} 19(8):809-14, 2000.
\bibitem{11} R. Bhagalia, J. A. Fessler and B. Kim, Accelerated Nonrigid Intensity-Based Image Registration Using Importance Sampling, \textit{IEEE Trans. Med. Imag.,} 28(8):1208-16, 2009.
\bibitem{12} M. Mellor, M. Brady, Phase mutual information as a similarity measure for registration. \textit{Med Image Anal.,} 9(4):330-43, 2005.
\bibitem{13} T. Kadir and M. Brady, Saliency, Scale and Image Description, \textit{IJCV}, 45(2): 83-105, 2001.
\bibitem{14} D. G. Lowe, Distinctive image feature from scale-invariant keypoints, \textit{IJCV}, 60(2): 91-110, 2004.
\bibitem{15} H. Luan, F. Qi, Z. Xue, L. Chen, D. Shen, Multimodality image registration by maximization of quantitative-qualitative measure of mutual information,\textit{Patt. Recogn.,} 41(1):285-98, 2008.
\bibitem{16} L. Itti and C. Koch, Computational modeling of visual attention, \textit{Nature Reviews Neuroscience}, 2(3):194-203, 2001.
\bibitem{17} F. Liu and M. Gleicher, Region Enhanced Scale-Invariant Saliency Detection, \textit{IEEE ICME 2006}, pp.1477-80, 2006.
\bibitem{18} M. Jenkinson and S. Smith, A global optimization method for robust affine registration of brain images, \textit{Med Image Anal.,} 5(2):143-56, 2001.
\bibitem{19} D. C. Alexander, C. Pierpaoli, P. J. Basser, J. C. Gee, Spatial transformation of diffusion tensor magnetic resonance images, \textit{IEEE Trans. Med. Imag.,} 20(11):1131-9,2001.
\bibitem{20} C. Studholme, DLG Hill, DJ Hawkes, An overlap invariant entropy measure of 3D medical image alignment, \textit{Patt. Recogn.,} 32(1):71-86, 1999.
\bibitem{21} D. Russakoff, C. Tomasi, T. Rohlfing, C. Maurer, Image similarity using mutual information of regions, In: \textit{ECCV 2004}, pp.596-607, 2004.
\bibitem{22} J. Kybic, High-dimensional mutual information estimation for image registration, \textit{IEEE ICIP 2004}, pp. 1779-82, 2004.
\bibitem{23} Z. Gu and B. Qin, Multi-modal and multi-temporal image registration in the presence of gross outliers using feature voxel-weighted normalized mutual information,\textit{ IEEE NSS-MIC 2006}, pp. 3209-12, 2006.
\bibitem{24} B. Qin and Z. Gu, Robust adaptive non-rigid image registration based on joint salient point sets in the presence of tumor-like gross outlier, \textit{Proc. of SPIE,} vol. 6833 pp. 683320-1-11, 2007.
\bibitem{25} Y. Ou and C. Davatzikos, DRAMMS: Deformable Registration via Attribute Matching and Mutual-Saliency Weighting, \textit{IPMI 2009}, LNCS 5636, pp. 50-62, 2009.



\end{thebibliography}

% biography section
%
% If you have an EPS/PDF photo (graphicx package needed) extra braces are
% needed around the contents of the optional argument to biography to prevent
% the LaTeX parser from getting confused when it sees the complicated
% \includegraphics command within an optional argument. (You could create
% your own custom macro containing the \includegraphics command to make things
% simpler here.)
%\begin{biography}[{\includegraphics[width=1in,height=1.25in,clip,keepaspectratio]{mshell}}]{Michael Shell}
% or if you just want to reserve a space for a photo:

%\begin{IEEEbiography}{Michael Shell}
%Biography text here.
%\end{IEEEbiography}

% if you will not have a photo at all:
%\begin{IEEEbiographynophoto}{John Doe}
%Biography text here.
%\end{IEEEbiographynophoto}

% insert where needed to balance the two columns on the last page with
% biographies
%\newpage

%\begin{IEEEbiographynophoto}{Jane Doe}
%Biography text here.
%\end{IEEEbiographynophoto}

% You can push biographies down or up by placing
% a \vfill before or after them. The appropriate
% use of \vfill depends on what kind of text is
% on the last page and whether or not the columns
% are being equalized.

%\vfill

% Can be used to pull up biographies so that the bottom of the last one
% is flush with the other column.
%\enlargethispage{-5in}



% that's all folks
\end{document}



1304.8052.extracted_bib
\begin{thebibliography}{99}

\bibitem{1} F. Maes, A. Collignon,D. Vandermeulen, G. Marchal,and P. Suetens, Multimodality image registration by maximization of mutual information, \textit{IEEE Trans. Med. Imag.,} 16(2):187-98,1997.
\bibitem{2} P. Viola and W. M. Wells III, Alignment by maximization of mutual information, in \textit{IJCV}, 24(2):137-54, 1997.
\bibitem{3} J. Orchard, Multimodal image registration using floating regressors in the joint intensity scatter plot, \textit{Med Image Anal.,} 12(4):385-96,2008.
\bibitem{4} M. Auer, P. Regitnig and G. A. Holzapfel, An automatic nonrigid registration for stained histological sections, \textit{IEEE Trans. Im. Proc.,} 14(4): 475-86, 2005.
\bibitem{5} P. Likar and F. Pernus, A hierarchical approach to elastic registration based on mutual information, \textit{Image Vis. Comput.,} 19(1-2):33-44, 2001.
\bibitem{6} D. Tomazevic, B. Likar, F. Pernus, 3-D/2-D registration by integrating 2-D information in 3-D, \textit{IEEE Trans. Med. Imag.,} 25(1):17-27,2006.
\bibitem{7} M. Staring, U. A. van der Heide, S. Klein, M. A. Viergever, and J. P. W. Pluim, Registration of Cervical MRI Using Multifeature Mutual Information, \textit{IEEE Trans. Med. Imag.,} 28(9):1412-21,2009.
\bibitem{8} J. Tsao, Interpolation Artifacts in Multimodality Image Registration Based on Maximization of Mutual Information, \textit{IEEE Trans. Med. Imag.,} 22(7):854-64,2003.
\bibitem{9} R. Gan, A. C. S. Chung, S. Liao, Maximum distance-gradient for robust image registration, \textit{Med Image Anal.,} 12(4):452-68, 2008.
\bibitem{10} J. P. W. Pluim, J. B. Maintz, M. A. Viergever, Image registration by maximization of combined mutual information and gradient information, \textit{IEEE Trans. Med. Imag.,} 19(8):809-14, 2000.
\bibitem{11} R. Bhagalia, J. A. Fessler and B. Kim, Accelerated Nonrigid Intensity-Based Image Registration Using Importance Sampling, \textit{IEEE Trans. Med. Imag.,} 28(8):1208-16, 2009.
\bibitem{12} M. Mellor, M. Brady, Phase mutual information as a similarity measure for registration. \textit{Med Image Anal.,} 9(4):330-43, 2005.
\bibitem{13} T. Kadir and M. Brady, Saliency, Scale and Image Description, \textit{IJCV}, 45(2): 83-105, 2001.
\bibitem{14} D. G. Lowe, Distinctive image feature from scale-invariant keypoints, \textit{IJCV}, 60(2): 91-110, 2004.
\bibitem{15} H. Luan, F. Qi, Z. Xue, L. Chen, D. Shen, Multimodality image registration by maximization of quantitative-qualitative measure of mutual information,\textit{Patt. Recogn.,} 41(1):285-98, 2008.
\bibitem{16} L. Itti and C. Koch, Computational modeling of visual attention, \textit{Nature Reviews Neuroscience}, 2(3):194-203, 2001.
\bibitem{17} F. Liu and M. Gleicher, Region Enhanced Scale-Invariant Saliency Detection, \textit{IEEE ICME 2006}, pp.1477-80, 2006.
\bibitem{18} M. Jenkinson and S. Smith, A global optimization method for robust affine registration of brain images, \textit{Med Image Anal.,} 5(2):143-56, 2001.
\bibitem{19} D. C. Alexander, C. Pierpaoli, P. J. Basser, J. C. Gee, Spatial transformation of diffusion tensor magnetic resonance images, \textit{IEEE Trans. Med. Imag.,} 20(11):1131-9,2001.
\bibitem{20} C. Studholme, DLG Hill, DJ Hawkes, An overlap invariant entropy measure of 3D medical image alignment, \textit{Patt. Recogn.,} 32(1):71-86, 1999.
\bibitem{21} D. Russakoff, C. Tomasi, T. Rohlfing, C. Maurer, Image similarity using mutual information of regions, In: \textit{ECCV 2004}, pp.596-607, 2004.
\bibitem{22} J. Kybic, High-dimensional mutual information estimation for image registration, \textit{IEEE ICIP 2004}, pp. 1779-82, 2004.
\bibitem{23} Z. Gu and B. Qin, Multi-modal and multi-temporal image registration in the presence of gross outliers using feature voxel-weighted normalized mutual information,\textit{ IEEE NSS-MIC 2006}, pp. 3209-12, 2006.
\bibitem{24} B. Qin and Z. Gu, Robust adaptive non-rigid image registration based on joint salient point sets in the presence of tumor-like gross outlier, \textit{Proc. of SPIE,} vol. 6833 pp. 683320-1-11, 2007.
\bibitem{25} Y. Ou and C. Davatzikos, DRAMMS: Deformable Registration via Attribute Matching and Mutual-Saliency Weighting, \textit{IPMI 2009}, LNCS 5636, pp. 50-62, 2009.



\end{thebibliography}