  This paper addresses how to construct features for the problem of image
correspondence, in particular, the paper addresses how to construct features so
as to maintain the right level of invariance versus discriminability. We show
that without additional prior knowledge of the 3D scene, the right tradeoff
cannot be established in a pre-processing step of the images as is typically
done in most feature-based matching methods. However, given knowledge of the
second image to match, the tradeoff between invariance and discriminability of
features in the first image is less ambiguous. This suggests to setup the
problem of feature extraction and matching as a joint estimation problem. We
develop a possible mathematical framework, a possible computational algorithm,
and we give example demonstration on finding correspondence on images related
by a scene that undergoes large 3D deformation of non-planar objects and camera
viewpoint change.
