  Despite the advances of deep learning in specific tasks using images, the
principled assessment of image fidelity and similarity is still a critical
ability to develop. As it has been shown that Mean Squared Error (MSE) is
insufficient for this task, other measures have been developed with one of the
most effective being Structural Similarity Index (SSIM). Such measures can be
used for subspace learning but existing methods in machine learning, such as
Principal Component Analysis (PCA), are based on Euclidean distance or MSE and
thus cannot properly capture the structural features of images. In this paper,
we define an image structure subspace which discriminates different types of
image distortions. We propose Image Structural Component Analysis (ISCA) and
also kernel ISCA by using SSIM, rather than Euclidean distance, in the
formulation of PCA. This paper provides a bridge between image quality
assessment and manifold learning opening a broad new area for future research.
