  This paper proposes a novel polarization sensor structure and network
architecture to obtain a high-quality RGB image and polarization information.
Conventional polarization sensors can simultaneously acquire RGB images and
polarization information, but the polarizers on the sensor degrade the quality
of the RGB images. There is a trade-off between the quality of the RGB image
and polarization information as fewer polarization pixels reduce the
degradation of the RGB image but decrease the resolution of polarization
information. Therefore, we propose an approach that resolves the trade-off by
sparsely arranging polarization pixels on the sensor and compensating for
low-resolution polarization information with higher resolution using the RGB
image as a guide. Our proposed network architecture consists of an RGB image
refinement network and a polarization information compensation network. We
confirmed the superiority of our proposed network in compensating the
differential component of polarization intensity by comparing its performance
with state-of-the-art methods for similar tasks: depth completion. Furthermore,
we confirmed that our approach could simultaneously acquire higher quality RGB
images and polarization information than conventional polarization sensors,
resolving the trade-off between the quality of RGB images and polarization
information. The baseline code and newly generated real and synthetic
large-scale polarization image datasets are available for further research and
development.
