  Hybrid Event-Based Vision Sensor (HybridEVS) is a novel sensor integrating
traditional frame-based and event-based sensors, offering substantial benefits
for applications requiring low-light, high dynamic range, and low-latency
environments, such as smartphones and wearable devices. Despite its potential,
the lack of Image signal processing (ISP) pipeline specifically designed for
HybridEVS poses a significant challenge. To address this challenge, in this
study, we propose a coarse-to-fine framework named DemosaicFormer which
comprises coarse demosaicing and pixel correction. Coarse demosaicing network
is designed to produce a preliminary high-quality estimate of the RGB image
from the HybridEVS raw data while the pixel correction network enhances the
performance of image restoration and mitigates the impact of defective pixels.
Our key innovation is the design of a Multi-Scale Gating Module (MSGM) applying
the integration of cross-scale features, which allows feature information to
flow between different scales. Additionally, the adoption of progressive
training and data augmentation strategies further improves model's robustness
and effectiveness. Experimental results show superior performance against the
existing methods both qualitatively and visually, and our DemosaicFormer
achieves the best performance in terms of all the evaluation metrics in the
MIPI 2024 challenge on Demosaic for Hybridevs Camera. The code is available at
https://github.com/QUEAHREN/DemosaicFormer.
