  Image restoration aims to restore high-quality images from degraded
counterparts and has seen significant advancements through deep learning
techniques. The technique has been widely applied to mobile devices for tasks
such as mobile photography. Given the resource limitations on mobile devices,
such as memory constraints and runtime requirements, the efficiency of models
during deployment becomes paramount. Nevertheless, most previous works have
primarily concentrated on analyzing the efficiency of single modules and
improving them individually. This paper examines the efficiency across
different layers. We propose a roadmap that can be applied to further
accelerate image restoration models prior to deployment while simultaneously
increasing PSNR (Peak Signal-to-Noise Ratio) and SSIM (Structural Similarity
Index). The roadmap first increases the model capacity by adding more
parameters to partial convolutions on FLOPs non-sensitive layers. Then, it
applies partial depthwise convolution coupled with decoupling
upsampling/downsampling layers to accelerate the model speed. Extensive
experiments demonstrate that our approach decreases runtime by up to 13% and
reduces the number of parameters by up to 23%, while increasing PSNR and SSIM
on several image restoration datasets. Source Code of our method is available
at \href{https://github.com/xiangyu8/MOFA}{https://github.com/xiangyu8/MOFA}.
