  Untrained networks inspired by deep image priors have shown promising
capabilities in recovering high-quality images from noisy or partial
measurements without requiring training sets. Their success is widely
attributed to implicit regularization due to the spectral bias of suitable
network architectures. However, the application of such network-based priors
often entails superfluous architectural decisions, risks of overfitting, and
lengthy optimization processes, all of which hinder their practicality. To
address these challenges, we propose efficient architecture-agnostic techniques
to directly modulate the spectral bias of network priors: 1)
bandwidth-constrained input, 2) bandwidth-controllable upsamplers, and 3)
Lipschitz-regularized convolutional layers. We show that, with just a few lines
of code, we can reduce overfitting in underperforming architectures and close
performance gaps with high-performing counterparts, minimizing the need for
extensive architecture tuning. This makes it possible to employ a more compact
model to achieve performance similar or superior to larger models while
reducing runtime. Demonstrated on inpainting-like MRI reconstruction task, our
results signify for the first time that architectural biases, overfitting, and
runtime issues of untrained network priors can be simultaneously addressed
without architectural modifications. Our code is publicly available.
