  We investigate the learning of implicit neural representation (INR) using an
overparameterized multilayer perceptron (MLP) via a novel nonparametric
teaching perspective. The latter offers an efficient example selection
framework for teaching nonparametrically defined (viz. non-closed-form) target
functions, such as image functions defined by 2D grids of pixels. To address
the costly training of INRs, we propose a paradigm called Implicit Neural
Teaching (INT) that treats INR learning as a nonparametric teaching problem,
where the given signal being fitted serves as the target function. The teacher
then selects signal fragments for iterative training of the MLP to achieve fast
convergence. By establishing a connection between MLP evolution through
parameter-based gradient descent and that of function evolution through
functional gradient descent in nonparametric teaching, we show for the first
time that teaching an overparameterized MLP is consistent with teaching a
nonparametric learner. This new discovery readily permits a convenient drop-in
of nonparametric teaching algorithms to broadly enhance INR training
efficiency, demonstrating 30%+ training time savings across various input
modalities.
