  We propose a general method for growing neural network with shared parameter
by matching trained network to new input. By leveraging Hoeffding's inequality,
we provide a theoretical base for improving performance by adding subnetwork to
existing network. With the theoretical base of adding new subnetwork, we
implement a matching method to apply trained subnetwork of existing network to
new input. Our method has shown the ability to improve performance with higher
parameter efficiency. It can also be applied to trans-task case and realize
transfer learning by changing the combination of subnetworks without training
on new task.
