  It is common to have continuous streams of new data that need to be
introduced in the system in real-world applications. The model needs to learn
newly added capabilities (future tasks) while retaining the old knowledge (past
tasks). Incremental learning has recently become increasingly appealing for
this problem. Task-incremental learning is a kind of incremental learning where
task identity of newly included task (a set of classes) remains known during
inference. A common goal of task-incremental methods is to design a network
that can operate on minimal size, maintaining decent performance. To manage the
stability-plasticity dilemma, different methods utilize replay memory of past
tasks, specialized hardware, regularization monitoring etc. However, these
methods are still less memory efficient in terms of architecture growth or
input data costs. In this study, we present a simple yet effective adjustment
network (SAN) for task incremental learning that achieves near state-of-the-art
performance while using minimal architectural size without using memory
instances compared to previous state-of-the-art approaches. We investigate this
approach on both 3D point cloud object (ModelNet40) and 2D image (CIFAR10,
CIFAR100, MiniImageNet, MNIST, PermutedMNIST, notMNIST, SVHN, and FashionMNIST)
recognition tasks and establish a strong baseline result for a fair comparison
with existing methods. On both 2D and 3D domains, we also observe that SAN is
primarily unaffected by different task orders in a task-incremental setting.
