  In this paper, we generalize the problem of single-index model to the context
of continual learning in which a learner is challenged with a sequence of tasks
one by one and the dataset of each task is revealed in an online fashion. We
propose a randomized strategy that is able to learn a common single-index
(meta-parameter) for all tasks and a specific link function for each task. The
common single-index allows to transfer the information gained from the previous
tasks to a new one. We provide a rigorous theoretical analysis of our proposed
strategy by proving some regret bounds under different assumption on the loss
function.
