  Conventional supervised learning typically assumes that the learning task can
be solved by learning a single function since the data is sampled from a fixed
distribution. However, this assumption is invalid in open-ended environments
where no task-level data partitioning is available. In this paper, we present a
novel supervised learning framework of learning from open-ended data, which is
modeled as data implicitly sampled from multiple domains with the data in each
domain obeying a domain-specific target function. Since different domains may
possess distinct target functions, open-ended data inherently requires multiple
functions to capture all its input-output relations, rendering training a
single global model problematic. To address this issue, we devise an Open-ended
Supervised Learning (OSL) framework, of which the key component is a subjective
function that allocates the data among multiple candidate models to resolve the
"conflict" between the data from different domains, exhibiting a natural
hierarchy. We theoretically analyze the learnability and the generalization
error of OSL, and empirically validate its efficacy in both open-ended
regression and classification tasks.
