  Recent studies show that task distribution plays a vital role in the
meta-learner's performance. Conventional wisdom is that task diversity should
improve the performance of meta-learning. In this work, we find evidence to the
contrary; (i) our experiments draw into question the efficacy of our learned
models: similar manifolds can be learned with a subset of the data (lower task
diversity). This finding questions the advantage of providing more data to the
model, and (ii) adding diversity to the task distribution (higher task
diversity) sometimes hinders the model and does not lead to a significant
improvement in performance as previously believed. To strengthen our findings,
we provide both empirical and theoretical evidence.
