  Data-Free Meta-Learning (DFML) aims to efficiently learn new tasks by
leveraging multiple pre-trained models without requiring their original
training data. Existing inversion-based DFML methods construct pseudo tasks
from a learnable dataset, which is inversely generated from the pre-trained
model pool. For the first time, we reveal two major challenges hindering their
practical deployments: Task-Distribution Shift (TDS) and Task-Distribution
Corruption (TDC). TDS leads to a biased meta-learner because of the skewed task
distribution towards newly generated tasks. TDC occurs when untrusted models
characterized by misleading labels or poor quality pollute the task
distribution. To tackle these issues, we introduce a robust DFML framework that
ensures task distributional robustness. We propose to meta-learn from a pseudo
task distribution, diversified through task interpolation within a compact
task-memory buffer. This approach reduces the meta-learner's overreliance on
newly generated tasks by maintaining consistent performance across a broader
range of interpolated memory tasks, thus ensuring its generalization for unseen
tasks. Additionally, our framework seamlessly incorporates an automated model
selection mechanism into the meta-training phase, parameterizing each model's
reliability as a learnable weight. This is optimized with a policy gradient
algorithm inspired by reinforcement learning, effectively addressing the
non-differentiable challenge posed by model selection. Comprehensive
experiments across various datasets demonstrate the framework's effectiveness
in mitigating TDS and TDC, underscoring its potential to improve DFML in
real-world scenarios.
