  Sampling is a key operation in point-cloud task and acts to increase
computational efficiency and tractability by discarding redundant points.
Universal sampling algorithms (e.g., Farthest Point Sampling) work without
modification across different tasks, models, and datasets, but by their very
nature are agnostic about the downstream task/model. As such, they have no
implicit knowledge about which points would be best to keep and which to
reject. Recent work has shown how task-specific point cloud sampling (e.g.,
SampleNet) can be used to outperform traditional sampling approaches by
learning which points are more informative. However, these learnable samplers
face two inherent issues: i) overfitting to a model rather than a task, and
\ii) requiring training of the sampling network from scratch, in addition to
the task network, somewhat countering the original objective of down-sampling
to increase efficiency. In this work, we propose an almost-universal sampler,
in our quest for a sampler that can learn to preserve the most useful points
for a particular task, yet be inexpensive to adapt to different tasks, models,
or datasets. We first demonstrate how training over multiple models for the
same task (e.g., shape reconstruction) significantly outperforms the vanilla
SampleNet in terms of accuracy by not overfitting the sample network to a
particular task network. Second, we show how we can train an almost-universal
meta-sampler across multiple tasks. This meta-sampler can then be rapidly
fine-tuned when applied to different datasets, networks, or even different
tasks, thus amortizing the initial cost of training.
