  We study random embeddings produced by untrained neural set functions, and
show that they are powerful representations which well capture the input
features for downstream tasks such as classification, and are often linearly
separable. We obtain surprising results that show that random set functions can
often obtain close to or even better accuracy than fully trained models. We
investigate factors that affect the representative power of such embeddings
quantitatively and qualitatively.
