  We study the statistical problem of estimating a rank-one sparse tensor
corrupted by additive Gaussian noise, a model also known as sparse tensor PCA.
We show that for Bernoulli and Bernoulli-Rademacher distributed signals and
\emph{for all} sparsity levels which are sublinear in the dimension of the
signal, the sparse tensor PCA model exhibits a phase transition called the
\emph{all-or-nothing phenomenon}. This is the property that for some
signal-to-noise ratio (SNR) $\mathrm{SNR_c}$ and any fixed $\epsilon>0$, if the
SNR of the model is below $\left(1-\epsilon\right)\mathrm{SNR_c}$, then it is
impossible to achieve any arbitrarily small constant correlation with the
hidden signal, while if the SNR is above $\left(1+\epsilon
\right)\mathrm{SNR_c}$, then it is possible to achieve almost perfect
correlation with the hidden signal. The all-or-nothing phenomenon was initially
established in the context of sparse linear regression, and over the last year
also in the context of sparse 2-tensor (matrix) PCA, Bernoulli group testing,
and generalized linear models. Our results follow from a more general result
showing that for any Gaussian additive model with a discrete uniform prior, the
all-or-nothing phenomenon follows as a direct outcome of an appropriately
defined "near-orthogonality" property of the support of the prior distribution.
