  We study border varieties of sums of powers ($\underline{\mathrm{VSP}}$'s for
short), recently introduced by Buczy\'nska and Buczy\'nski, parameterizing
border rank decompositions of a point (e.g. of a tensor or a homogeneous
polynomial) with respect to a smooth projective toric variety and living in the
Haiman-Sturmfels multigraded Hilbert scheme. Their importance stems from the
role of border tensor rank in theoretical computer science, especially in the
estimation of the exponent of matrix multiplication, a fundamental and still
unknown quantity in the theory of computation. We compare
$\underline{\mathrm{VSP}}$'s to other well-known loci in the Hilbert scheme,
parameterizing scheme-theoretic versions of decompositions. The latter ones are
crucial in that they naturally explain the existing severe barriers to giving
good lower bounds on ranks. We introduce the notion of border identifiability
and provide sufficient criteria for its appearance, which rely on the
multigraded regularity of Maclagan and Smith. We link border identifiability to
wildness of points. Finally, we determine $\underline{\mathrm{VSP}}$'s in
several instances and regimes, in the contexts of tensors and homogeneous
polynomials. These include concise $3$-tensors of minimal border rank and in
particular of border rank three.
