  We present squareplus, an activation function that resembles softplus, but
which can be computed using only algebraic operations: addition,
multiplication, and square-root. Because squareplus is ~6x faster to evaluate
than softplus on a CPU and does not require access to transcendental functions,
it may have practical value in resource-limited deep learning applications.
