  Activation functions are one of the key components of a neural network. The
most commonly used activation functions can be classed into the category of
continuously differentiable (e.g. tanh) and linear-unit functions (e.g. ReLU),
both having their own strengths and drawbacks with respect to downstream
performance and representation capacity through learning (e.g. measured by the
number of dead neurons and the effective rank). In reinforcement learning, the
performance of continuously differentiable activations often falls short as
compared to linear-unit functions. From the perspective of the activations in
the last hidden layer, this paper provides insights regarding this
sub-optimality and explores how activation functions influence the occurrence
of dead neurons and the magnitude of the effective rank. Additionally, a novel
neural architecture is proposed that leverages the product of independent
activation values. In the Atari domain, we show faster learning, a reduction in
dead neurons and increased effective rank.
