  Transfer in Reinforcement Learning aims to improve learning performance on
target tasks using knowledge from experienced source tasks. Successor
Representations (SR) and their extension Successor Features (SF) are prominent
transfer mechanisms in domains where reward functions change between tasks.
They reevaluate the expected return of previously learned policies in a new
target task to transfer their knowledge. The SF framework extended SR by
linearly decomposing rewards into successor features and a reward weight vector
allowing their application in high-dimensional tasks. But this came with the
cost of having a linear relationship between reward functions and successor
features, limiting its application to tasks where such a linear relationship
exists. We propose a novel formulation of SR based on learning the cumulative
discounted probability of successor features, called Successor Feature
Representations (SFR). Crucially, SFR allows to reevaluate the expected return
of policies for general reward functions. We introduce different SFR
variations, prove its convergence, and provide a guarantee on its transfer
performance. Experimental evaluations based on SFR with function approximation
demonstrate its advantage over SF not only for general reward functions, but
also in the case of linearly decomposable reward functions.
