  The application of reinforcement learning algorithms onto real life problems
always bears the challenge of filtering the environmental state out of raw
sensor readings. While most approaches use heuristics, biology suggests that
there must exist an unsupervised method to construct such filters
automatically. Besides the extraction of environmental states, the filters have
to represent them in a fashion that support modern reinforcement algorithms.
Many popular algorithms use a linear architecture, so one should aim at filters
that have good approximation properties in combination with linear functions.
This thesis wants to propose the unsupervised method slow feature analysis
(SFA) for this task. Presented with a random sequence of sensor readings, SFA
learns a set of filters. With growing model complexity and training examples,
the filters converge against trigonometric polynomial functions. These are
known to possess excellent approximation capabilities and should therfore
support the reinforcement algorithms well. We evaluate this claim on a robot.
The task is to learn a navigational control in a simple environment using the
least square policy iteration (LSPI) algorithm. The only accessible sensor is a
head mounted video camera, but without meaningful filtering, video images are
not suited as LSPI input. We will show that filters learned by SFA, based on a
random walk video of the robot, allow the learned control to navigate
successfully in ca. 80% of the test trials.
