  Traditional Reinforcement Learning (RL) has focused on problems involving
many states and few actions, such as simple grid worlds. Most real world
problems, however, are of the opposite type, Involving Few relevant states and
many actions. For example, to return home from a conference, humans identify
only few subgoal states such as lobby, taxi, airport etc. Each valid behavior
connecting two such states can be viewed as an action, and there are trillions
of them. Assuming the subgoal identification problem is already solved, the
quality of any RL method---in real-world settings---depends less on how well it
scales with the number of states than on how well it scales with the number of
actions. This is where our new method T-Learning excels, by evaluating the
relatively few possible transits from one state to another in a
policy-independent way, rather than a huge number of state-action pairs, or
states in traditional policy-dependent ways. Illustrative experiments
demonstrate that performance improvements of T-Learning over Q-learning can be
arbitrarily large.
