  Current theoretical and computational models of dopamine-based reinforcement
learning are largely rooted in the classical behaviorist tradition, and
envision the organism as a purely reactive recipient of rewards and
punishments, with resulting behavior that essentially reflects the sum of this
reinforcement history. This framework is missing some fundamental features of
the affective nervous system, most importantly, the central role of goals in
driving and organizing behavior in a teleological manner. Even when
goal-directed behaviors are considered in current frameworks, they are
typically conceived of as arising in reaction to the environment, rather than
being in place from the start. We hypothesize that goal-driven cognition is
primary, and organized into two discrete phases: goal selection and goal
engaged, which each have a substantially different effective value function.
This dichotomy can potentially explain a wide range of phenomena, playing a
central role in many clinical disorders, such as depression, OCD, ADHD, and
PTSD, and providing a sensible account of the detailed biology and function of
the dopamine system and larger limbic system, including critical ventral and
medial prefrontal cortex. Computationally, reasoning backward from active goals
to action selection is more tractable than projecting alternative action
choices forward to compute possible outcomes. An explicit computational model
of these brain areas and their function in this goal-driven framework is
described, as are numerous testable predictions from this framework.
