<p>As per the comments to the question, human research observing this distinction does exist. CHCH possibly alludes to an article by Gläscher, Daw, Dayan and O'Doherty (2010) which concisely defines the difference between model-free learning and model-based learning:</p>

<blockquote>
  <p>Reinforcement learning (RL) uses sequential experience with situations (“states”) and outcomes to assess actions. Whereas model-free RL uses this experience directly, in the form of a reward prediction error (RPE), model-based RL uses it indirectly, building a model of the state transition and outcome structure of the environment, and evaluating actions by searching this model.</p>
</blockquote>

<p>Gläscher et al. (2010) report fMRI evidence for neural activity consistent with model-based learning in the human intraparietal sulcus and lateral prefrontal cortex, and for model-free learning in the ventral striatum. They conclude:</p>

<blockquote>
  <p>This finding supports the existence of two unique forms of learning signal in humans, which may form the basis of distinct computational strategies for guiding behavior.</p>
</blockquote>

<h1>References</h1>

<ul>
<li>Gläscher, J., Daw, N., Dayan, P., &amp; O'Doherty, J. P. (2010). States versus rewards: dissociable neural prediction error signals underlying model-based and model-free reinforcement learning. Neuron, 66(4), 585-595.</li>
</ul>

CC BY-SA 3.0
2015-04-22T21:56:07.567Z
2015-04-22T21:56:07.567Z
Christian Hummeluhr
<p>In reinforcement learning, there is a stark distinction between model-based and model-free learning algorithms, where model-free methods don't make use any explicit information about the dynamics of the environment.</p>

<p>It seems like this distinction would have some analog in human learning, but I'm having a very hard time finding any mention of it. Perhaps it would conditioning versus more cognitive forms of learning? I'd be overjoyed if someone could find an article using the term 'model-free' to refer to some aspect of human learning, or just reassure me of the term's correct human analog.</p>

<p><strong>Does research on human learning featuring a model-free/model-based distinction exist?</strong></p>

CC BY-SA 3.0
2012-10-01T20:50:25.527Z
Are you using google? Google scholar search for "model free reinforcement learning" brings up, as the first hit - before anything else - a cognitive neuroscience study with over 103 citations.  There is indeed a developing literature on this topic, and the appropriate term is indeed "model free."
CHCH
CC BY-SA 3.0
2012-10-02T00:17:21.073Z
@CHCH is referring to [this article](http://www.sciencedirect.com/science/article/pii/S0896627310002874) which for me is also first result. As this is not your first question, I am disappointed by the lack of initial research. It is also not clear to me what you are trying to ask here. Although you make some fun points, I am not sure if this is a question and am voting to close as NARQ.
Artem Kaznatcheev
CC BY-SA 3.0
2012-10-02T04:48:48.360Z
Sorry about the google miss -- this was question I rediscovered from a few years ago. I should've regoogled before posting, but I didn't realize that something would have changed in a couple years. Sorry for the mishap.

However, I don't understand how this isn't a question. What part could use rewording?
zergylord
CC BY-SA 3.0
2012-10-02T08:25:36.367Z
It is a question, and at its root is a very interesting one, but at this point it is very broad.  I don't want to speak for anyone, but I think the others are trying to say that now that you know the terminology, we have "answered" this particular question, so if you use that information to make the question more specific to what you want to know, it will be stronger.  FWIW, I'm glad to see you back again as I think you do ask great/interesting questions, this one just needs a bit of tuning and specificity to it.
Chuck Sherrington
CC BY-SA 3.0
2012-10-02T16:39:06.983Z
Thanks Chuck! I agree that the question in the current form is answered via CHCH's article.

Possible next steps:
1.Delete question and wait to post the inevitable follow up question.
2. I could Community Wiki it and then answer with a blurb from the article (unless CHCH wants to for the credit)
3. I could (as soon as properly formulated) modify the question to something more in depth since the terminology question was so easily answered.

I'd prefer option 2, but I thought I'd ask as maybe the community has some procedure that people have agreed upon.
zergylord
CC BY-SA 3.0
2012-10-03T07:42:25.900Z
Ideally, answer it yourself and go a bit more in depth in your answer while reading through the google hits you now found. It's a genuine question, I prefer not closing it. Just make sure you google a bit better next time. ;p
Steven Jeuris
CC BY-SA 3.0
2014-09-29T21:28:33.937Z
I would particularly interested in an answer for the laymen among us!
Seanny123
<terminology><learning><cognitive-modeling><artificial-intelligence><reinforcement-learning>
'Model-free' learning in humans
zergylord
# 'Model-free' learning in humans

(Asked by: username_0 on 2012-10-01)

In reinforcement learning, there is a stark distinction between model-based and model-free learning algorithms, where model-free methods don't make use any explicit information about the dynamics of the environment.

It seems like this distinction would have some analog in human learning, but I'm having a very hard time finding any mention of it. Perhaps it would conditioning versus more cognitive forms of learning? I'd be overjoyed if someone could find an article using the term 'model-free' to refer to some aspect of human learning, or just reassure me of the term's correct human analog.

**Does research on human learning featuring a model-free/model-based distinction exist?**

**Tags:** terminology, learning, cognitive-modeling, artificial-intelligence, reinforcement-learning

> **Comments:**
> **username_1**: Are you using google? Google scholar search for "model free reinforcement learning" brings up, as the first hit - before anything else - a cognitive neuroscience study with over 103 citations.  There is indeed a developing literature on this topic, and the appropriate term is indeed "model free."  
> **username_2**: @username_1 is referring to [this article](http://www.sciencedirect.com/science/article/pii/S0896627310002874) which for me is also first result. As this is not your first question, I am disappointed by the lack of initial research. It is also not clear to me what you are trying to ask here. Although you make some fun points, I am not sure if this is a question and am voting to close as NARQ.  
> **username_0**: Sorry about the google miss -- this was question I rediscovered from a few years ago. I should've regoogled before posting, but I didn't realize that something would have changed in a couple years. Sorry for the mishap.  However, I don't understand how this isn't a question. What part could use rewording?  
> **username_3**: It is a question, and at its root is a very interesting one, but at this point it is very broad.  I don't want to speak for anyone, but I think the others are trying to say that now that you know the terminology, we have "answered" this particular question, so if you use that information to make the question more specific to what you want to know, it will be stronger.  FWIW, I'm glad to see you back again as I think you do ask great/interesting questions, this one just needs a bit of tuning and specificity to it.  
> **username_0**: Thanks Chuck! I agree that the question in the current form is answered via username_1's article.  Possible next steps: 1.Delete question and wait to post the inevitable follow up question. 2. I could Community Wiki it and then answer with a blurb from the article (unless username_1 wants to for the credit) 3. I could (as soon as properly formulated) modify the question to something more in depth since the terminology question was so easily answered.  I'd prefer option 2, but I thought I'd ask as maybe the community has some procedure that people have agreed upon.  
> **username_4**: Ideally, answer it yourself and go a bit more in depth in your answer while reading through the google hits you now found. It's a genuine question, I prefer not closing it. Just make sure you google a bit better next time. ;p  
> **username_5**: I would particularly interested in an answer for the laymen among us!  

---

## 1 answers

### Answer by username_6 on 2015-04-22. Score: 6 | Accepted answer

As per the comments to the question, human research observing this distinction does exist. username_1 possibly alludes to an article by Gläscher, Daw, Dayan and O'Doherty (2010) which concisely defines the difference between model-free learning and model-based learning:

> Reinforcement learning (RL) uses sequential experience with situations (“states”) and outcomes to assess actions. Whereas model-free RL uses this experience directly, in the form of a reward prediction error (RPE), model-based RL uses it indirectly, building a model of the state transition and outcome structure of the environment, and evaluating actions by searching this model.

Gläscher et al. (2010) report fMRI evidence for neural activity consistent with model-based learning in the human intraparietal sulcus and lateral prefrontal cortex, and for model-free learning in the ventral striatum. They conclude:

> This finding supports the existence of two unique forms of learning signal in humans, which may form the basis of distinct computational strategies for guiding behavior.

# References

* Gläscher, J., Daw, N., Dayan, P., & O'Doherty, J. P. (2010). States versus rewards: dissociable neural prediction error signals underlying model-based and model-free reinforcement learning. Neuron, 66(4), 585-595.


---

