<p>I was recently told about a <a href="https://www.ncbi.nlm.nih.gov/pmc/articles/PMC9910357/pdf/nihms-1867344.pdf" rel="nofollow noreferrer">Science paper</a> that suggests the old model of association of cause with effect is incorrect. In this model, cue comes, reward occurs, cue comes again, prediction of reward occurs; over time, calibration between prediction &amp; presence of reward is refined.</p>
<p>This new model proposes that when a reward occurs, animals look back and scan for events that were salient enough to act as causes for a future reward. Mesolimbic DA mediates this process and the authors hypothesise that the OFC provides the relevant cue-reward information.</p>
<p>I was fascinated to read this finding. However, I have no relevant expertise to judge the animal models used (other than recognising the paper was published in Science so they're probably very convincing). For those in the field, have the experiments provided robust evidence to support the new model? To speculate, say the prior in the old model among neuroscientists was 0.7, what's the posterior since this paper? I understand no replications have been published but I'd appreciate if someone could point me to a preprint if one exists.</p>
<p>Thanks for your time.</p>

CC BY-SA 4.0
2024-01-19T14:38:58.710Z
I think both of these models are oversimplifications and I think that's appreciated by the people proposing these models, as well; there's not going to be any way to put a quantitative prior or confidence on it, it's more of a https://en.wikipedia.org/wiki/All_models_are_wrong situation as well as "what's the least misleading way to summarize this in a human-interpretable way".
Bryan Krause
CC BY-SA 4.0
2024-01-28T10:32:51.010Z
The paper is very recent so it's unlikely that any replication has emerged yet. However, the authors claim that their model makes some predictions that agree and disagree with other theoretical and experimental findings. Maybe you can take those as an early meta-judgement. For example ref 50 makes a very thorough test of the RPE hypothesis (in my opinion). Other examples as well. The paper you sent mentions dopamine ramps and they say that this models can account for them, but to me it is not very clear where they show this nor how. The experimental evidence seems solid enought though.
JFR
CC BY-SA 4.0
2024-01-28T10:35:17.400Z
Moreover, at the bottom of page 5, the following sentence is a bit imprecise I think: A critical postulate of the TDRL RPE account is that dopamine responses drive value learning of the immediately preceding state. With TD-lambda models this is not necessarily true, so I dont' know against which type of TD learning they are comparing ANCCR against. It may well be that all these mechanisms coexist since, after all, multiple areas from the cortex project back to the dopaminergic nuclei potentially regulating each mechanism. These should not be taken as any more than general thoughts from me.
JFR
<neuroscience><cognitive-neuroscience><experimental-neuroscience>
What is the role of mesolimbic dopamine in associative or reinforcement learning?
Malcolm Forbes
# What is the role of mesolimbic dopamine in associative or reinforcement learning?

(Asked by: username_0 on 2024-01-19)

I was recently told about a [Science paper](https://www.ncbi.nlm.nih.gov/pmc/articles/PMC9910357/pdf/nihms-1867344.pdf) that suggests the old model of association of cause with effect is incorrect. In this model, cue comes, reward occurs, cue comes again, prediction of reward occurs; over time, calibration between prediction & presence of reward is refined.

This new model proposes that when a reward occurs, animals look back and scan for events that were salient enough to act as causes for a future reward. Mesolimbic DA mediates this process and the authors hypothesise that the OFC provides the relevant cue-reward information.

I was fascinated to read this finding. However, I have no relevant expertise to judge the animal models used (other than recognising the paper was published in Science so they're probably very convincing). For those in the field, have the experiments provided robust evidence to support the new model? To speculate, say the prior in the old model among neuroscientists was 0.7, what's the posterior since this paper? I understand no replications have been published but I'd appreciate if someone could point me to a preprint if one exists.

Thanks for your time.

**Tags:** neuroscience, cognitive-neuroscience, experimental-neuroscience

> **Comments:**
> **username_1**: I think both of these models are oversimplifications and I think that's appreciated by the people proposing these models, as well; there's not going to be any way to put a quantitative prior or confidence on it, it's more of a https://en.wikipedia.org/wiki/All_models_are_wrong situation as well as "what's the least misleading way to summarize this in a human-interpretable way".  
> **username_2**: The paper is very recent so it's unlikely that any replication has emerged yet. However, the authors claim that their model makes some predictions that agree and disagree with other theoretical and experimental findings. Maybe you can take those as an early meta-judgement. For example ref 50 makes a very thorough test of the RPE hypothesis (in my opinion). Other examples as well. The paper you sent mentions dopamine ramps and they say that this models can account for them, but to me it is not very clear where they show this nor how. The experimental evidence seems solid enought though.  
> **username_2**: Moreover, at the bottom of page 5, the following sentence is a bit imprecise I think: A critical postulate of the TDRL RPE account is that dopamine responses drive value learning of the immediately preceding state. With TD-lambda models this is not necessarily true, so I dont' know against which type of TD learning they are comparing ANCCR against. It may well be that all these mechanisms coexist since, after all, multiple areas from the cortex project back to the dopaminergic nuclei potentially regulating each mechanism. These should not be taken as any more than general thoughts from me.  

---

## 0 answers

