<p>I indeed think that it is not possible to conclude that the scales are used differently, based on variance alone. Normally one would assume the scale is used properly and the effect you see is indeed the result of (not) having a trained ear. </p>

<p>There do exist ways to investigate how one uses a subjective scale. In a <a href="http://www.mentalworkload.com/userfiles/files/2015_kramer_ea_how%20busy%20is%20busy_%20Validation%20of%20the%20dutch%20IWS.pdf" rel="nofollow">paper</a> I presented, we looked at how people interpreted a subjective workload scale. This scale contained verbal descriptors that conveyed a particular amount of workload, and this amount was estimated by the participants. This scale was designed to be linear and cover workload from "no workload" to "work too demanding". Therefore, we would expect a linear line that starts at 0 (no workload) to 150 (complete overload; we used 150 instead of 100). The results indeed confirmed this and the groups perceived the items on the scale similarly. </p>

<p>So far, the method is kind of similar to your paper. The difference is that we gave the participants anchor-points to help them rate the items: The zero-point (no workload) and the 150-point (complete overload). This way the participants had the same reference points, which should enable them to use the entire range, regardless of their experience or something. </p>

<p>You could also use these anchor points to see how consonance ratings differ. Let them listen to a perfect dissonant and a perfect consonant tone and say they are rated zero and ten respectively (or something like that). If the two groupsc rate the sounds similarly now, they have probably interpreted the scale differently before. However, if you still find the same difference as before, it may suggests that untrained musicians can indeed not distinguish the sound very well (and the scale was interpreted properly before). </p>

CC BY-SA 3.0
2016-05-11T13:06:59.353Z
Thanks for your answer, it's helpful. Although I'm not sure why you say that the paper I mention did *not* have the anchor points that your paper did, since the ends of the scale (1=extreme dissonance, 7=extreme consonance) were pointed out to subjects in the former paper too.
z8080
CC BY-SA 3.0
2016-05-11T13:19:44.553Z
@wildetudor In my paper, the descriptors of workload were the actual stimuli, analogous to your sounds. The anchor points served thus as example stimuli (which is the difference between papers). If you would provide participants with an example stimulus (the actual sounds) and say it's maximally dissonant/consonant, they have a better frame of or reference.
Robin Kramer-ten Have
CC BY-SA 3.0
2016-05-08T10:17:56.313Z
2016-05-08T10:17:56.313Z
Robin Kramer-ten Have
<p>No, I don't think so .  Greater variance could reflect variance in perception just as much as variance in the way the rating scale is used. </p>

CC BY-SA 3.0
2016-05-04T21:47:35.203Z
2016-05-04T21:47:35.203Z
Ruth
<p>I was reading a paper that reports average ratings of chords by two different groups of subjects (trained musicians, and non-musicians). </p>

<p><a href="https://i.sstatic.net/Gu7BI.jpg" rel="nofollow noreferrer"><img src="https://i.sstatic.net/Gu7BI.jpg" alt="enter image description here"></a></p>

<p>The paper makes the argument that "These group differences may reflect either the manner in which the two groups of subjects used the rating scale or in their perceptions of the chords."</p>

<p>Obviously, one would hope that the latter is the case rather than the former, although from this graph one can't be sure since error bars, which would be very informative in this case, are not shown. My question is: is it correct to say that the answer to this question can be given if one compares the variances across groups, e.g.with Levine's test? And that, if found to be unequal, one can argue that the group difference reflects an artefact, i.e. a difference in the way the two groups of subjects used the rating scale?</p>

<experimental-psychology><statistics>
Group differences reflecting different use of a rating scale vs actual difference in perception
z8080
# Group differences reflecting different use of a rating scale vs actual difference in perception

(Asked by: username_0 on 2016-05-03)

I was reading a paper that reports average ratings of chords by two different groups of subjects (trained musicians, and non-musicians).

[![enter image description here](https://i.sstatic.net/Gu7BI.jpg)](https://i.sstatic.net/Gu7BI.jpg)

The paper makes the argument that "These group differences may reflect either the manner in which the two groups of subjects used the rating scale or in their perceptions of the chords."

Obviously, one would hope that the latter is the case rather than the former, although from this graph one can't be sure since error bars, which would be very informative in this case, are not shown. My question is: is it correct to say that the answer to this question can be given if one compares the variances across groups, e.g.with Levine's test? And that, if found to be unequal, one can argue that the group difference reflects an artefact, i.e. a difference in the way the two groups of subjects used the rating scale?

**Tags:** experimental-psychology, statistics


---

## 2 answers

### Answer by username_1 on 2016-05-08. Score: 2 | Accepted answer

I indeed think that it is not possible to conclude that the scales are used differently, based on variance alone. Normally one would assume the scale is used properly and the effect you see is indeed the result of (not) having a trained ear.

There do exist ways to investigate how one uses a subjective scale. In a [paper](http://www.mentalworkload.com/userfiles/files/2015_kramer_ea_how%20busy%20is%20busy_%20Validation%20of%20the%20dutch%20IWS.pdf) I presented, we looked at how people interpreted a subjective workload scale. This scale contained verbal descriptors that conveyed a particular amount of workload, and this amount was estimated by the participants. This scale was designed to be linear and cover workload from "no workload" to "work too demanding". Therefore, we would expect a linear line that starts at 0 (no workload) to 150 (complete overload; we used 150 instead of 100). The results indeed confirmed this and the groups perceived the items on the scale similarly.

So far, the method is kind of similar to your paper. The difference is that we gave the participants anchor-points to help them rate the items: The zero-point (no workload) and the 150-point (complete overload). This way the participants had the same reference points, which should enable them to use the entire range, regardless of their experience or something.

You could also use these anchor points to see how consonance ratings differ. Let them listen to a perfect dissonant and a perfect consonant tone and say they are rated zero and ten respectively (or something like that). If the two groupsc rate the sounds similarly now, they have probably interpreted the scale differently before. However, if you still find the same difference as before, it may suggests that untrained musicians can indeed not distinguish the sound very well (and the scale was interpreted properly before).

> **Comments:**
> **username_0**: Thanks for your answer, it's helpful. Although I'm not sure why you say that the paper I mention did *not* have the anchor points that your paper did, since the ends of the scale (1=extreme dissonance, 7=extreme consonance) were pointed out to subjects in the former paper too.  
> **username_1**: @wildetudor In my paper, the descriptors of workload were the actual stimuli, analogous to your sounds. The anchor points served thus as example stimuli (which is the difference between papers). If you would provide participants with an example stimulus (the actual sounds) and say it's maximally dissonant/consonant, they have a better frame of or reference.  

---

### Answer by username_2 on 2016-05-04. Score: 1

No, I don't think so . Greater variance could reflect variance in perception just as much as variance in the way the rating scale is used.


---

