Summary: A recent paper by Sadler et al. looked at what aspects of teachers’ knowledge were important in increasing students’ knowledge. They found that teachers’ subject knowledge was important (and argue that broad, not deep, knowledge is important) but the effect differed depending on whether the specific subject had a common associated misconception. In this case, subject knowledge wasn’t enough — teachers needed to know students’ likely misconceptions in order to have an effect on student knowledge.
Unfortunately, the paper is not available without a journal subscription, so I’m going to give a few more details here. The whole paper is centred on physics — whether it might apply to computing is an interesting question.
The hypothesis was two-fold:
- Teachers’ knowledge of a physics concept affects gains in students’ knowledge of that concept.
- Teachers’ knowledge of common misconceptions of a concept also has an effect on gains in students’ knowledge.
The method was simple and cute. A series of multiple choice questions were developed to test certain concepts in physics (e.g. when a candle burns, where does the wax go?). The test was administered to the students before and after the year where the topic was covered — if they went from wrong to right, they were adjudged to have learnt the concept. But the test was also administered to the teachers, who were asked both to pick the right answer (assessing their own knowledge), and to pick what they thought was the incorrect answer most likely to be picked by students (assessing their awareness of likely student misconceptions).
As a bit of a complication, they also included a couple of maths and reading questions in the student test, and split their sample based on the performance in those questions. The idea was that the low maths/reading ability group includes those that didn’t bother on the test, and thus the split stops them muddying the results. (Another possible interpretation is that these students are sufficiently lacking in core skills that they’re unlikely to be able to make gains in science.)
Their sample was just under 10,000 students, and around 180 teachers. They put the results into a statistical model (hierarchical logistic regression) that I must admit I don’t know a whole lot about, so I’m not going to offer any comment/criticism on this method.
The end result was a model of how the various factors (e.g. teachers’ performance on a question) affected the difference in students’ performance between the pre-test and post-test.
Let’s start with the group who scored well on the maths/reading questions. In the case where a question had no common misconception, the model predicted that even if these students were faced with a teacher without the appropriate subject knowledge, they would still manage to learn. However, if the teacher did have the subject knowledge, the learning effect was doubled. That makes a clear argument for having teachers who have subject knowledge of revelant topics.
However, this teacher effect disappeared if the question had a common misconception. In this case, the teacher having the subject knowledge alone did not increase the student’s learning beyond the no-knowledge baseline learning. A learning effect was only seen if teachers had also demonstrated knowledge of the likely misconception in this area. The message here being that if the students have a common misconception (e.g. that the candle wax becomes liquid), the teachers need to know this in order to correct it and improve students’ learning.
So in the strong maths/reading group, the predictions were borne out. Meanwhile, over in the weak maths/reading group, it’s a different story. Given no misconceptions and faced with a teacher without the relevant subject knowledge, they learnt nothing. If the teacher had the knowledge, the students did learn, but only a low amount — less than the strong group learnt without a knowledgeable teacher. In the case where there was a misconception, students learnt very little (or nothing) regardless of the teacher’s subject knowledge or knowledge of misconceptions. There are a variety of reasons that this group (which was only 23% of the whole) may have learnt less, and so interpreting the result too closely is difficult. See the paper for full discussion.
So in the strong group, the predictions were borne out: teacher knowledge mattered, but knowledge of likely student misconceptions was also needed. This effect persisted even after the authors introduced things like years of teaching experience as a factor. The authors also argue that subject knowledge is not really about deep knowledge of any given area, but instead an accurate knowledge across all areas/concepts being taught (even if shallow).
Relevance to Computing
Physics is a subject where students can clearly come to it with preconceptions about its concepts. Long before we teach students gravity, they will have dropped or thrown things. Other work also suggests that physics is about correcting wrong preconceptions. In computing, it’s less obvious that they will have preconceptions — but that doesn’t mean they won’t quickly form misconceptions. Part of Beth Simon et al’s SIGCSE work this year on peer instruction involved forming examples that would tease out problems in student knowledge via multiple choice questions so that they could be discussed among the students. It might be that using those sorts of questions in a study like this might show similar results for computing.
Edit: just after I posted this, Peter Newbury — who works with Beth Simon — posted his own summary of the paper over here which has a bit more detail than mine on the statistical results of the paper.
Reference: Sadler, Connert, Coyle, Cook-Smith and Miller, “The Influence of Teachers’ Knowledge on Student Learning in Middle School Physical Science Classrooms”, American Education Research Journal, early access, 2013.