A study of college freshmen in the United States and in China found that Chinese students know more science facts than their American counterparts -- but both groups are nearly identical when it comes to their ability to do scientific reasoning.
But when you look at the researchers' description of the underlying study, you see that this conclusion isn't supported and leads me to question the researcher's own ability to reason scientifically at least in the domain of education.
The researchers administered three tests to incoming college freshmen from China and America who had just enrolled in a calculus-based introductory physics course.
The first test, the Force Concept Inventory, measures students’ basic knowledge of mechanics and the student's understanding of mechanics and forces. "The Force Concept Inventory is not 'just another physics test.' It assesses a student’s overall grasp of the Newtonian concept of force. Without this concept the rest of mechanics is useless, if not meaningless." (Force Concept Inventory, Hestenes, Wells, and Swackhamer, The Physics Teacher, Vol. 30, March 1992, 141-158).
The second test, the Brief Electricity and Magnetism Assessment, measures students’ understanding of electric forces, circuits, and magnetism.
The third test, the Lawson Classroom Test of Scientific Reasoning, measures generic science reasoning skills. You can see the kinds of questions on the exam of the appendix of this study.
The tests were given to Chinese students and American students. According to the researcher, in China, "every student in every school follows exactly the same curriculum, which includes five years of continuous physics classes from grades 8 through 12" and "schools emphasize a very extensive learning of STEM content knowledge" In the United States, "only one-third of students take a year-long physics course before they graduate from high school. The rest only study physics within general science courses. Curricula vary widely from school to school, and students can choose among elective courses" and "science courses are more flexible, with simpler content but with a high emphasis on scientific methods."
Keep those descriptions in mind because they'll be important for the conclusions drawn by the researchers.
Now let's turn to the results.
On the FCI, "[m]ost Chinese students scored close to 90 percent, while the American scores varied widely from 25-75 percent, with an average of 50." Clearly the Chinese students understand mechanics better than their American counterparts. One the BEMA, "Chinese students averaged close to 70 percent while American students averaged around 25 percent -- a little better than if they had simply picked their multiple-choice answers randomly." I guess all those Physics course helped the Chinese students understand physics, whereas all that emphasis on scientific methods at the expense of content didn't pan out so well for the Americans. These results are hardly surprising. Knowledge is domain specific and transference between domains is generally minimal.
On the Lawson Classroom Test of Scientific Reasoning "[b]oth American and Chinese students averaged a 75 percent score." So, the Chinese students were just as capable as the American students even though their course supposedly didn't emphasize "scientific methods" like the American students did.
The researchers, however, concluded:
Lei Bao, associate professor of physics at Ohio State University and lead author of the study, said that the finding defies conventional wisdom, which holds that teaching science facts will improve students’ reasoning ability.
“Our study shows that, contrary to what many people would expect, even when students are rigorously taught the facts, they don’t necessarily develop the reasoning skills they need to succeed,” Bao said. “Because students need both knowledge and reasoning, we need to explore teaching methods that target both.”
What? This isn't the conventional wisdom. The conventional wisdom is that that learning facts in a domain will improve the ability to reason in that domain. This wasn't tested in the study. What was tested in the study, via the FCI and the BEMA, was the students' understanding in the domain (physics) which was significantly higher for the Chinese students compared to the American students. Not surprisingly, the American students didn't understand much physics since they didn't learn many physics facts and their "scientific methods" instruction failed to fill the void. Constructivists take heed.
What the study also showed is that learning facts in one domain will not necessarily lead to transference to a different domain and an improvement in reasoning skills in general, whatever they may be (assuming they exist). Again, not a surprising outcome. But, the researchers' spin obscures this conclusion.
And here's the kicker.
Bao explained that STEM students need to excel at scientific reasoning in order to handle open-ended real-world tasks in their future careers in science and engineering..
Ohio State graduate student and study co-author Jing Han echoed that sentiment. “To do my own research, I need to be able to plan what I’m going to investigate and how to do it. I can’t just ask my professor or look up the answer in a book,” she said
The irony is that this physicist didn't do a very good job conducting an investigation in a foreign domain (education). If he wanted to know who was more capable of "handl[ing] open-ended real-world tasks" he should have tested this in a domain specific way. He should have given both groups open-ended real world physics problems and determined which group handled them better. I'm thinking it would have been the Chinese students.
And then we have the most unsupported conclusion of the study:
“The general public also needs good reasoning skills in order to correctly interpret scientific findings and think rationally,” he said.
How to boost scientific reasoning? Bao points to inquiry-based learning, where students work in groups, question teachers and design their own investigations. This teaching technique is growing in popularity worldwide.
The American students who presumably were instructed in inquiry-based techniques fared no better than the Chinese students in general reasoning ability. Inquiry-based teaching once again failed to show results. And it certainly did the students no favors when it came to the students understanding of physics in which they performed poorly.
I see nothing in this study that shows any benefits for inquiry learning. If anything, the study supports the notion that you can't teach general reasoning directly, both methods of teaching failed. What the study also clearly shows is the continuing importance of learning content if you want to understand something.