Those who saw IBM’s Watson defeat former winners on Jeopardy! in 2011 might be forgiven for thinking that artificially intelligent computer systems are a lot brighter than they are. While Watson was able to cope with the highly stylized questions posed during the quiz, AI systems are still left wanting when it comes to commonsense. This was one of the factors that led researchers to find that one of the best available AI systems has the average IQ of a four-year-old.
To see just how intelligent AI systems are, a team of artificial and natural knowledge researchers at the University of Illinois as Chicago (UIC) subjected ConceptNet 4 to the verbal portions of the Weschsler Preschool and Primary Scale of Intelligence Test, which is a standard IQ test for young children. ConceptNet 4 is an AI system developed at MIT that relies on a commonsense knowledge base created from facts contributed by thousands of people across the Web.
While the UIC researchers found that ConceptNet 4 is on average about as smart as a four-year-old child, the system performed much better at some portions of the test than others. While it did well on vocabulary and in recognizing similarities, its overall score was brought down dramatically by a bad result in comprehension, or commonsense “why” questions.
“If a child had scores that varied this much, it might be a symptom that something was wrong,” said Robert Sloan, professor and head of computer science at UIC, and lead author on the study. “We’re still very far from programs with commonsense–AI that can answer comprehension questions with the skill of a child of eight.” (via Top notch AI system about as smart as a four-year-old, lacks commonsense)