Pop quiz: Educational data in the United States is
a) Used ineffectively
b) Judged by relative, often arbitrary scales
c) Uninformative to look at
d) Significantly underappreciated
To David Stewart, the answer is all of the above, and then some. As the founder and CEO of Tembo, Stewart saw the untapped potential of educational data and decided to do something about it. Standardized tests are a staple of the American education system, a way to place students in some sort of context and gauge the quality of a particular district, school, or classroom. Measure achievement, the mantra goes, and we can understand what holds students back and take corrective action.
“It seems like everyone has recognized the importance of data,” says Michael Moore, Tembo’s Director of Product Development, “and made the culture shift to collecting everything they can.” This has expanded beyond standardized tests to metrics on reading speed, laptop use, or absenteeism. “The issue is seeing it in context in a useful way.”
It’s a common problem that is becoming increasingly pervasive as information gets cheaper and Big Data becomes the status quo: what to do with the results? Fields as diverse as national security, sports science, and molecular biology are all acquiring more bits than ever, but the promised answers are lagging. More information may be a good thing, but turning it into actionable knowledge is not trivial: given the complexity of treating, packaging, and communicating data to the uninitiated, there’s no reason to assume information will always be used intelligently.
“The biggest issue to me is,” explains Stewart, “is that educators are not technologists or data people, data people aren’t educators, and neither one is a design-focused person.” Decoupling these three capabilities leads to problematic short-circuits in which data isn’t fully interpreted, results aren’t properly communicated, or findings aren’t effectively fed back into schools.
Typically, school districts handle their own data or hire IT companies, like Pearson, Amplify, and Acumen Solutions, to help out. The new wave of challengers includes Tembo and other companies like Schoolzilla and Tableau, all of which are bringing novel approaches to the table.
Tembo is already exposing concerning problems with data use in some of the country’s largest public school systems. There, millions of dollars are being spent on interim assessment systems, intended to track student performance throughout the year and adapt teaching strategies in advance of the high-stakes year-end tests. The problem is that there’s almost zero correlation between Common Core skills scores on the interim test and the end-of-year test. The difficulty levels are different, and the magnitudes of those discrepancies even differ among subject areas. Just because a student does well on a mid-year test doesn’t mean she will do well at the end of the year, making it impossible to track improvement. “You have to anchor the standards to something meaningful,” Stewart suggests, “like the prior year’s performance,” or the level of difficulty.
Tracking specific students over time reveals a deeper layer, and this sort of longitudinal analysis is one of Tembo’s strengths. When Stewart was working for New York City Public Schools, he found that not all gradations of achievement are equally telling. For example, 8th graders who scored in the lower end of the proficient range of one particular test had a 54% chance of graduating high school four years later; that number jumped to 83% for those in the middle of the proficient range. So while everyone supports getting as many students up the educational ladder as possible, there can be real outcome-based differences between seemingly similar categories.
So far, Tembo has focused on the analysis and design side of the equation, attempting to wring all possible information from the numbers and translate those findings into understandable morsels. Of course, the last step – using this information to drive improvement – may be the most important, and for now, that part’s still up to the schools.
“As the data proliferates,” says Steve Cartwright, the company’s Director of Analytics, “we really need to bring the individuals that are doing the teaching along for the ride.” Because, even to the data geeks at Tembo, it’s still ultimately about the classroom, where the rubber hits the road. “There are a lot of smart people all over the country trying to figure out the perfect lesson, the perfect way of instructing, and then replicating it for all students,” explains Stewart. But it’s more personal than that, and education is still struggling with escaping the one-size-fits-all approach. Given the widely differing starting points of each individual – learning style, home environment, motivation level – “you’re never going to solve it with an algorithm,” concedes Stewart. “Which is why the teacher really matters.”
As educators, graphic designers, and data analysts continue to develop helpful ways of using educational information, the big data revolution may begin to deliver on its promise, yielding real, tangible, personalized results. After all, despite the different approaches and occasionally aggressive opinions, education experts all have the same benevolent goal of improving the system.
“The best intentions in the world are leading to ineffective behaviors,” Stewart says, “and we need to find a way to change that.”Go Back to Top. Skip To: Start of Article.