Chemists looking to design and test new medicines are awash in a sea of bad data, according to a report released today by an international panel of experts. The panel, made up of researchers from 46 nonprofit institutions, universities, and biotech and pharmaceutical firms, say they are setting up a TripAdvisor-like crowdsourcing portal to disseminate up-to-date information about chemical probes that they see as the heart of the problem.
The issue with faulty chemical probes has been growing rapidly in recent years. These small, druglike molecules are used primarily to block the activity of specific proteins to determine their roles in biochemistry. Ideally, this helps researchers design drug compounds that perform similar functions but retain attributes needed for successful medicines, like nontoxicity and the ability to travel through the human body. Today, thousands of such probes exist. But most of them interact with nontarget proteins as well or have other unwanted “off-target” effects.
“It’s become a really serious problem,” says Paul Workman, a chemical biologist and chief executive of the Institute of Cancer Research (ICR) in London, who was a member of the expert panel. The upshot, say Workman and others, is that many probes produce spurious results that can lead researchers to wrong conclusions about the proteins and drug molecules they are studying. For example, some probes have been found to cause side effects, such as triggering chemical changes in proteins or causing proteins to clump together in aggregates that damage cells. And that, the panel argues today in a commentary in Nature Chemical Biology, leads to uncounted hours of wasted time, effort, and money on the part of biomedical researchers. Several years ago, for example, poor chemical probe data led researchers to pursue a final stage clinical trial on a cancer drug candidate called iniparib. Such trials typically cost hundreds of millions of dollars. In 2011 iniparib was deemed a failure after it was found to be less effective at hitting its target than initially thought.
Despite such failures, researchers often continue to use faulty probes. Up to now, chemical biologists have relied on the normal self-correcting mechanisms of science—journal articles, reviews, and conference proceedings—to set the record straight. But they haven’t worked in this case. “The self-correction has not been effective enough,” Workman says. The issue, he says, is that few researchers have the time or expertise to keep track of exactly which probes are proving successful. Instead, when researchers look for a new probe, most simply go to Google Scholar and gravitate to the probes that have the most citations. But these are often the older, less reliable probes, Workman says. One such probe, known as LY294002, has had 30,000 citations since 1994 and 1100 since the start of 2014. However, Workman says, “It’s a terrible compound, 10 years out of date, with lots of off-target effects.” More accurate probes have been on the market for years, but researchers still use LY294002 because it appears to have a strong track record.
Chemical biologists say they hope crowdsourcing will solve the problem. They have set up a new wiki site, called the Chemical Probes Portal, with backing from ICR, the Broad Institute, the Structural Genomics Consortium, and the Wellcome Trust. At the site researchers will be able to add annotations to different chemical probes, to ensure their colleagues have the most up-to-date comparative information they need. The effort could be a challenging one, because even studies that use the same probe often use it in different conditions and at different doses, says Kevan Shokat, a chemical biologist at the University of California, San Francisco. However, he adds, as long as researchers use the portal in an iterative way to steadily improve the understanding of different probes, it should help the community. “I do think that would be a good service,” Shokat says.