What science says about the number of fish in the waters off New England shapes the rules that govern the region's struggling fishing fleet. And lately for key species, those estimates have been way off.
Some federal population estimates for bottom-dwelling groundfish, such as cod and flounder, have swung wildly. That leaves fishermen scrambling to deal with sudden drops or gains in the portfolio of stocks they depend on to make a living. In addition, a pattern has emerged showing the same bad predictions about key fish species are repeated.
Some question how population estimates with such obvious flaws can be used to as a basis for steep cuts in the catches, including massive reductions enacted in May.
"I think it's irresponsible to shut down fisheries based on such inaccurate stock assessments," said Steve Cadrin, a former federal stock assessment scientist and a professor at the University of Massachusetts-Dartmouth.
Federal scientists acknowledge errors in assessments of critical New England fish stocks and say they're working hard to fix them. But they add that their overall methods are proven sound.
Chris Legault, an assessment scientist for the National Oceanic and Atmospheric Administration, noted that the same types of data and methods have been used successfully in most fisheries nationwide, including in the Northeast and in the healthy Mid-Atlantic fishery just to the south.
"There is an ability to make these predictions ... and have some confidence in them," Legault said.
Counting fish is notoriously tough. Fishery scientists joke it's just like counting trees in the forest - if the trees were invisible and moving. But stock assessments try to draw as precise a picture as possible.
Complex data about the fish, such as the amount of a given species at a certain age, is estimated through ocean surveys or historical data. It's then run through scientific models to project the species' population. Regional and federal managers, guided by the scientific advice, then decide how much is safe to take out without overfishing the stock,
But if the underlying population estimates are way off, the scientific advice becomes worthless.
Lately, there's no proof the science is capable of giving the accurate advice regulators need, said Gloucester fisherman Vito Giacalone. "All the proof, really, is that it's not capable."
The saga of the Gulf of Maine cod offers the prime example of a faulty assessment. Deemed robust in 2008, by 2011 the stock was said to be struggling so badly that huge cuts were needed to protect it. In May, fishermen took a devastating year-to-year cut in their catch allotment of 78 percent. Many fishermen say a cut that steep simply doesn't allow them to catch enough fish to stay in business.
Meanwhile, the assessments of certain flounder, haddock and cod stocks have been plagued by a persistent flaw that causes future populations to be overestimated while projections of how much fishermen will catch are underestimated. So fishermen can be told to catch a certain amount one year, only to learn later it was too much, and a cut is coming for future seasons.
In the midst of the problems, the Groundfish Plan Development Team, a group of fishery analysts and experts, has fretted over the reliability of the assessments.
"If there is no confidence in the projections, then it seems they should not be used to determine if a particular catch will end overfishing or rebuild as required," the team wrote last August.
Peter Shelley of the Conservation Law Foundation believes the science is being thrown off by flawed data, which he said is likely missing unreported amounts of discarded or illegally caught fish. And he blames fishery managers for caving to political pressure and pushing the fishery to its weakened state by constantly opting for the highest possible catches.
Dealing with struggling stocks is also likely a factor in the bad predictions, Legault said.
"When the stocks are really low, it becomes very difficult to make these predictions because small, random events suddenly become magnified," he said.
The significant uncertainty doesn't shut down the demands of fishery law, which requires regulators to act on the best available information. That's even if, as recent assessments have indicated, it might not be that good.
The Northeast's top federal scientist, Bill Karp, said the law is asking science to know more about the behavior of some fish stocks in the dynamic New England fishery than it can currently deliver.
"It's clearly not good enough to answer some of these questions," he said.
Fishery law also mandates rebuilding troubled fish stocks in 10 years. Tom Nies, executive director of the New England Fishery Management Council, said that assumes regulators know the population of that fish now, what its healthy population looks like and how to get it there. "We often don't know any of those with any certainty," he said. There should be ways for managers to account for that uncertainty, he said, and ensure catch quotas don't spike or plummet based on one assessment.
Giacalone envisions an entirely new way of doing business, with assessment methods that better account for water temperatures, food supply and other environmental factors that make fish move, congregate and thrive.
"I think 15, 20 years (from now) we'll look back at this period as us operating like the world was flat," he said.