Join the informed, opinionated journalists of The Times' editorial staff in lively discussions at our blog Ed Cetera.
April 30, 2008 5:25 PM
Posted by Bruce Ramsey
A month and a half ago I wrote a column on the federal grants system and how it stifles true innovation in science. You can read the column here. I received several very thoughtful responses on that, not all of them immediately. One of the more interesting came in today, signed by Jefferson Foote. Here it is, slightly condensed:
I am a scientist. I studied at Harvard, Berkeley, and Cambridge. I trained under two Nobel Prize winners. One research team I was on made an advance that has saved 100,000 lives (honest disclosure: wasn't my idea). I was a faculty member at the Hutch [the Fred Hutchinson Cancer Research Center] for a dozen years. Sounds good, but Part II is that I'm a poster child for scientific peer review. At the Hutch I filed 16 funding applications to the National Institutes of Health. The agency rejected 15 outright and funded the 16th on a one-year, non-renewable, minimum budget. Thus the issues you touched on in your column are ones I've thought about, talked about, and asked about for a long time.
In my view, the peer review system you described is so risk-averse that the highest-rated applications propose nothing new, thus present no likelihood of failure. Here's a clip from a friend's letter:
"Two years back, I sharply changed tactics for a cancer grant and had surprising success. The change was going from a proud presentation of accomplishments and great plans, to a very unsure presentation about our previous work and kindergarten-level experiements that could only confirm what had been published by others. Result: 75 points improvement and funded at 3.1 percentile!"
(Note that the NIH has an inverted percentile scale: 3 corresponds to what everyone else would call 97th percentile, a smashingly good rating.)
The scientific community has always been full of quirky personalities, and through institutional codes and customs we protect our wackos. Peter Duesberg has tenure. He is also a member of the National Academy of Sciences and thereby has the right to publish in the Proceedings of the NAS, a prestigious journal, over the objections of the editor. Peter's rights per se have not been violated.
However, these rights are almost immaterial, for a simple reason: one can only do research for which there is money. The key insight of your article was that science is not some exercise in pure intellect, but is no different from pork bellies in its subjection to the law of supply and demand. While some academic scientists might claim that they, unlike industrial colleagues, are unfettered to go wherever their imagination takes them, the insidious practicality is that the funding search tethers most of them to conventional ideas.
The examples of peer bias in your column were of research questions with a controversial political angle. While those examples are appropriate to discuss, I believe that most suppression of new and unorthodox proposals does not occur for a political reason. Resistance to new ideas by a complacent establishment is common to all creative areas. George Lucas's struggles against the studio system to make "a new kind of science fiction film" are widely known. But Lucas is unusual only in that he triumphed; Melville wasn't so lucky, nor was Socrates. Maintenance of the status quo benefits the great multitude of researchers who already have their snout in the trough. As a friend used to put it, "More people live off cancer than die from it."
Who is to blame? That's easy. Pervasive incrementalism is entirely the creation of the scientific community, who established and who maintain the peer review system. The review panels, advisory councils, study section administrators, program directors, and institute heads are all PhDs and MDs. The defects of the system cannot be blamed on uninformed bureaucrats or Bush cronies.
What would work better? I can list positive and negative examples:
--The NIH some years ago introduced in grant reviews a separate numerical score for innovation. In my experience, which includes having sat on grant review panels, the most recent last summer, this produced no change. Reviewers regard proposals holistically and never concede that something they think is a bad idea is innovative.
--In my experience, venture capitalists and technology transfer professionals do no better than scientists in evaluating innovation, and in many ways are even more herd-like. At the Hutch I invented a way of engineering therapeutic antibodies that made them safer and my method simultaneously got around a major patent in that field. The Technology Transfer Office was unenthusiastic, and in any case, found no commercial backers. I patented and commercialized the invention myself, and currently it is a core technology of an Australian biotech (Arana) with a market cap of $250 million. (Return to the Hutch: $0.00.)
--Through incessant emphasis in training, social scientists are highly conscious of appearing "subjective." Many social science journals require authors to submit two versions of a manuscript, a complete one for the editor, and one for reviewers that has the authors' names and affiliations removed. Further regulations prohibit excessive self-citation that might give away identity. Few natural scientists are aware of this widespread practice. When I suggest blind submissions to other scientists, they often break up laughing, not because it's a stupid idea, but at the gross lack of objectivity that would suddenly be revealed if papers and grants began being judged solely on intrinsic intellectual merit.
--One US government agency that does fund unorthodox research is the Defense Advanced Research Projects Agency. In fact, they revel in it, and absolutely shun any project perceived as incremental. Their key operating principles are (a) a blank check for grantees, and (b) 100% turnover of agency staff every four years. The output? You might recall that DARPA invented the Internet.
--A second positive example is the Medical Research Council Laboratory of Molecular Biology, in Cambridge, U.K., where I worked for seven years. A legendary place; Crick and Watson were associated with its predecessor.
The organizational structure at LMB is unusual in having tiny labs, typically one staff scientist with a tech and a postdoc. Money is plentiful by U.K. standards, more-or-less unrestricted, and the result is that you get very senior scientists doing whatever they please, with their own hands. The Howard Hughes Medical Institute is now trying to establish an institute with the same organizational model as LMB, in a suburb of Washington, DC. The head of HHMI, Gerry Rubin, who is the driving force behind the new institute, is an alumnus of LMB.
A economist named Pierre Azoulay, at MIT Sloan School of Management, has a study in progress on factors that favor scientific creativity. In his words: "Our hypothesis is that freedom to experiment, long horizons to evaluate results, some tolerance for early failures, and timely feedback on performance stimulate researchers to be more creative. We would like to quantify these impacts." When I had an exchange with him last fall, the project was just getting going, and I don't expect a publication for quite some time. Still, I think Pierre's on the right track.
Jul 30, 08 - 03:22 PM
Should puppies be banned?
Jul 30, 08 - 01:01 AM
Jul 29, 08 - 05:55 PM
Notes on Finland
Jul 28, 08 - 01:30 AM
Jul 27, 08 - 12:00 AM
Summertime and the fishing is perplexing
Achenblog by Joel Achenbach
Postman On Politics