The biomedical community has become too risk-averse, according to a recent NY Times piece. I agree, although I don't agree with the dramatic presentation (it's not some dirty scientific secret - it's not hard to find scientists, and the leaders of the funding agencies themselves talking about it). Here are the basic issue:

Yet the fight against cancer is going slower than most had hoped, with only small changes in the death rate in the almost 40 years since it began. One major impediment, scientists agree, is the grant system itself. It has become a sort of jobs program, a way to keep research laboratories going year after year with the understanding that the focus will be on small projects unlikely to take significant steps toward curing cancer.

Before we go on, we need to first note that claims about lack of progress have been overblown. Cancer survival rates, for most cancers, are much better now than they were 30 years ago. But the funding system is conservative. Why? Frankly, it's a perverse result of competition. The biomedical community has undergone a tremendous expansion, and even though funding has increased substantially over the last 30 years, the number of scientists entering the field has grown even more. It works like this: let's say that 15% of the applications for a given grant review section are excellent, but only 7% can be funded with the available money - then it's basically a crapshoot, and any reason why your proposal won't work is enough to tank your grant. Thus, risky projects, that is, ones that offer a potentially high payoff, but their chances of success are harder to gauge (because these projects explore an area where our existing knowledge is shaky), are very likely to be trashed by the reviewers. Projects that make incremental progress are, obviously, much less likely to have more certain prospects of success, because they stick close to what we already know. A project that has the potential for a major breakthrough is going to be much more uncertain, because it's a bigger leap away from certain knowledge. If uncertainties are viewed as weaknesses in a proposal, then risky projects don't get funded. And keep in mind that generally, there are few projects that have breakthrough potential and very clearly have a high probability of success. Making breakthroughs isn't easy. This is the conclusion the Times' piece reaches:

The institute’s reviewers choose such projects because, with too little money to finance most proposals, they are timid about taking chances on ones that might not succeed. The problem, Dr. Young and others say, is that projects that could make a major difference in cancer prevention and treatment are all too often crowded out because they are too uncertain. In fact, it has become lore among cancer researchers that some game-changing discoveries involved projects deemed too unlikely to succeed and were therefore denied federal grants, forcing researchers to struggle mightily to continue.

As once scientist notes in the piece,

“You can’t prove it will work in advance,” he said. “If you could, it wouldn’t be a high-risk idea.”

The Times' story relates a typical example:

For 25 years, Eileen K. Jaffe received federal grants to run her lab. As a senior scientist at the Fox Chase Cancer Center, with a long list of published papers in prestigious journals, she is a respected, established researcher. Then Dr. Jaffe stumbled upon results that went against textbook explanations, suggesting that it might be possible to find an entirely new class of drugs that could disable proteins that fuel cancer cells. Now she wants to find chemicals that might be developed into such drugs. But her grant proposal was rejected out of hand by the institutes of health, not even discussed by a review panel. She had no preliminary data showing that the idea was likely to work, something reviewers always want to see, and the idea was just too unprecedented. Dr. Jaffe epitomizes the scientist who realizes that if she were to single-mindedly pursue her unorthodox idea, her “career may be ruined in the process,” in the words of Dr. Brawley of the American Cancer Society. Dr. Jaffe is just conceiving her project; it is much to soon to know whether it will result in a revolutionary drug. And even if she does find potential new drugs, it is not clear that they will be effective. Most new ideas are difficult to prove, and most potential new drugs fail.

Having preliminary data is huge. You must have accomplished a good deal of the project by the time you submit the proposal; this is generally done by siphoning off money from other projects. This system also puts younger, new investigators at a disadvantage - they're still getting their labs staffed and running, and thus aren't able to generate the amounts of preliminary data their senior competitors generate. There is another problem here: when your faculty job, and the existence of your lab, depends on the success of your grant proposal, you're less likely to propose a risky grant. As a result, there is too much conservative science. This problem is understood in business: the whole point of a limited liability company is to allow entrepreneurs to take reasonable business risks and pursue potentially good ideas, without being personally ruined if the business plan fails to work out. This is where private foundations, which don't have the same legally-mandated beaureaucratic structure as the NIH (the NIH reports to Congress, and Congress' oversight has a big influence on the way things are done). They could fill an important role by funding more risky projects. On the other hand, private foundations simply haven't matched the resources of the NIH in funding basic cancer research done in academic labs: since 1971, the National Cancer Institute has funded $106 billion in research, while the American Cancer Society (which funds my research), has put out $3.4 billion since 1946. Just to keep things in perspective: nobody is arguing that what the NIH has done has been a waste. The incremental progress over the years has added up, and cancer survival rates now are much better than they were 10, 20, and 30 years ago. But we need our scientific community to stay fresh, we need to nurture innovations, and that involves working some sort of cultural change. We need to figure out how to avoid mediocrity, which is difficult in a rapidly growing community. An afterthought for you history buffs out there (and in honor of Lincoln's 200th birthday): People frequently misunderstand that taking risks in science is a good thing. It's not about risking someone life, or world destruction (no one's arguing that we should do riskier biological weapons research). It's about be willing to handle uncertainty and make big moves. If the role of risk isn't clear in science, it should be in war: willingness to take calculated risks is why Grant won the US Civil War, and why McClellan was booted out of his command.