I usually limit my book reviews to Goodreads or Shelfari but this one deserves much more attention. In Happy Accidents: Serendipity in Modern Medical Breakthroughs; When Scientists Find What They're NOT Looking for, Morton Meyers reviews examples of the unpredictability of scientific progress. This could just be a collection of interesting anecdotes - and some of the stories are truly fascinating. My favorite is probably the discovery of platinum compounds for the treatment of cancer. It came about from the accidental electro-dissolution of a platinum electrode during an experiment studying the effect of electricity on cell cultures! But Meyers goes further and uses these examples to make larger observations about the way science operates today in both academia and industry. A quote from the preface foreshadows the tone of the book:

The dominant convention of all scientific writing is to present discoveries as rationally driven and to let the facts speak for themselves. This humble ideal has succeeded in making scientists look as if they never make errors, that they straightforwardly answer every question they investigate. It banishes any hint of blunders and surprises along the way. Consequently, not only the general public but the scientific community itself is unaware of the vast role of serendipity in medical research. Typically, a discoverer may finally admit this only towards the end of his or her career, after the awards have been received.

And starting on page 304:

An applicant for a research grant is expected to have a clearly defined program for a period of three to five years. Implicit is the assumption that nothing unforeseen will be discovered during that time and, even if something were, it would not cause distraction from the approved line of research. Yet the reality is that many medical discoveries were made by researchers working on the basis of a fallacious hypothesis that led them down an unexpected fortuitous path. .... The peer review system forces investigators to work on problems others think are important and to describe the work in a way that convinces the reviewers that results will be obtained. This is precisely what prevents funded work from being highly preliminary, speculative or radical. How can a venture into the unknown offer predictability of results? .... Indeed the basic process of peer review demands conformity of thinking and disdains a maverick's approach. .... What it comes down to is this: Who on a review committee is the peer of a maverick?

The fact that some of us in the Open Science community are discussing this does not mean that we are advocating for the abolition of peer review or the NIH. We are not that naive. We still submit proposals and manuscripts for publication in peer-reviewed journals (although given a choice we probably would pick an Open Access journal over one running on a paid subscription model). The point is what we do in addition to all those traditional processes. We can share our failed experiments. We can share our research plans. We can discuss science freely admitting what we don't know. We can record our talks at closed meetings and make them public. We can initiate and participate in serious scientific conversations going on in the blogosphere without worrying about everyone's title and rank. Basically, we can collaborate in ways that are most conducive to serendipitous discoveries. The free social software, databases and other infrastructure now available make this information exchange easier than ever. The key question for a researcher today: to hoard or not to hoard? To me, it seems likely that data hoarders will find it more and more difficult to claim priority for a contribution when competing against loose associations of open collaborators motivated by insatiable curiosity. Some of the folks from the funding side are getting it. Take a look at SubMeta.