For policy makers and politicians, there is confusion over what peer review is and how it should be used in policy decision making and, really, how improved technology and new science (including any ethics issues) make critical evaluation by the independent peers of those who create advancements more important than ever.

A. Alan Moghissi, president of the Institute for Regulatory Science, and Michael S. Swetnam, of the Potomac Institute for Policy Studies, write in Technology&Innovation that review criteria must be identified, the process must be transparent, and over reliance on specific peer reviewers must be avoided.  Institutional, intellectual and personal conflicts of interest must also be avoided.

Well, we all instinctively know that but it needs to be reaffirmed repeatedly if we expect policy-makers to regard scientists as trusted guides.   Global warming studies are the obvious disconnect between data and policy, but once it was revealed that the data behind some of the claims was not only not peer reviewed, but sometimes not even data, it grew a cottage industry of skepticism.   Rigorous peer review will win back trust from the public and politicians.

Maintaining the integrity and improving peer review in the federal government remains a key issue for Elmer Yglesias of the Science and Technology Policy Institute, Institute for Defense Analyses, who writes "there are indications that the (federal peer review) system may be over stretched and prone to error."

This means that in an era of tightening budgets - no more Pres. Bush doubling NIH funding and spending for NASA as well - the low funding rates coupled with increased proposals due to more Ph.D.s than ever applying for grants means the federal system may not be up to the challenge.

What to do?   According to Yglesias, since NSF surveys indicate that reviewers' workloads have increased and the number of reviewers per proposal has been declining, the best solution to ensure review's integrity may lie in an Internet-based calibration system.

Nowhere may the process of peer review be more important than in determining what should be funded in military medical research, says Dr. John F. Glenn, principal assistant for research and technology for the U.S. Army Medical Research and Materiel Command, who made the point that candidate products may include prototypes for vaccines, drugs, diagnostics and other medical devices, information products, such as for training and behavioral interventions, as well as human performance information and information on a wide range of hazards.

"A distinctive feature of the core research programs is that they are end product-focused," wrote Glenn. "This leads to three separate roles for peer review – evaluation of research project quality, evaluation of research program quality, and independent validation of research products."

Important to Glenn is determining that a peer review process can validate that end products have met their objectives. He noted that the U.S. Food and Drug Administration's review process employs both internal and external scientific review panels and that the FDA's process is accepted as a substitute by the Department of Defense.

"Unfortunately, there is no comparable, consistently employed peer review process for the validation of medication information or clinical practice guidelines," wrote Glenn. "Such processes are needed as they are intrinsic for evidence-based medicine."

Not only does good peer review help validate a projects science, wrote lead author E. Melissa Kaine, MD, Captain, Medical Corps, U.S. Navy, it can also foster innovation.

According to Kaine, the Congressionally Directed Medical Research Program (CDMRP) is comprised of 18 individual programs seeking to eradicate diseases and support the 'warfighter.' The CDMRP emphasizes innovative, high-risk, high-gain research and employs a two-tier process to evaluate scientific merit, innovation and impact, followed by an external review. These processes, she said, support the mission to fund innovative research. While 'innovative' may be difficult to define, research that is transformative, paradigm shifting and that which represents more than an incremental advance in existing knowledge, serves as definitive.

One innovative aspect of the CDMRP review process is the use of layperson reviewers in addition to scientific experts.

"Consumer reviewers are included and involved at every stage of the funding cycle," she noted. "These reviewers are focused on the impact of the proposed project and provide valuable insight into a projects potential to result in positive outcomes."