In my experience, scientists are much too dismissive; most of them have a hard time fully appreciating other people’s work.
This dismissiveness follows a kind of power law: a few of them spend a large amount of time being dismissive (e.g., David Freedman); a large number spend a small amount of time being dismissive. The really common form of dismissiveness goes like this (from a JAMA abstract):
This is the dismissiveness of dichotomization: division of studies into valid and invalid, proper and improper, unbiased and biased, accurate and inaccurate. As if it were that simple. Such dichotomization throws away a lot of information. It leads to such absurdities as a meta-analysis of 2000 studies that decided that only 16 were worth inclusion. As if the rest contained no information of value.In this second article, we enumerate the major issues in judging the validity of these studies, framed as critical appraisal questions. Was the disease phenotype properly defined and accurately recorded by someone blind to the genetic information? Have any potential
differences between disease and nondisease groups, particularly ethnicity, been properly addressed? . . . Was measurement of the genetic variants unbiased and accurate? [bold added]
In the case of the term accurate the problem is easy to see. To draw a sharp line between accurate and inaccurate makes little sense and ignores the harder and more valuable question how accurate?
The average scientist is religious in many ways, and this is one of them. It is part of what might be called religious method: the dichotomization of persons into good and bad. An example is saying you are going to heaven or to hell — nothing between.
Comments