In 2000, Hal Pashler and I published a paper called “How persuasive is a good fit? A comment on theory testing.” For more than 50 years, experimental psychologists have supported mathematical theories by showing that the equations of the theory could fit their data. We pointed out that this was a mistake because no account was taken of the flexibility of the theory. A too-flexible theory can fit anything. However obvious this may sound to outsiders, the practice we criticized was common (and continues).

Recently I asked Hal: Is the problem we pointed out an example of something more general? Neither Hal nor I had a good answer to this. Both of us thought the practice we had criticized was what Feynman called cargo-cult science — looks like science but isn’t — but that was more of a derogatory description than anything else.

Now I think I have a helpful answer: What we pointed out was an example of the general point Thorstein Veblen made in The Theory of the Leisure Class: The growth of worse-than-useless practices among the well-off. Foot-binding. Hood ornaments. Long words and bad writing in scholarly articles. Conspicuous waste. The last chapter of Veblen’s book is about academia. The practice Hal and I criticized is worse than useless in the sense that it caused people to think that a theory had passed a meaningful test when that was not the case. Hal and I were unable to find even one theory supported in this meaningless way that had gone on to gather meaningful support.