Particle physics experiments usually invest a considerable part of the time used to produce a measurement in the task of determining the corresponding uncertainty on the estimate, or -when a new effect is observed (say a quantity is measured away from zero, when zero would be the "null hypothesis", predicted by the current model)- estimating the statistical significance of the observation.

The technology is quite advanced, as physicists have gradually lost their typical autharchy ("statistics is trivial, so I will figure it out by myself") and progressively inherited sound procedures from statisticians. Also, in HEP there are well-established rules for assigning summarizing names to different significance levels. In this minimal glossary, a 3-sigma effect (a measurement of a quantity found away by more than three standard deviations from the value expected by the null hypothesis), which occurs less than twice in a thousand trials by pure statistical fluctuations if uncertainties are Gaussian and correctly evaluated, is called "evidence" for the new effect being measured. A 5-sigma effect is dubbed "observation" of the new effect, and is usually taken as a conclusive proof of the genuine nature of the observed phenomenon: a five-sigma effect occurs by pure fluctuations less than three times in ten millions, for Gaussian distributed data.

The above wording is well known not just by professionals in the field, but even interested laymen reading this blog or anyhow interested in HEP nowadays. So it is a bit surprising to find physics professors in theoretical physics preferring to rely on polls to determine whether an experimental result genuinely exceeding three standard deviations can be called "evidence" of the effect or whether it should rather be dubbed as "inconclusive" because of personal bias. Even more surprising, at least to me (but probably I should know better, given the particular individual doing this), is to be called "out on a limb" because I call things with their proper, well-established name.

[By the way, it is somewhat amusing that I am writing these lines from Madrid, in the course of attending a conference on the controversy of Hypothesis testing...]

The recent results on the Higgs boson search by CMS and ATLAS can be eyeballed to exceed, if taken together, a significance of three standard deviations by a good margin (unofficial fits by Philip Gibbs estimate a total of 4 standard deviations): although the experiments have not combined their results yet, and from a scientific standpoint we need to wait for them to do so, the crossing of the "evidence" mark is a rather unavoidable conclusion for any consumer in good faith.

Further, one should consider that ATLAS alone has quoted a 3.6 standard deviation effect, in the same region of interest already pinpointed (from my perspective, being a CMS member and thus previously aware of the CMS results when ATLAS released theirs) by a CMS 2-point-something-sigma effect. What I am saying is that the dreaded "look-elsewhere effect" can be waived in some cases, such as if an experimenter has already identified a narrow mass range where he then observes another experiment finding a significant effect. We should not forget that the look-elsewhere effect and the corresponding trials factor depend on the user: for instance, a theorist might have a model implying the Higgs mass being 125+-1 GeV, and so for that theorist the 3.6 sigma by ATLAS need no derating at all: he or she only looked there.

So calling the result a "firm evidence" in this blog -explaining very clearly in the body that the words are my own conclusion on the presented results- cannot be really questioned on the substance.

But professor Strassler objects, being very concerned of media storms, of horrible backfiring if the result is not confirmed, and even reaching the conclusion that I must be under a lot of hot water (i.e., angry reactions by my LHC colleagues) because of allowing myself to report the result in the subjective way I mentioned above: he shows he does not know what are the rules of a HEP collaboration, but we can forgive him for that. Rather, let us concentrate on the issue of "firm evidence".

So what does professor Strassler uses to counter my very standard "plain English" translation of the ATLAS and CMS results ? He quotes the results of a poll he conducts with experimentalists and theorists he knows, asking them if they believe the results constitute a proof of Higgs existence. Note, the question is quite different from asking for a plain English translation of the presented evidence. In any case, he gets many answers along the lines that the results are still inconclusive; he does discard some by theorists he does not consider qualified to express an opinion (yes, Lubos, you are in that category!). To be fair, he warns his readers that this is a biased test; but the fact he uses that biased test to cast doubt on my judgement of the LHC result remains.

Finally, I should mention I am also attacked for failing to insert a disclaimer in my Higgs evidence post, explaining to users that many three-sigma effects have boiled down to nothing in the past. This is beyond amusing - any reader who has spent here more than a minute knows how much care I always put into explaining that in particle physics the bar is set very high before claims can be made, and that evidences for new physics unavoidably evaporate with more investigation. I am responsible for not boring you readers here, and disclaimers are among the things one should really avoid in a blog.

I tried to get professor Strassler back to reality in the comments thread of his post, in vain: he does not seem willing to discuss the issue at hand -why he does not like a three-sigma result to be called evidence- nor the ancillary issue of unfairly contrasting answers on whether people believe the Higgs has been "observed" with my own description of the result; he only insists in criticizing my "firm evidence". That's too bad -he is a theorist with a genuine interest in experimental issues (he claims he is working on a trigger for the LHC experiments!) so it would be nice to discuss things without personal attacks polluting the ground.