Back in 2009 I wrote in this blog a rather technical post about something I had called the “greedy bump bias.” (GBB) The effect I referred to had emerged from a 2003 CDF study of mine, where I considered the extraction of small signals sitting on top of much larger backgrounds.That work went unpublished, but then I took revenge with the blog post... But never put a full stop on a lingering question, until today.
The problem itself is extremely common in particle physics. Suppose that you are looking for a new particle which would appear as a narrow bump in some mass distribution. You do not know how many signal events there are—that is what you want to measure—but, crucially, you also do not know exactly where the bump should be. So you fit both things at once: its size and its position.
That sounds completely innocent, and if the fit is considering the precise statistical distribution of the data, encoded in a likelihood maximization, you do not expect to incur in some pitfall. However, there is a subtle consequence. If the bump is allowed to move in the fit --i.e., if the mass is not constrained--, it can make use of the statistical fluctuations in the data. A slightly positive fluctuation on one side of the place where the signal is sitting may attract the fitted peak; a negative fluctuation on the other side may push it away. The fitted position has no preferred direction, but the fitting algorithm does have a preference: it moves in whichever direction improves the fit.
Because of that freedom, which in Statistics is described as the presence of a nuisance parameter m (the free-floating mass), the fitted signal size ends up, on average, a little too large. That was the effect I discussed in my old post, which you can read here. In that article, I investigated it numerically with pseudoexperiments and observed how the bias becomes small when the signal is strong.
What I did not understand then was that this little estimation bias has a rather beautiful relation to one of the best-known statistical nuisances of particle physics: the look-elsewhere effect (LEE).
The look-elsewhere effect (about which I wrote many times in this column, e.g. here and here) concerns discovery rather than measurement. Imagine that there is actually no new particle in the data. If you test one predetermined mass value and find a three-sigma excess, that has one probability of occurring by chance. But if you have searched hundreds of possible masses and reported the most significant excess, three sigma is considerably less impressive: you gave the background many opportunities to fool you.
The usual way of saying this is that the "global significance" of the signal is smaller than the "local significance", when the latter is what your fit finds. Or, equivalently, that the search incurs a “trials factor”, TF.
The two effects mentioned above - GBB and LEE- have usually been discussed separately. The greedy bump bias is an alternative-hypothesis effect: there really is a signal, and fitting its unknown location biases its measured amplitude upward. The look-elsewhere effect is on the other hand a null-hypothesis effect: there is no signal, and searching over unknown locations increases the probability of finding a strong fluctuation somewhere. Overlooking the LEE has been the source of many claimed resonances in particle physics searches, some of them passing refereeing and getting published. Incidentally, the requirement that a signal passes the 5-sigma significance threshold before it can be the subject of claim of observing a new physics phenomenon is a patch invented in 1968 by Arthur Rosenfeld, which started to be used only when the top quark search was in full swing at the Tevatron collider. But that is another story...
Now, when in 2010 two ATLAS colleagues published a deep study of the LEE, I immediately sensed that their result had some connection with the GBB effect I had unearthed, but I could not quite make the connection clear to myself. Not any more: The new paper I wrote (with some AI help) asks whether that separation is hiding something. And the answer is yes!
Geometry, geometry
The key to understanding the connection between GBB and LEE is to think of all possible signal shapes as directions in a high-dimensional data space. For each possible mass, width, or other shape parameter, there is a slightly different signal template. If the templates are normalized, the family of all of them forms a curved surface.
Changing the signal amplitude then moves us radially outward from the background-only point. This gives the geometry sketched in the top panel below.
The point at the tip of the cone in the top panel is the background-only hypothesis. Once a nonzero signal is present, its amplitude points outward while variations of its mass, width, and so on correspond to D tangent directions along the surface of possible signal shapes. When we optimize their value, in the presence of a sufficiently strong signal, noise pushes the fitted position around in those D tangent directions.The displacement itself averages to zero, but optimization turns its effect on the fitted peak height into a positive quantity.
The remarkably simple leading result is B=D/(2Q)
This result was not, as I initially believed, completely new. Closely related optimization-bias calculations exist in statistics, astronomy, and random-field theory, and an important part of writing the paper was discovering how much of the local result was already implicit in earlier work. The interesting part turned out to lie elsewhere.
Turning the signal off
Now imagine shrinking the signal toward zero. Something dramatic happens at the tip of the cone in Figure 1. If the signal amplitude is zero, asking for its mass makes no sense. A nonexistent resonance does not have a mass or a width. In statistical language, those parameters become unidentifiable under the null hypothesis. This is precisely the singularity that generates the look-elsewhere problem.
So away from the tip, the same directions describe how a real signal can move around to exploit noise. At the tip, they describe how background noise can be searched over to manufacture the most signal-like fluctuation. The mathematics makes the connection unexpectedly sharp.
At high significance the trials factor TF grows roughly as a power Q^D. The exponent is again exactly the number of locally adjustable signal parameters. More intriguingly, the two quantities can be connected directly: B = 1/2 d/dQ log TF(Q). This is an equation I did not find in the literature. In words: the strong-signal optimization bias is one half of the logarithmic slope of the high-threshold look-elsewhere trials factor.
I find this relation rather satisfying. It does not say that the greedy bump bias and the look-elsewhere effect are the same phenomenon - They are not, of course. But what it says is still very relevant: it says that their leading behavior is generated by the same local geometry of the signal family.
The global size of the search matters enormously for the look-elsewhere effect: it does not matter for the leading greedy bias of a strong signal. Taking the logarithmic derivative above removes precisely that constant global search-volume factor and leaves the local contribution due to continuous optimization.
From local to global
Of course, a strong-signal approximation cannot possibly tell the whole story. If the signal becomes weak enough, the maximum found by the fit need no longer be the slightly displaced version of the true signal peak: some completely unrelated fluctuation elsewhere may become larger. That leads to the second part of the paper, from which I singled out another figure to explain here.
The upper panel in the figure above (fig.3 in the paper) shows the bias in the globally fitted maximum as the true signal strength is reduced. For large Q, all curves collapse toward the local prediction. It does not matter whether we searched over a mass range spanning six, ten, or fourteen signal widths: the real signal anchors the fit in its own neighborhood. But as Q decreases, however, remote maxima caused by statistical fluctuations begin to win over the true signal strength, driving the fit to consider them as the "signal". This is exactly what the LEE is.
The larger the search interval, the more possible competitors there are, so the global bias starts to depend strongly on the volume searched in the phase space spanned by the "nuisance parameters" in the fit.
The lower panels show this happening directly: the contribution from remote peaks grows, together with the probability that one of them overtakes the signal-associated maximum. This provides a rather concrete way of watching the local greedy-bump problem morph continuously into the global look-elsewhere problem.
A little devil hiding somewhere
The study also allowed me to unearth a small detail. A common approximation in look-elsewhere calculations is to imagine sufficiently high background peaks as approximately independent rare events—a Poisson process. That works beautifully at very high thresholds. But at the moderate thresholds relevant to the crossover, the maxima of a smooth random field are not independent: they sort of "know" about one another - they share the nuisance parameter space and nudge each other. In particular, two maxima cannot sit arbitrarily close together: a smooth curve has to descend from one maximum and climb again before producing another positive bump.
Using a statistical calculation I could not have performed alone (I got help from ChatGPT 5.6 sol), I could quantify this correlation and show that the probability of two nearby maxima is suppressed as the fourth power of their separation. This matters for the finite-threshold corrections to the simple Poisson picture, and it also gives the paper a hierarchy which I now find quite useful conceptually: the number of adjustable parameters determines the universal leading local bias; the curvature of the signal-template family determines the first corrections to that result; the total search volume determines how many remote competitors can appear. and the correlations among those extrema determine the first departures from the asymptotic Poisson approximation.
What started for me many years ago as a small nuisance in fitting a Gaussian bump therefore turned out to connect several pieces of statistical theory that I had completely overlooked, then maybe briefly touched, but never considered jointly. Perhaps this is the part I like most about the result:the greedy bump did not merely turn out to be greedy. It had been looking elsewhere all along.
[If you are interested in the paper, it is available here].
