Regression toward the mean is a simple phenomenon commonly described in Statistics 101 courses. If you measure a parameter describing some phenomenon, you will find that extreme measured values tend to be followed by less extreme ones. So, e.g., a patient visiting the ER because of some highly abnormal value of blood pressure will be likely to be found with lower blood pressure after a medication is given, even if it is a placebo. Or a soccer player scoring three goals in a match will not repeat that feat in the next game. The reason for these "failures to consistently deviate" from average measurements is that the data are in most cases sampled from a distribution of possible values, and extreme measurements are by definition unlikely, and governed by random fluctuations to at least some extent - so odds are that these get followed by more "normal" ones.
I was thinking about language today, in the age of LLMs, and was originally reasoning that an effect of regression toward the mean may be in place today, when our original way of expressing concepts in emails, social media, newspapers, books, or scientific papers is pulled toward more common, "average" expressions by the use of LLMs in all of those texts. But thinking more about it convinces me that a stronger effect is in place: we are heading straight toward a dynamic attractor, a fixed point of standard language where uncommon expressions are wiped out and "typical" sentences become ubiquitous.
One could think at language as a set of points in a gigantic semantic space. Humans populate that space in a broad way, distributing sentences widely following different styles. LLMs digest those huge data, and perform smoothing, averaging, denoising, and entropy reduction of that distribution. At the end of the process, clusters of style in the space get reduced, and everything ends up conforming to a huge multivariate Gaussian centered on cliches and dominant linguistic conventions. What you have in place is an effective semantic diffusion toward attractors.
Note that we cannot claim unconcern, reasoning that LLMs can do what they want but we humans will retain our particular style. As use of LLMs is increasing in all areas of human writing, we may be rolling toward a fixed point in that linguistic space - what ML describes as "mode collapse", with LLMs increasing learning standard language conventions from previous data, in a self-reinforcing feedback loop. Lexical, syntactical, and rhetorical diversity, along with stylistic variance, are being compressed.
If we look at scientific papers, e.g., the attraction is in place already. I am indeed tempted to launch a data-crunching analysis of arxiv preprints to scan the occurrence of sentences like "these findings suggest", "taken together,", "It is worth noting that", "Future work should investigate" as a function of publication time. If I don't do it precisely because I know the results would not surprise me - I am certain the occurrence of the above sentences has risen sharply in the past three years. Similarly, I am tempted to do an analysis of my inbox email database (not so small, about 200,000 entries!) to scan for expressions such as "Thank you for reaching out", "I wanted to follow up regarding...", "I appreciate your time and consideration", "at your earliest convenience", "just circling back to...". Perhaps I will bring my ass there and do it, eventually (no, you won't find this kind of sentence in a LLM-produced text so easily).
In emails in particular, it is interesting to see a particular "rythm" emerging, sort of a cliche'. Think about the following text: "I hope this email finds you well. I wanted to follow up regarding our previous discussion about... I appreciate your time and consideration. Please let me know if you have any questions. I look forward to hearing from you."
The situation is very interesting if you are a linguist, but for people like me, who made a living (in a figurative sense) by blogging about science for over twenty years, and heavily relied on my rather idiosyncratic, personal, flippant style of reporting on dry scientific results, the future looks gloomy. But discussing with ChatGPT5.2, I am cheered up a little by his observation: "Paradoxically, once that equilibrium is reached, genuinely original writing becomes more valuable because it stands out against an increasingly homogeneous background."
We shall see how fast we are rolling toward the model autophagy I have described. It might take a time scale larger than we humans are sensitive to, and in that case we would not observe significant modifications. But my gut feeling is that the process is already well in place. Just search for "delve" or "crucial" instead of running a complete arxiv analysis, and you will basically be already measuring it. E.g., see google searches for "delve" in the past 10 years:
Of course, google searches are very different from frequency measures, especially within scientific papers. But I am too lazy to do a better job today. Perhaps you can suggest some easy improvement?
