The limits of prediction: from the Oracle of Delphi to artificial intelligence
Why do language models work, but financial models fail?
By Francisco Rodrigues, Universidade de São Paulo.
I n the early 2000s, while randomly browsing television channels — since YouTube didn’t exist yet — I came across a documentary about Nostradamus, the famous 16th century French astrologer who claimed to predict the future using a magic mirror or a bowl of water. The 1981 documentary (The Man Who Saw Tomorrow) was presented and narrated by Orson Welles and related passages from Nostradamus’ books to historical events. It was truly impressive: according to the production, he had predicted the French Revolution, the rise of Hitler, the Second World War, the assassination of President John F. Kennedy in 1963 and even the Iranian Revolution of 1979. It all sounded incredibly accurate and seemed to make sense.
However, this impression began to crumble when the 1981 documentary examined predictions made after 1982. For example, World War III was predicted to begin in 1999, last 27 years, and end with the defeat of the Antichrist, after which there would be a thousand years of peace. By the time we reached the 2000s, that war had still not taken place. Other predictions about the future also turned out to be completely wrong. In other words, by the end of the documentary, it was clear that the predictions had been accurate in the past because they were based on events that had already occurred, and Nostradamus’s ambiguous phrases could easily be applied to them. However, as far as the future was concerned, there was no way to adjust them — the predictions were clearly wrong.
Throughout history, predicting the future has always been a valued and essential task. As far back as ancient Greece, important decisions were preceded by consultations with oracles. Citizens travelled to cities such as Delphi and Delos to hear sacred prophecies. In the temple dedicated to Apollo, the Pythia — a priestess inspired by Apollo — would enter a trance and utter cryptic words. Priests would then interpret these messages and translate them into guidelines for kings, generals, and ordinary citizens.
The messages were unclear and open to multiple interpretations. A famous example of this occurred around the 6th century BC, when Croesus, the king of Lydia (an area that now forms part of Turkey), wanted to attack the Persian Empire, which was then ruled by Cyrus the Great. Before starting the war, he consulted the Oracle of Delphi to ask whether he should go ahead. The Pythia’s answer was ambiguous: “If Croesus crosses the River Halys, he will destroy a great empire.” Encouraged by this response, Croesus assumed that the ruined empire would be that of the Persians. He crossed the river and began his attack. However, it was his own empire that was destroyed.
In other words, it wasn’t clear what the ‘great empire’ would entail. While the answer was technically correct, it left room for different interpretations. This ambiguity also served to preserve the oracle’s authority, even in the face of unexpected outcomes.
Even today, fortune-tellers use the same techniques as in ancient times. However, predicting the future remains an extremely difficult task, even when it comes to more serious predictions made without the use of magical powers. This is evident in drawings and projects from the 1900s that attempted to envision what the year 2000 would be like. Our vision of the future is always biased. For instance, many artists at the turn of the 20th century imagined that cities would be filled with airships and steam-powered flying cars by the year 2000, and that robots would perform all the housework while wearing Victorian attire. Although technology advanced, social customs and aesthetics remained stuck in the present. In other words, when trying to predict the future, we tend to extrapolate current technological trends without being able to imagine significant cultural, social or scientific changes.
However, scientific progress has demonstrated that it is possible to gain insight into the future through scientific knowledge. Today, statistical, mathematical and machine learning tools can be used to make predictions based on large volumes of data, replacing magic and guesswork with quantitative models. These models can anticipate economic trends, consumption patterns, epidemics and even social behaviour. Examples include economic indices, which estimate the impact of public policies on unemployment or GDP growth, and electoral models such as those developed by Nate Silver, which predict results with high accuracy based on polling data and historical patterns.
However, these predictions are subject to important limitations. Like the Oracle of Delphi, modern algorithms operate in an uncertain environment. While they can use past data to make inferences about the future, they cannot capture rare events, historical ruptures, or the full complexity of human decision-making. A striking example of this is the 2008 financial crisis. Despite the sophisticated models used by banks and risk agencies, almost no one predicted the imminent collapse of the global financial system. Many algorithms had been trained using data from a period of apparent stability, which rendered them “blind” to hidden systemic risks. When the unexpected happened, the models failed precisely because they hadn’t considered the possibility of a structural breakdown. Therefore, models learn from data, so if the data does not provide enough information, the models are unable to predict the future. There is no magic involved.
“Prediction is very difficult, especially when it comes to the future.” — Niels Bohr, winner of the 1922 Nobel Prize in Physics.
The recent outbreak of the novel coronavirus (SARS-CoV-2) has once again highlighted the limitations of data-based forecasts. Although epidemiology has robust models at its disposal, the virus’s rapid spread, political decisions, population behaviour and genetic mutations have created a highly dynamic and uncertain scenario. Models have had to be constantly updated, and their projections have often been challenged or misinterpreted. A fundamental point to take into account here is that when the behaviour of the modelled system changes, historical data can become useless for predicting the future. For instance, urban mobility models based on pre-pandemic patterns became obsolete overnight when lockdowns and remote working transformed the movement of people in cities. Furthermore, the emergence of vaccines has introduced new factors that influence the course of the pandemic — and consequently, forecasts.
When the behaviour of the modelled system changes, historical data can become useless for predicting the future.
In addition, other diseases and even social behaviours have been influenced by the pandemic. For instance, in a recent paper we demonstrated the substantial impact of the pandemic on dengue surveillance and transmission dynamics in Brazil. This has important implications for interpreting epidemiological data and responding to outbreaks. In other words, the evolution of the number of recorded dengue cases has been affected by lockdown measures and hospital occupancy, which directly impacts future forecasts based on this data. Therefore, most epidemiological models need to be adjusted to take this scenario into account. The same applies to crime data and product sales.
Despite all this uncertainty, it is possible to predict the future if certain conditions are met. The first of these concerns the relationship between the signal and the noise present in the data. The signal represents consistent and significant patterns in the data, while noise corresponds to random fluctuations, measurement errors and momentary variations that obscure the system’s actual behaviour. For instance, when analysing a city’s average daily temperatures throughout the year, the signal would be the predictable seasonal variations — such as the hottest summer and the coldest winter — while the small daily oscillations caused by momentary weather changes would constitute the noise. Generally, the greater the noise relative to the signal, the more difficult it is to predict the system’s behaviour.
Therefore, for prediction to be viable, the first condition is that the data shows predictable behaviour, whereby the signal corresponds to a detectable pattern amidst the noise. For instance, when conducting an experiment to predict the position of a ball in free fall, the fewer the external interferences, such as air resistance or variations in the accuracy of the measuring instruments, the more accurate the prediction will be. In a controlled environment where noise is minimised, the ball’s movement follows the laws of physics with great precision, enabling mathematical models to produce highly reliable predictions. In this case, the signal will be much stronger than the noise.
The second essential ingredient for predicting the future is memory, or rather the fact that a system’s current state is influenced by its previous states. Systems with memory are influenced by the past, which can provide valuable insights into the future. This principle forms the basis of many predictive models: by identifying recurring historical patterns, future trends can be projected. For example, a plant’s current height is influenced by accumulated factors from previous days, such as the amount of sunlight, water, and nutrients it has received. Similarly, time series in economics or climatology often exhibit correlations over time: variations in GDP, average temperatures and unemployment rates, for example, tend to follow trajectories influenced by previous conditions.
In memoryless systems, also known as Markovian systems, the future state depends exclusively on the present state, regardless of the complete history. For instance, in a game of draughts, the board’s configuration at any given moment — i.e. the position of all the pieces — contains all the information required to determine the possible moves and the game’s subsequent state. In other words, to decide the next move, it is sufficient to know the current board configuration, without needing to know the full sequence of previous moves. This is what characterises a Markovian process. While this can simplify modelling, it also limits the ability to capture more complex effects or long-term dependencies, as predictions are based solely on the current situation.
“Life can only be understood by looking back; but it can only be lived by looking forward.” — Søren Kierkegaard, Danish philosopher, theologian, poet and social critic.
Another fundamental premise of forecasting is that the data must be stable. The behaviour of the system must remain relatively consistent throughout the forecast period. If a structural change occurs, known as concept drift, a model trained using old data may become invalid. A clear example of this is the SARS-CoV-2 pandemic. When vaccines became available, a paradigm shift occurred that profoundly altered the dynamics of the epidemic. Data collected before the introduction of vaccines became largely obsolete for predicting future behaviour, as the system began to operate under new conditions. The same reasoning applies to predicting television viewing levels based on 1990s data — a period before YouTube and other digital platforms existed. This is a classic example of how more data does not necessarily lead to better performance. If the data reflects a time when the system behaved significantly differently to how it does today, using it can not only be useless, but also compromise the accuracy of a machine learning model.
Another challenge in prediction is ensuring that the data contains relevant variables or characteristics that reflect the system’s dynamics. Insufficient information or the absence of key variables can severely limit a model’s predictive capacity. For instance, in models predicting the number of dengue cases, the absence of data on temperature, humidity, social factors, and mobility patterns can significantly reduce the accuracy of the predictions. Without the right variables, the model may only capture a fraction of the phenomenon’s complexity. (For those interested, we have written a paper in which this effect is evident in dengue cases. Link to paper).
In addition, it is essential that the model captures the relevant patterns without being overly complex (to avoid overfitting) or too simple (to avoid underfitting). Overfitting occurs when the model fits the training data perfectly, including noise and details specific to that set, but loses its ability to generalise, rendering it useless for predicting new data. In this case, the model can “memorise” the data, but cannot “learn” it. This is what happens in some exaggerated interpretations of Nostradamus’ prophecies, for example: they accurately describe past events, but fail to anticipate the future.
Conversely, an underfitting model is so oversimplified that it fails to capture the underlying patterns in the data. Consequently, it is unable to explain the past or make reliable predictions for the future. For example, it would be like estimating a store’s daily sales using only the day of the week as a variable, while ignoring important factors such as promotions, holidays, weather, local events and seasonality. If the model is overly simplistic — for instance, if it uses a fixed average for each day of the week — it will likely fail to capture real variations in sales.
Finding the balance between complexity and simplicity is therefore one of the main challenges in developing effective predictive models.
Thus, based on what we have discussed so far, five fundamental conditions must be met for it to be possible to predict the future with any degree of confidence: (i) the signal must stand out in relation to the noise in the data; (ii) the system must have memory, i.e. the current state must depend, at least in part, on past states; (iii) the behavior of the data must be stable, so that the patterns observed in the past are maintained in the future; (iv) the relevant variables for prediction must be available and well measured; and (v) the machine learning model must have the appropriate complexity — neither excessively simple to the point of underestimating the patterns, nor so complex that it ends up adjusting to the noise.
Once these challenges have been overcome, it becomes possible to produce highly accurate forecasts using mathematics and computing. A typical example of this is weather forecasting. Thanks to continuous improvements in meteorological modelling, we can now predict temperature, rainfall and other phenomena with relative accuracy over a period of a few days. At a larger scale, long-term climate models consistently indicate global warming and highlight the associated risks of an increase in the planet’s average temperature.
Similarly, infectious disease forecasting models provide fairly accurate overviews of known diseases. However, the emergence of new epidemics, such as the current one caused by the SARS-CoV-2 virus, still represents a major challenge due to the lack of previous data and the speed with which the situation can change. Nevertheless, we now have a good understanding of how various epidemics evolve, particularly so-called neglected diseases such as dengue and malaria. In cases where data is scarce, our previous studies (link to the paper) have shown that transfer learning can be an effective solution. This approach involves training the model on data from other diseases (such as influenza) to improve predictions for the target disease (such as SARS-CoV-2).
Forecasts in the financial market, on the other hand, are considerably more uncertain. This is because share prices are influenced by a huge number of variables, such as economic policies, rumours about companies’ situations, geopolitical conflicts, government decisions and technological innovations. Furthermore, even if a model were capable of providing accurate forecasts, they would have little practical value. This is because knowing the future would change the present. For example, if investors knew for certain that the price of a share would fall tomorrow, they would sell today, causing the price to fall immediately. This is a classic example of how anticipating a future event can backfire and directly influence the present, thereby rendering the prediction impossible. Therefore, when there is this kind of feedback loop between prediction and behaviour, predicting the future becomes even more challenging because the very act of prediction alters the conditions being predicted.
Unlike in the financial market, language models have proven to be remarkably effective at predicting future sequences of words or sentences within a given context. In this case, the models predict the next word (or ‘token’) in a text, based on the preceding context. But why does this happen? Essentially, it is because all five of the fundamental principles of prediction that we discussed earlier are well established. Firstly, these models are trained on huge databases in which the signal is much stronger than the noise. There are clear and recurring patterns in written language, making it easier to identify statistical regularities. Sentences also have memory: their syntactic and semantic meaning depends directly on the previous words. The grammatical structure of language imposes rules that make the next word highly dependent on the established context.
Another crucial factor is that the relevant variables for text construction, such as vocabulary, style, syntactic structure and discursive coherence, are well represented in the huge volume of data. This enables efficient “learning” from the data, without the need for explicit rules. Paradoxically, using large volumes of data also helps to avoid overfitting: although the models are complex, their ability to generalise to new contexts is enhanced by the diversity of the examples. Finally, language is relatively stable. During text generation, there are no abrupt changes in the external context, which favours internal consistency and facilitates prediction. The writing pattern tends to remain consistent in terms of style and topic throughout a text.
Therefore, the combination of large volumes of data, language stability, memory and a regular structure means that models such as ChatGPT, DeepSeek and Claude perform impressively in predicting the next word — thereby convincingly simulating human language use.
The impressive success of language models (LLMs) raises an inevitable question: if we have enough data, will we be able to predict other complex systems, such as epidemics or economies, with the same precision? In other words, are we about to resurrect Laplace’s demon?
To put this in context, in 1814 Pierre-Simon Laplace formulated a deterministic view of the universe. According to this view, if an intelligence knew the position and velocity of all the particles in the universe at a given instant, it would be able to accurately predict the entire future and reconstruct the entire past. This hypothetical being became known as Laplace’s demon.
However, two scientific revolutions in the 20th century profoundly challenged this view: quantum mechanics, which introduced the concept of fundamental uncertainty to the laws of nature; and chaos theory, which demonstrated that deterministic systems can exhibit long-term unpredictability due to their extreme sensitivity to initial conditions.
Nevertheless, advances in machine learning methods have once again challenged the boundary between predictability and unpredictability. Recent work by researchers at the University of Maryland, for example, has shown that chaotic systems can be predicted to some extent using neural networks (link to the publication), even without explicit knowledge of the equations that govern their dynamics. In a recent study, we demonstrated that the properties of chaotic systems can be predicted using recurrent neural networks (see publication for details). This raises profound questions: is unpredictability merely an epistemological limitation of our knowledge rather than an ontological limitation of reality itself? Are we getting closer to the Laplacian vision by other means with the new models?
Although we are still a long way from achieving the kind of intelligence that fully understands the world, such as Laplace’s demon, the progress of data-based modelling shows that under certain conditions, chaos can reveal patterns and uncertainty can be partially tamed. This means that to some extent, the future can be glimpsed.
In the case of quantum mechanics, Heisenberg’s uncertainty principle establishes a fundamental limit: the more precisely we know the position of a quantum particle, such as an electron, the greater the uncertainty about its velocity (or momentum) — and vice versa. This uncertainty is not a technological or epistemological limitation, but an ontological one, i.e. it is part of the very nature of the quantum world as we understand it today. But could machine learning overcome this barrier? Could we, for example, predict the knowledge gap — or anticipate, based on hidden patterns in the data, what we are traditionally unable to measure simultaneously?
While this hypothesis is fascinating, as far as we know, there is currently no evidence to suggest that any machine learning technique can violate Heisenberg’s uncertainty principle. However, what models can do is extract probabilistic patterns more efficiently, thereby improving predictions within the statistical limits allowed by quantum theory itself. In other words, algorithms can help us to better understand the probability distributions that govern quantum systems, but they cannot escape the fundamental constraints imposed by physics.
Nevertheless, with more data, more sophisticated models and a growing understanding of the complexity of natural and artificial systems, we may be able to refine our predictions to the extent that they approach Laplace’s deterministic vision in some domains. We don’t know yet, but the fact remains that the limits of prediction are being challenged by the current artificial intelligence revolution. The future is uncertain by nature, but with each technological advance, we seem to gain a little more predictability.
To find out more:
Popular Science Book:
- The End of Certainty: Time, Chaos, and the Laws of Nature, Ilya Prigogine.
Technical Book:
- Forecasting: Principles and Practice, Rob J. Hyndman and George Athanasopoulos.
Papers:
- Luiza Lober, Kirstin O Roster, and Francisco A Rodrigues, Forecasting infectious diseases in Brazilian cities: integrating socio-economic and geographic data from related cities through a machine learning approach, Chaos, Solitons and Fractals: the interdisciplinary journal of Nonlinear Science, and Nonequilibrium and Complex Phenomena, Volume 187, 115417 (2024).
- Kirstin Roster, Tiago Martinelli, Colm Connaughton, Mauricio Santillana, and Francisco A. Rodrigues, Impact of the COVID-19 pandemic on dengue in Brazil: Interrupted time series analysis of changes in surveillance and transmission, PLOS Neglected Tropical Diseases (2024).
- L. Lober, M. S. Palmero, F. A. Rodrigues, Parameter Inference in Non-linear Dynamical Systems via Recurrence Plots and Convolutional Neural Networks, Physical Review E, 2025.
If you’re curious about my research, visit this link:
See you next time!
