Synth unlocks the future of predictive intelligence for LLMs and AI Agents. Miners have been competing to build the best forecasting models. But have they been able to forecast BTC price action? The following is an analysis of the data from February 3rd to 16th, 2025.

Bitcoin Price Action

During the two weeks from February 3rd to February 16th, Bitcoin's price action was mostly characterized by moments of low volatility, with prices ranging between $96,000 and $98,000 for most of the time considered.

However, two key moments of higher volatility can be identified even during this apparent calm in the market. The first occurred on the very first day under examination (Feb 3rd), when the price was most likely still influenced by President Donald Trump's U.S. tariff announcement just before the previous weekend. During this day (February 3rd), the price ranged between approximately $91,000 and $102,000.

The second significant moment of price volatility occurred at around 1:30 PM (UTC) on February 12th, when the U.S. CPI index data was disclosed. The results led to a spike in volatility, as the price dropped from approximately $96,000 to $94,000 within ten minutes. It is worth noting that the release time of the CPI data was known in advance, meaning that miners incorporating this information into their models—anticipating a step jump in volatility—could have scored particularly well.

The plots below depict price action (1-minute timeframe), log returns (1-minute timeframe), and rolling volatility (5-minute window) for the period under consideration. In particular, the log returns and rolling volatility plots allow us to observe the difference in volatility between the moments described above and the rest of the market days. Furthermore, the exact minute when the CPI index data was released has been marked to highlight the observed spike in volatility.

Miners Performance

For this commentary, we have selected six miners: the three best miners according to leaderboard scores at the beginning of the period (midnight, February 3rd) and the three best miners by leaderboard scores at the end of the period (11:59 PM, February 16th). This resulted in the selection of the following miners:

• best miners at midnight on February 3rd: miners 5, 10, and 11;

• best miners at 11:59PM on February 16th: miners 4, 58, and 140.

What follows is a short commentary on the miners' performance during the selected period, concerning the volatility regimes just described. In particular, there will be a discussion of Continuous Ranked Probability Score (CRPS), Softmax, and Leaderboard values. Miners who were the best performers at the beginning of the period will be visualized in shades of red in the plots, while miners who were the best performers at the end of the period will be visualized in shades of green.

To facilitate visualization, we selected four periods per day that are roughly equidistant from each other and plotted the scores at those times. Additionally, for visualization purposes, some of the plotted CRPS values have been clipped at the 95th percentile of the observed scores, as some miners were given huge scores that would have rendered the rest of the plot meaningless.

Continuous Ranked Probability Score (CRPS)

In general, it is possible to observe how higher volatility periods are associated with larger CRPS values. It is important to highlight that lower CRPS values indicate better performance for a given prompt. However, higher CRPS values across all miners during certain periods do not necessarily imply that all miners were performing worse. Instead, these were just periods of higher volatility, which naturally results in larger CRPS values.

On the very first day, when BTC price volatility was higher than in the following days, the miners' CRPS values were all between the 60K and 70K range. As the price volatility returned to more regular levels, the miners' values also tended to decrease, going as low as below 20K points. When the U.S. CPI data was revealed, the increase in volatility caused the CRPS values to increase again, this time reaching the 40K-50K range.

While at this point Miners 5, 10, and 11 still seemed to be performing better than the other miners, it was during the subsequent period, when volatility decreased again, that the latter miners outperformed the former ones. This could potentially be due to models used by Miners 4, 58, and 140 that were better adapted to the regime switch from higher to lower volatility or that were better fine-tuned during the latter days of the competition. It is worth noting, however, that all miners obtained lower CRPS values during the last days, ranging between 10K and 15K. This might be a sign of further model improvement during low-volatility periods.

Softmax Scores

Being a simple mathematical transformation of the CRPS values, the Softmax scores also provide a good reflection of the miners' varying performances across periods of high and low volatility. Unlike with the CRPS values, in this case, higher scores indicate better-performing miners.

In particular, we can observe how the Softmax scores tend to be close to—or lower than—0.01. However, for some miners, these scores rose above 0.1 as volatility decreased between February 5th and February 9th.

Another noteworthy aspect was that the forecasts provided by Miner 11 appeared to adapt better than those of other miners to the change in volatility regime that occurred on February 12th due to the release of the CPI index data. This miner achieved a Softmax score close to 0.06 in the second half of the day, while all other miners received scores between 0 and 0.02. Finally, as observed with the CRPS values, a return to a lower-volatility state, combined with possible model improvements, led miners 140 and 58 to outperform their competitors, while obtaining relatively low scores compared to the higher scores observed in previous days.

Leaderboard Scores

Miners 5, 10, 11, and 4 were among the best-performing miners for most of the period under consideration. In fact, their EWMA scores remained between 0.03 and 0.07 during the first week under review. However, due to increased competition and possible improvements in other miners' models, their scores began to decline during the second week (February 10th to 17th). During this period, their scores dropped to values between 0.01 and 0.02, while new miners who joined the competition later, such as Miners 140 and 158, quickly caught up within a matter of days and even managed to take the lead (along with Miner 4) around the last day of the competition (Sunday, February 16th).

Takeaways

The price of Bitcoin has remained largely stable throughout the two weeks under analysis, though notable spikes in volatility were observed on February 3rd and February 12th. During these periods of stability, miners generally performed better, thriving in the low-volatility environment. However, their performance tended to decline when volatility increased—not necessarily because they were performing worse, but simply due to the nature of higher volatility causing increased CRPS values. Additionally, since the competition is still in its early stages, miners are likely still refining and optimizing their models.

This highlights a key challenge for Synth miners: the need to develop and fine-tune models capable of anticipating or rapidly adapting to sudden changes in volatility, rather than relying solely on straightforward price forecasting. Moving forward, the ability to navigate volatility with agility will be a critical factor for success in this competitive space.

About Mode

Mode is building Full Stack DeFAI, the L2 scaling DeFi to billions of users through AI agents and AI-driven financial applications. Bolstered by a $6 million grant from Optimism, Mode continues to push for an open, efficient, and inclusive financial future on the Superchain.

Website: mode.network

Forum: forum.mode.network

Community TG Groups: Join our Telegram groups