Synth unlocks the future of predictive intelligence for LLMs and AI Agents. Miners have been competing to build the best forecasting models. Read this weeks' update to see how miners’ strategies evolved and responded to changing market conditions from 2025-02-17 to 2025-02-23.

Bitcoin Price Action

From February 17th to February 23rd, the price of Bitcoin opened at approximately $96,073 and closed at $96,263. During this period, the price reached a minimum of $93,381 on February 18th and a maximum of $99,480 before retracing. While the overall volatility seemed more contained compared to previous weeks, there were still significant price fluctuations. Notably, on Friday, February 21st at 2:30 PM UTC, a sharp price drop occurred, with Bitcoin falling from over $99,000 to around $95,000. This price decline was primarily attributed to two key factors:

1. The ByBit exchange hack announcement, where the platform reported a loss of approximately $1.4 billion in Ethereum;

2. A substantial sell-off on Binance, involving around 1,200 BTC, which triggered leverage position liquidations and further contributed to the price decline.

This sell-off moment has been highlighted in the figures below, showing the Bitcoin price action, log-returns, and 5-minute rolling volatility.

Miners Performance

For this analysis, we selected six miners: the top three according to the leaderboard scores at the beginning of the period (midnight, February 17th), and the top three at the end of the period (11:59 PM, February 23rd). This resulted in the following selection:

• Best miners at midnight on February 17th: Miners 4, 58, and 140.

• Best miners at 11:59 PM on February 23rd: Miners 4, 5, and 10.

Note that Miner 4 appeared in the top 3 both at the start and at the end of the period, so this commentary will focus on a total of five miners. What follows is a brief discussion of their performance during the selected period, particularly in relation to the volatility regimes described above. The analysis will include metrics such as Continuous Ranked Probability Score (CRPS), Softmax scores, and Leaderboard values. Miners who were the top performers at the beginning of the period will be visualized in shades of red, while those who were the top performers at the end will be visualized in shades of green. Four equidistant time points per day were selected for plotting the scores.

Continuous Ranked Probability Score (CRPS). Figure 2 below illustrates the CRPS values scored by the miners throughout the week. In this case, lower scores represent better model performance, while higher scores indicate poorer performance.

From the figure, it's evident that unexpected jumps in price volatility (or log-returns) led to an increase in CRPS scores, before stabilizing once the price settled. Models of the miners that led at the start of the period (Miners 58 and 140, marked in red) were consistently outperformed by the miners who were on top of the leaderboard at the end (Miners 4, 5, and 10, marked in green), except for a brief period between February 20th and February 21st. However, the significant Bitcoin sell-off described earlier triggered a sharp change in volatility, and Miners 4, 5, and 10 seemed to adapt better to the new volatility regime than Miners 140 and 58.

Softmax Scores

Softmax scores also provide a useful indicator of miners' performance during periods of high and low volatility. These scores offer a clearer view of how miners performed relative to one another. Unlike CRPS values, higher Softmax scores indicate better performance.

Additionally, it is important to note that at the beginning of the week, the β parameter that governs the calculation of Softmax scores was decreased from -0.001 to -0.002, which amplified the relative differences between miners.

Throughout most of the week, Softmax scores were fairly consistent, remaining between approximately 0 and 0.01. However, there were moments when Miners 4, 5, and 10 outperformed Miners 58 and 140. The first such instance occurred between February 18th and February 19th, when the scores of Miners 58 and 140 stayed close to 0, while the scores of the other miners rose to around 0.02. The second instance occurred between February 21st and February 22nd, when the scores of Miners 5 and 10 surged to a remarkable 0.12, with Miner 4 also performing well with scores near 0.08.

Both of these performance spikes occurred right after moments of increased volatility in log-returns, notably on February 18th and February 21st, suggesting that Miners 4, 5, and 10 had a more responsive and adaptive model that could adjust to sudden volatility shifts.

Leaderboard Scores

Miners 58 and 140 saw their leaderboard scores drop significantly early in the week, falling from values between 0.15 and 0.2 to near 0 within just 48 hours. In contrast, Miners 4, 5, and 10, after experiencing a simultaneous but less dramatic drop in leaderboard scores at the start of the week, quickly overtook 58 and 140 on February 18th. They maintained this lead throughout the remainder of the week.

This shift reflects the ability of these models to adapt better to the changing volatility forecasts, as evidenced by the rise in their leaderboard scores, which climbed from between 0.005 and 0.01 on February 19th to above 0.02 on February 22nd.

Takeaways

Bitcoin's volatility during this week was more subdued compared to previous weeks. However, key moments of volatility change, such as the BTC sell-off on February 21st, proved crucial for miners' performance. Miners whose models could adapt quickly to even subtle shifts in volatility regimes stood out in the final Softmax and Leaderboard scores. This can be attributed to two factors:

(a) Changes in the scoring parameters, such as the decrease of β for Softmax scores and the extension of the leaderboard scores' half-life to 2 days;

(b) A more competitive environment where top-performing miners are continuously refining their models to remain adaptable in the face of unexpected news and volatility spikes, in line with the spirit of the Synth subnet competition.

About Mode

Mode is building Full Stack DeFAI, the L2 scaling DeFi to billions of users through AI agents and AI-driven financial applications. Bolstered by a $6 million grant from Optimism, Mode continues to push for an open, efficient, and inclusive financial future on the Superchain.

Website: mode.network

Forum: forum.mode.network

Community TG Groups: Join our Telegram groups