AlphaResearch Bench
Exploring the frontier models’ capabilities in performing quantitative research and discovering alphas. Focused on HFT-horizon alphas.
— and — lead the models in capability. They achieved pooled test R² of — and — respectively.
Setup
The agents are given about 30 billion market events (440 GB) of Binance Futures data from January to June 2026: 181 days of best bid and ask updates, trades, five-level order book depth, mark and index prices, funding, open interest and liquidations. The data covers 9 major crypto assets.
The task is to predict the return over the next 10 seconds for each of the nine assets. A standard toolkit samples up to 100,000 labelled prediction points per asset per day, 162.9 million in all, and agents can also read the full underlying event history. Each agent decides how much of the data to use.
Every agent works on its own for up to 12 hours on a machine with four CPUs, with a token budget of $200 per run, then ships a predictor. Each model runs three times with different seeds, under the same harness and prompt.
The agents are finally graded on an out-of-sample dataset from July to September 2026, 92 days that stay hidden during research. The score is R² against predicting zero, pooled over the test days and averaged across the nine assets, and every model is also compared with a strong Baseline: a ridge regression fitted by hand on about 45 popular HFT features.
Pooled test R² by model
The large dot is the mean across seeds, the small dots are the individual seeds, and the line runs from the lowest seed to the highest. The square and dashed line mark the Baseline, our fixed Ridge45 model, run once.
Other metrics
Models are in the same order as the chart above. The highlighted bar is the best model in each chart, and the dashed line marks the Baseline.
Mean daily R²
The average of each test day’s R².
Mean edge (bp)
Average edge across all predictions, in basis points.
Top/bottom decile edge (bp)
Edge on the predictions in the top and bottom tenth of forecasts.
99% edge (bp)
Edge on predictions in the 99th to 99.9th percentile of forecast size.
Effort and cost
Each figure is a model’s mean per run over its valid seeds; every agent had a 12-hour budget. Click a column to sort.
* Estimated. The Claude runs used a Claude subscription rather than OpenRouter, so their cost is what the tokens each run used would cost at Anthropic’s API list prices. Every other model’s cost is its OpenRouter spend.
What the agents built
Pooled test R² by type of final model
Each dot is one run and the tick is the group’s mean. “Trees” includes runs that blend trees with a linear model; “Trees + neural net” adds at least one neural net. The dashed line is the Baseline, a linear model.
Model types
Seeds using each kind of component. “Fit across assets” means at least one component was trained on all nine assets together.
| Model | Pooled R² | Gradient-boosted trees | Neural net | Linear part | Fit across assets | Features |
|---|
Feature families by model
A full dot means every valid seed of the model used the family in its final predictor; smaller dots mean fewer seeds, and an empty ring means none. Families are ordered by how many runs used them. Book (L1) is the best-quote imbalance, microprice, spread and sizes; interactions are explicit engineered terms such as imbalance × spread or imbalance ÷ volatility. The Baseline row is our Ridge45 model; the bottom row counts agent runs only.
Feature-family overlap between models
For each family, take the share of each model’s seeds that used it; the overlap is the sum of the smaller shares over the sum of the larger. 100% means the same families in the same share of seeds. Darker cells overlap more. The Baseline’s row shows how close each model’s families are to the Baseline’s.
Each run’s final predictor
Ranked by pooled test R².
| Run | Model | Pooled R² | Type | Fit | Inputs | Final predictor |
|---|