I’ve spent years deep in quantitative trading research, and I’ve also been a heavy user of AI models since the first public releases, back when the impressive trick was writing a limerick.
Recently though, things have started to pick up speed.
The current generation writes real research code, runs it, reads its own output, and argues with me about the interpretation, and somewhere in there an idea I would have laughed at two years ago started looking buildable.
Now, I want to be delicate with this post, as the genre it belongs to is mostly garbage:
The “AI replaces all quants by 2030” forecasts, usually written by people who have never cleaned a corporate action in their lives.
So, we’re going to approach this the right way: from the mechanics up.
The question on the table is simple:
“What would it actually take to train a machine that does quantitative research at industrial scale?”
By this I mean a system that gets handed a dataset and an objective, runs its own experiments, kills its own bad ideas, and hands back something robust, running a few thousand copies in parallel.
Essentially a quant research desk, but ultra-scaled like compute.
Now, we won’t turn this into an academic ML sleeper, but to reason about any of this the right way, we first need to understand a few basics.
How a model gets good at things
A base language model is trained to predict text, and that alone gets you a generally capable generalist. It writes decent pandas, knows what a Sharpe ratio is, and can recite a few momentum papers.
Now, just predicting something based on what’s already known has a pretty low ceiling. What’s actually made novel research and fresh ideas possible is reinforcement learning (RL).
RL often sounds more mysterious than it is, so let’s think about how a junior quant actually learns.
Nobody hands them a rulebook covering every market state; they start with a hypothesis, put on positions, the P&L comes back, and the strategies that kept making money survive while the ones that kept losing get removed.
RL is that loop, run on a model, millions of times, with a reward/penalty standing in for the P&L:
Give the model a task where a computer (“grader”) can score the outcome: a math problem with a known answer, code with unit tests, etc.
Let it generate a full attempt, reasoning, tool calls and all. That whole attempt is called a trajectory.
Grade the outcome mechanically. Correct earns a reward (+1), wrong earns nothing.
Update the model so trajectories that earned a reward become more probable, and trajectories that didn’t become less probable.
Step 4 is where a lot of people in the space degenerate to bizarre overfitting, so let’s slow down a bit.
You might picture it writing down a rule like INVERSE_VOL = GOOD somewhere. That update is actually a nudge to billions of weights that makes the entire chain of behavior that preceded a reward slightly more likely to happen again.
The public proof this works is DeepSeek-R1, which was trained with rewards linked to final-answer correctness. Nobody told it to double-check its work, and by the end of training it was pausing mid-solution to re-evaluate its own reasoning anyway, because trajectories containing that behavior kept getting rewarded.
That’s pretty much it: a grader, a lot of attempts, and a rule that says be more like your better self.
Alright, that was the bulkiest academic part of this post. So now that you have some starting notion of how RL works, we can get our hands dirty.
The quant research game
Say you wanted to do this for quant research. Each training episode would look something like a fresh, self-contained research assignment:
Universe: 1,500 US equities
Period: 2004-2020
Fields: date, ticker, adjusted_close,
volume, market_cap
Objective: Construct a market-neutral
long/short momentum strategy.
You may write and execute Python.
The agent sees 2004 through 2020. The years 2021-2025 exist only on the grader’s side, and the agent is never shown them, or even told exactly where the boundary sits.
Then, you let it cook.
To demonstrate, here’s a plausible sequence of attempts, with numbers loosely borrowed from how this actually goes:
Naive 12-month momentum, deciles, equal weight, daily rebalance: Sharpe 0.83, monstrous turnover, 38% drawdown.
It notices cross-sectional vol varies wildly and tries inverse-vol weighting: 1.34.
Skip the most recent month (the classic 12-1 construction): 1.55.
Monthly rebalance instead of daily: 1.73.
All three together: 2.13.
Grade the out-of-sample years, reward accordingly, and repeat.
Across one experiment this is basically just a parameter search, and honestly RL would be overkill for that; a grid search finds rebalance = monthly too.
The interesting thing happens when the next experiment is Japanese equities and mean reversion, then the one after that is commodity carry, then futures trend, then earnings revisions, each with different hidden periods, different costs, different regimes.
Run 100,000 episodes of this.
Now, no single “trick” generalizes (e.g., “always monthly rebalance”). The research model will find that inverse-vol weighting helps in some experiments and is irrelevant in others.
What will generalize though are the research behaviors that led to good outcomes:
“Check whether risk differences are driving your portfolio before believing your signal”
“Test whether the effect lives entirely in microcaps, then impose liquidity constraints”
“Distrust any spectacular in-sample Sharpe on principle.”
“Compare neighboring parameters; a strategy that dies at
lookback = 11was never alive at 12”“Run subperiods, and check the whole result isn’t three good years carrying seven bad ones”
“When two constructions perform the same, keep the simpler one”
Those are the habits of a good systematic researcher, and under this training loop they’d be reinforced the same way Deepseek R1 learned to double-check algebra: they just keep getting paid. (chart below)
Now, markets give you one history, and you can’t generate 50,000 more S&P 500s to practice on, which is exactly why RL on raw prices has such a miserable track record.
The research process above sidesteps that limitation though, as the thing being practiced is the process, and you can manufacture endless distinct research problems from enough historical data.
You WILL train the world’s best p-hacker, at first.
Suppose your reward function is a naive one:
“Sharpe above 2 pays +1, anything else pays 0”
In order to earn a reward, your agent will “discover” that some obscure universe filter plus a 173-day lookback, excluding Tuesdays, rebalanced every 37 days, prints a Sharpe of 2.04 on the historical window.
This failure is known in RL as reward hacking.
Fortunately for us though, Finance met this exact adversary years before the AI labs did.
When you test N random, zero-edge strategies on the same history, the best one’s Sharpe grows like the square root of the log of N. In other words, search enough noise and noise will clear any bar you set (chart below).
Bailey and López de Prado formalized the fix as the deflated Sharpe ratio: given how many trials you ran, this is the Sharpe that means anything at all.
The standard complaint about Sharpe deflation is that human researchers lie about N. For instance, a junior quant runs 400 variations, has their pitch deck say 3, and no one can really prove otherwise until it starts losing money.
An RL environment can have none of those problems, but only if you structure it right from the beginning. To do this, the environment must see every experiment the agent ran. Every backtest, every parameter touched, the full trajectory has to be logged.
This way, you can compute the deflated Sharpe exactly, per episode, and pay the reward on THAT.
Making the grader evil enough
Here’s a high-level view of what I’d say is the ideal framework for making this possible:
Hidden holdouts with undisclosed boundaries
The agent sees a train window and a validation window. The grader scores a third period the agent never touches and is never told the edges of, so there’s no boundary to quietly optimize toward.
A continuous, multi-term reward
No “reward if 2+ Sharpe“ targets.
Score out-of-sample Sharpe, drawdown, turnover, transaction costs, factor neutrality, and cross-universe robustness together, smoothly, as single hard-targets like Sharpe are what get hacked.
Randomized episodes
Asset class, region, regime, cost model, liquidity assumptions, all drawn fresh each episode, so the only thing stable enough to learn is the research process.
Deflate, deflate, deflate
The grader counts every experiment in the trajectory and deflates the reward accordingly.
Ten clean experiments that led to a Sharpe of 1.4 outscores 400 experiments that wound up with a 2.1
Robustness probes
After the agent submits, the grader scrambles its parameters, resamples the universe, and stress-tests the costs.
The high-level view loop can be visualized as follows:
What comes out the other side, if it works, is a model whose instincts on a brand-new dataset go like this:
“This looks cross-sectional, so first I’ll characterize coverage and missingness. There’s heavy heteroskedasticity, so I’ll compare equal-weight and risk-normalized books before trusting any signal. Quintile portfolios first; no parameter optimization yet. Interesting, the effect is concentrated in small caps. Impose ADV constraints and retest. Sharpe falls from 2.1 to 1.2, so before abandoning it, check whether it survives in large caps and internationally.”
The model behaves that way because across the 100,000+ graded research problems, behaving that way is what got it rewarded.
THAT’S the version of “AI quant” one should take seriously: a fleet of these, each one a few dollars of compute, pointed at every dataset you own, running the full protocol every single time.
Why this doesn’t exist yet
What’s interesting is that, right now, this is where the actual arms race is, since the RL algorithms are public and the labs are in an outright land rush for training environments. Anthropic reportedly discussed spending over $1 billion on them in a single year, and a dozen startups now sell simulated Salesforce instances and browser sandboxes by the seat.
There’s also academic LLM “alpha mining” agents too, all you have to do is make one GitHub search and you’ll see 50.
However, almost all of them are practically just prompted systems. Clever, yes, but it isn’t actually letting the model self-learn the behaviors needed to come up with novel ideas and strategies.
We just proposed a sample framework to get there, but a few things make it difficult:
The grader is uniquely evil here
A math environment’s verifier is truth itself (e.g., x+y is objectively z). Our verifier, historical performance, is a noisy proxy for the thing we care about, which is future performance.
The environment is made of data
A hundred thousand distinct, bias-free episodes means point-in-time universes, survivorship-aware histories, real cost surfaces, across assets and regimes. You can’t scrape that, and using broken data just scales a broken process.
The first problem has a solution this post just sketched, and the Sharpe deflation math is twelve years old at this point. The second problem is an infrastructure problem, and it’s the one we’ve decided to go after.
If you’re an Alphanume client, you got an email from us recently about an infrastructure upgrade so we can start scaling the quality and breadth of the dataset catalog.
Our mission with Alphanume, since day one, has been to build the best quantitative finance data provider, full stop. We aren’t fully there yet, but the progress behind the scenes is real.
The point-in-time, survivorship-aware catalog we provide to human researchers is, structurally, the same thing a machine researcher’s environment is made of. We just want to be the ones holding the data once someone figures this out at scale.
Final Thoughts
This has objectively been the most interesting rabbit hole I’ve gone down in a while.
The original Situational Awareness paper theorized that in the future, million-deep swarms of research agents would be able to produce novel research to solve problems in biology and other vital fields.
Admittedly, I, like most people, read predictions like that as “huh, yeah maybe.”, but when you actually try running this yourself with current frontier models, you start to think that such a system might not be that far-fetched.
Now, I don’t know whether the first genuinely RL-trained quant research swarm comes out of a frontier lab or some niche options desk. Again, my background is just on the quantitative research side of things, but you can see for yourself that models with frontier-level intelligence are getting substantially cheaper along with the compute necessary to run them.
It’s inevitable that when enough people at least try a framework like this, at least one will genuinely work, for awhile anyway. Heck, there’s even a chance that we’re way behind the curve on this one and it’s already a solved problem somewhere.
This post was a bit different than my typical pure-quant research/experimental style, but I was just so enthused by this space that I felt compelled to write it up. Of course, no one knows exactly what the future will look like, but if (when) these possibilities are realized, it will be very interesting.
As always, thanks for reading, and we’ll see you in the next one.