Research
Elm Wealth's Crystal Ball Challenge gave Claude, ChatGPT, Gemini and Grok real historical Wall Street Journal front pages and withheld the actual market outcomes, finding Claude beat human participants in 76 percent of roughly 200 sessions
ChatGPT beat human participants in 63 percent of sessions in the same study, but researchers found all four models systematically took on more risk relative to the information they were given than the trade context justified. The study is one of the more rigorous public attempts to isolate how large language models actually behave when asked to trade on real historical information rather than a live, backtested feed.
Most AI-trading research this site has covered either backtests a model against live market data, which risks contamination from training-data leakage, or asks a model to trade hypothetically without any real stakes. Elm Wealth's design sidesteps both problems in a genuinely clever way: give the model an actual historical Wall Street Journal front page, the real market-moving information investors had in hand on that specific day, but withhold what the market actually did next, then score the simulated trade against the real outcome once it is revealed.
Claude's 76 percent win rate against human participants across roughly 200 sessions is a meaningfully strong result for a model reading static news text rather than structured market data, and ChatGPT's 63 percent shows the pattern held, though less strongly, across model families. That both models beat human traders on the same information set is the more interesting finding than either number alone, it suggests the edge came from something closer to disciplined information processing than from a single model's idiosyncratic strength.
The caveat researchers flagged matters as much as the headline win rate: all four models tested generally took on more risk than the trade context justified, sizing positions or committing capital more aggressively than the available information supported. That is precisely the failure mode this site has warned about in coverage of Situational Awareness's forced unwind, a model or a fund built around a model can be directionally right more often than a human counterpart while still being poorly calibrated on how much conviction a given signal actually earns.
For readers weighing whether to trust an AI assistant with real capital, the practical read-through is that raw win rate is not the same question as risk-adjusted return, and Elm Wealth's own framing treats the excess-risk finding as the more important result. A model that wins more often but sizes positions too aggressively can still underperform, or blow up, relative to a human trader with worse hit rates but tighter risk discipline, which is exactly the gap that platforms like Scalable Capital's new Agentic Investing feature will need to manage as real customer capital starts flowing through these same model families.
Read the original: Elm Wealth - Elm Wealth's Crystal Ball Challenge gave Claude, ChatGPT, Gemini and Grok real historical Wall Street Journal front pages and withheld the actual market outcomes, finding Claude beat human participants in 76 percent of roughly 200 sessions. Commentary is the independent editorial view of Share Trading; the original article is credited to its publisher.