Research

A systematic audit of 15 academic trading-agent papers finds every one carries a security hole and most fail basic market stress

FARSIGHT, a new evaluation framework submitted 17 September 2026, tests published financial LLM trading schemes against flash-crash-like turbulence and three attack categories; 80% fail at least one core robustness metric and 100% exhibit at least one security vulnerability, and the paper shows the same failure can trigger a crash by accident or be forced by an adversary with minimal resources.

· Source: arXiv


This site has spent September tracking two separate threads on agent reliability: Finance Agent v2's independent benchmark capping every frontier model at roughly 61% on realistic analyst work, and VentureBeat's reporting on long-running agents quietly dropping the compliance rules set at session start. FARSIGHT is a third angle on the same underlying problem, applied specifically to the schemes researchers publish as working trading agents rather than to general-purpose models. Mengxiao Wang and Nitesh Saxena built a scheme-level audit framework testing 15 academic financial LLM trading-agent papers on two axes: robustness under market turbulence, including flash-crash-like scenarios, and security against three attack categories, attacks on the information sources an agent reads, attacks on the agent itself, and agent-as-attacker behaviour where a compromised agent is weaponised against the market it trades in.

The numbers are stark for a body of published, peer-reviewed work: 80% of the 15 schemes failed at least one core robustness metric, and 100% carried at least one security vulnerability. The paper's own framing is the detail worth sitting with, that these are not two unrelated failure categories sitting side by side. A minor error in one of these schemes can cascade into a market-wide crash on its own, and the same structural weakness that produces that accidental cascade gives an adversary a route to trigger an equivalent collapse deliberately, with minimal resource investment. That collapses the usual distinction agent builders draw between 'make it robust' and 'make it secure' into effectively the same engineering problem.

The DX Research Group's population-scale study this site covered on 15 September found that a fleet's operating layer, risk sliders and leaderboard mechanics, explained agent behaviour far better than the strategy prompt did, and that sizing stayed blind to volatility across every regime measured. FARSIGHT is auditing a different layer, the published research these production systems draw on, and finds the same pattern one level up: the academic schemes that get cited as proof a given architecture works were not built or tested against the adversarial, reflexive conditions a live market actually produces. For a reader treating a trading-agent paper's backtest as evidence of production readiness, this audit is a direct answer: on this sample, published academic validation and demonstrated robustness under stress are two different claims, and only one of them was being checked.


Read the original: arXiv - A systematic audit of 15 academic trading-agent papers finds every one carries a security hole and most fail basic market stress. Commentary is the independent editorial view of Share Trading; the original article is credited to its publisher.