Tabular data: dimensionality would explain why large models remain behind conventional methods

Marta Garnelo and Wojciech Czarnecki rule out four common explanations and show that a leading model is the only one, on nine methods, whose accuracy drops when the number of dimensions increases.

Why do big language models fail where a random forest of 2001 still does very well? In an article on arXiv on August 3, 2026, Marta Garnelo and Wojciech M. Czarnecki examined five hypotheses by evaluating a state-of-the-art model of pure inference — without refining, without tools, without scaffolding — on 31 sets of reference data, compared to 252 configurations of classical models.

Four common explanations are ruled out: data noise, CSV format limits, number tokenization, and batch size. It remains a discriminant factor, dimensionality. The language model is the only one of the nine methods tested whose accuracy deteriorates as the number of variables increases, while all conventional approaches remain stable or improve.

The observed behavior is revealing: in two dimensions, the model predicts as a local method based on distances, with up to 91.6% agreement on a test grid. Beyond this, no classic model reproduces its predictions. The authors do not claim to explain the mechanism internal to the work, but they clearly define the problem — which is precisely the founding premise of the expanding field of tabular foundation models.