Twenty graph models flattened on toxicity prediction: published comparisons did not stand

Data sets, pre-treatments and evaluation protocols made the announced performance inoperable. A unified framework rejudges more than 20 approaches on equal terms.

Failure due to toxicity remains one of the main causes of abandonment of a drug candidate in both preclinical and clinical settings. Hence the interest in deep graph learning, which works directly on molecular representation and is more expressive than conventional approaches to predicting chemical toxicity.

The problem raised by this work is not the method but its evaluation. Comparisons available in the literature are difficult to interpret: each publication retains its own data sets, its own pre-processing chain and its own evaluation protocol, making the figures announced immeasurable between them.

The authors propose a unified and standardized frame of reference, and re-evaluate over twenty representative approaches under constant experimental conditions, on several data sets and several partitioning strategies. The exercise is complemented by a structured analysis of the literature, designed to put methodological trends and claimed performance in context. The framework is published in free software, so that the community can replay comparisons to the same.