Rett syndrome: two LLMs that need to reach agreement are analysing the effects of trofinetide across 26,763 clinical notes

At Texas Children’s Hospital, a consensus pipeline combining two language models identifies 18 treatment responses; experts validate 90.2 per cent of its decisions.

Trofinetide, approved in 2023, is the first treatment for Rett syndrome, but its follow-up in real-world conditions is hampered by the cumbersome manual review of files. Researchers have designed a pipeline where two large language models each classify clinical scores and must agree, in order to extract 18 phenotypes of response to treatment: four adverse effects, 13 clinical manifestations of the syndrome and drug refusal. They applied it to 26,763 notes concerning 108 patients at Texas Children's Hospital with long-term exposure to trofinetide.

The architecture and the instruction were developed on a single phenotype, diarrhea, then fixed and applied to the other 17 with only a change in the definition. The best instruction, which requires active and severe diarrhea, achieves 84.2% accuracy, 47 points more than a regular expression search. On a sample of decisions in the pipeline, the experts agreed with the system in 90.2% of cases (92 out of 102). The authors specify that the tool indicates whether a symptom is documented at each visit, without judging whether it is improving, and that it adapts to other drugs by simple configuration.