Beyond the simple mention, the tool classifies the actual use of an observatory's data over six levels. It reaches 89.1% accuracy and is already operating at China's national astronomical data centre.
Observatories keep bibliographies to justify their usefulness, but counting the articles that cite an instrument does not say whether its data have actually been used. FacilityAudit-Agent offers a six-level contribution scale, from L0 (no use) to L5 (proprietary data), including simple mention and auxiliary or major support.
The tool builds a plan of each PDF, annotates the sections with their role and the instruments cited, and then lets a language model with tools fetch the necessary text. The collected elements are combined by an explicit decision tree and a confidence score; uncertain cases are returned to a human. Out of 39,358 hand-annotated article-instrument pairs, it achieves 89.1% accuracy, 5.8 to 12.1 points better than three reference methods, and adapts to new instruments without retraining.
The system is already deployed at China's National Astronomical Data Center, where it processes publications related to the LAMOST spectroscopic telescope, the FAST radio telescope and the Einstein Probe satellite.
Read the full article on the original source
Go to the source →