A team identifies 14 types of drifts and detects 3,964 on 300 tracks, and then reduces those remaining at the end of the task by 36.8%.
The number of agents capable of conducting a large part of a research project alone is increasing, but their rigour leaves much to be desired. Researchers propose RIDAR, a framework for identifying and correcting breaches of scientific integrity committed by these agents: non-reproducible results, an approach that is unfaithful to what is announced, and validity compromised.
They first established a taxonomy of 14 types of breaches covering the entire research cycle. Their a posteriori detector achieves an F1 score of 85.2% out of 60 trajectories annotated by experts. Applied to 300 trajectories of agents on 50 different research tasks, it identifies 3,964 breaches, present in 99.7% of cases: almost no execution is exempt from them.
Intervene during execution
A second version of the tool monitors the agent along the way and corrects him or her as it happens. It reduces the number of shortcomings still present at the end of the task by 36.8%. The authors announce the publication of the code and data.
Read the full article on the original source
Go to the source →