Scientists from Canada, Singapore, and the USA conducted a study in which fifteen different large language models were tested. The purpose of the test was to determine the ability of these models to find methodological flaws in scientific publications concerning clinical trials.
The results showed a significant superiority of the models over human experts. On average, large language models were able to detect methodological errors in the range of 53 to 99 percent. Meanwhile, the percentage of error detection by humans was only about 27 percent.
The most impressive result was demonstrated by the Opus 5.5 model, which was able to identify serious errors with a rate of 99 percent. The data from this scientific study has been officially posted on the preprint portal medRxiv.org.