In the August issue, we introduced you to the paper of Pierre-Antoine Juge et al. about the participation of junior members in EULAR task forces.
In fact, junior scientists are not only involved as members in these task force projects, but also work as fellows, with their main job then being the conduction of systematic literature reviews (SLRs) to inform the task force about all available literature in the field.
While SLRs are the basis of evidence-based recommendations, their conduction is a labour-intensive procedure, often taking fellows weeks and months of tedious work to screen through thousands of abstracts and extract data from tens and hundreds of studies.
Artificial intelligence might now be sufficiently developed to support these review processes and thus save many hours of work, or at least support and structure the review process. Various algorithms have been recently developed and are currently being tested for usability in large international task force projects. Nonetheless, data on how well current algorithms perform are sparse to date. Concerns about the implementation of AI algorithms in SLR process are related to their ability to
- actually, find all relevant articles (or perform at least as well as the conventional team of two fellows performing a manual screening)
- be user-friendly (for example, not requiring coding experience or a long learning period before implementation of the model)
- and be transparent and reproducible.
Algorithms commonly apply active learning techniques, meaning the model learns and improves while the reviewer labels the records as relevant or irrelevant. The remaining articles are repeatedly reordered, relevant abstracts are shown to the reviewer first, while irrelevant articles are downgraded in the stack. Thus, the reviewer doesn’t have to go through all the abstracts but will screen only a pre-defined selection.
A recently developed and continuously evolving software for this purpose is called ASReview, developed by researchers from Utrecht University in the Netherlands. Philipp Bosch and colleagues recently tried the software on an SLR they had performed to inform the 2023 update of the EULAR recommendations for the use of imaging in large vessel vasculitis (Bosch et al., RMD 2024). They found 96% of the included articles while screening only 20% of the original references (unpublished results).
Picture from Boetje and van de Schoot, Systematic Reviews (2024) 13:81
In the next step, the software will now be implemented in the review process of a new, similar task force project on imaging recommendations in polymyalgia rheumatica and will be compared against the manual screening done by two fellows. It will be interesting to see how well the software will perform and whether it can support future generations of fellows by shortening review processes while still finding all relevant literature needed to inform major international task forces and recommendation projects, as commonly performed in EULAR.
Victoria Konzett on behalf of the Newsletter Sub-committe
… with kind inputs and insights into unpublished data from Philipp Bosch (Long member of EMEUNET, and current member of the EULAR Quality of Care committee)
