AI-assisted full-text screening led to a 38% timesaving compared to manual screening

Title: AI-assisted full-text screening led to a 38% timesaving compared to manual screening

Global Evidence Summit 2024 Poster

Authors: Mhd Amin Alzabibi, Muayad Azzam, Jana Khawandi, Hassan Kawtharany, Jamil Nazzal, Sahar Alhussei, Mohammad Fraitekh, Mustafa Qadir, Artur Nowak, Ewa Borowiack, Ewelina Sadowska, Reem A. Mustafa

‍

Background

The poster discusses the potential of Artificial Intelligence (AI) and machine learning to enhance the efficiency of systematic reviews. It compares the accuracy and efficiency of full-text screening assisted by AI-generated suggestions and text highlights to manual screening, using systematic reviews that informed a diagnostic test accuracy guideline by the American Society of Hematology.

Methods

The study involved title and abstract screening of 28,142 papers and full-text screening of 5,060 papers. A sample of 500 studies was selected for both manual screening and AI-assisted screening. The average time spent screening per report by a single reviewer was measured for both methods.

Results

Compared to manual screening, AI-assisted screening had a sensitivity of 89.2% and a specificity of 46.7%. AI-assisted screening led to a 38% time saving, equivalent to 69 seconds per study. Users reported positive feedback and noted facilitation despite some initial challenges. The average time spent screening per report was 180.42 seconds for manual screening and 111.54 seconds for AI-assisted screening.