Rapid evidence publication in the COVID-19 era: impact on study quality and study identification

Wright C, Misso K, Green R, Marjenberg Z. Rapid evidence publication in the COVID-19 era: impact on study quality and study identification (SA43). Poster presented at ISPOR Europe; 6-9th November 2022; Vienna: Austria, 2022

Objectives

The COVID-19 pandemic has created a rapidly changing research evidence dissemination landscape. Our aims were to assess database yield to evaluate the optimal source combination and contribution of preprint servers in a systematic review (SR) of post-COVID syndrome. Additionally, we examined whether the recent surge in the publication of preprints and non-peer reviewed research has impacted study quality. 

Methods

Six databases were searched to identify eligible studies for a SR of post-COVID syndrome. Unique contribution (%), precision, recall, proportion of preprints, and numbers needed to read (NNR) were calculated for each database, as was the optimal combination of databases to identify all the included studies. Quality assessment of included studies was carried out using a modified Newcastle-Ottawa scale. 

Results 

Of 37 included studies, 11 were preprints at searching (03/22), and 8 remained preprints at data extraction (06/22). Embase retrieved 70% of the included studies, whereas Medline Daily Update/Epubs Ahead-of-Print and Europe PMC retrieved the lowest yield (9%). WHO COVID-19 database contributed the greatest number of unique references (n=9) and all preprints (n=11). The most effective combination of databases was Embase and WHO COVID-19 with 96% recall (NNR 95). Adding handsearching increased recall to 100%. Overall mean study quality was lower in the preprints than the accepted publications, however this difference was not significant (p=0.078).

Conclusion

Study results published during the COVID-19 pandemic were used to inform government policy in many countries. The rapid availability of preprints via non-profit online servers, at the expense of peer review, became the norm. Preprint servers make a valuable contribution to SRs of post-COVID symptoms. Whilst WHO COVID-19 database provided the most comprehensive results, handsearching and reference checking continue to play an important role in evidence identification. We found there was no significant difference in study quality between preprints versus peer-reviewed articles.