How important are comprehensive literature searches and the assessment of trial quality in systematic reviews? Egger M et al. Health Technol Assess. 2003;7(1):1-76.
Over ten years old but still a really important paper. They report a number of important findings, that could form part of a list of heuristics, for instance:
- The importance of trials that are difficult to find vary by speciality.
- Unpublished trials show less beneficial effects than published trials.
- Trials that are difficult to locate tend to be smaller and of lower methodological quality than those that are easier to find and published in English.
The conclude starts with this passage:
“Systematic reviews that are based on a search of English language literature that is accessible in the major bibliographic databases will often produce results that are close to those obtained from reviews based on more comprehensive searches that are free of language restrictions.”
They also raise the prospect that extensive searches could introduce bias by the inclusion of poorer quality trials. They later state:
“…We believe that in situations where resources are limited, thorough quality assessments should take precedent over extensive literature searches and translations of articles”
The important take home message for me is that sometimes ‘less is more’. By focussing on easily accessible studies could actually mean better results. So, can rapid reviews – which focus on a single bibliographic database (perhaps two) – actually produce better results?