Critical appraisal tool for reviewing reviews from Critical Appraisal Skills Programme (CASP) Public Health Resource Unit49
| Quality checklist for reviewing reviews | Ludwick 200928 | Greenhalgh 200926 | Ekeland 201057 |
|---|---|---|---|
| Clearly focused question Population, Intervention Outcome | Y Primary care in 7 countries EMR Lessons from implementation | Y – EPR Tensions and paradoxes | Y – Telemedicine effectiveness |
| Right type of study Address the question, appropriate study design | Not clear—inclusion/exclusion criteria and search terms not stated, included studies not listed | Y | Y |
| Answering ‘Y’ to the two screening questions above is an indication to proceed with remaining eight questions | |||
| Finding relevant studies | Conducted in 2008 on articles published between 2000–2007 in Canada, USA, UK, NZ, Australia, Denmark, Sweden. No search terms given but 6 databases, author tracking, broad website searching | Y | Conducted between Feb'09 and Jul'09 on articles published 2005–2009 in comprehensive range of databases |
| Quality assessment | N | Y | Y |
| Combining studies justified | N—included studies not listed, results not displayed, variation not discussed | Y | Y |
| Results | Brief selective, narrative concluding: Summarised as already known: health information systems can help mitigate service demand, which is due to increase further, adoption is hampered by clinician concerns (privacy, patient safety, quality of care, decline in efficiency post implementation), physicians are not proactive in adopting HIS (high costs, risks of liability, data security). Summarised as new findings: HIS do not affect efficiency, quality of care or safety, quality of implementation process is key, risks mitigated with training, bar coding systems, pilots, shared terminology, strong IT management matching usability, computing skills, system fit to organisational culture | Implementation was complex and technically challenging. Subtle, contingent benefits where accessed—individual clinician is main factor in level of use and coping with inaccurate/ incomplete data, inadequate server—supports better quality care, clinician confidence, prevention of medication errors but no evidence of improved safety. Risk to patient privacy. Expect complex interdependencies and tensions (clinical, technical, political, commercial) high implementation workload when on a national level. Impact of change agents and causal influences | Summarised as what was already known: evidence regarding the effectiveness of telemedicine is patchy, quality of research is poor. Summarised as new findings: evidence base of robust knowledge is growing but new knowledge needed, further research required in economic analyses, patient perspectives of effectiveness |
| Precision | Not stated | Not stated | Not stated |
| Applicable locally | Not clear | Y | Y |
| Perspective of outcomes | Not clear | Y | Y |
| Evidence for policy or practice change | N—report methodology is not explicit, content is insubstantive | Y | Y |
| Additional comments | Lack of clarity of method and results | Pragmatic, well-presented meta-narrative review. Discusses philosophical positions (positivist, interpretivist, critical, recursive) identifies research traditions (human computer interaction, evidence based medicine, symbolic interactionism and ethnomethodology, workplace redesign, safety critical systems research) | A pragmatic, well-presented and comprehensive realist review |
| GRADE63 Quality of evidence (magnitude of effect) | ++ (low) | +++ (moderate) | +++ (moderate) |
HIS, Health Information Systems; EPR, Electronic Patient Records; GRADE, Grading of Recommendations, Assessments, Developments and Evaluation.