Problems of poor reporting of health research

Many research studies have identified serious reporting shortcomings in published health research. These problems are widespread and range from the failure to report and publish whole research studies depending on the nature of their research findings, publication bias',3 to the selective publication (or non-publication) of specific study outcomes ‘selective outcome reporting bias’.4 Even when all study outcomes are reported, the interpretation of the study results is sometimes distorted by the authors of the primary study.5 Other problems in the reporting of health research literature include the inadequate and misleading reporting of adverse events,6 the omission of information about study methods which would allow implementation of the intervention in clinical practice,7 and the omissions from, or misinterpretation of, results in the abstracts of study publications.8 Serious deficiencies in the reporting of research have been well documented across many medical specialties and various study designs.

Systematic reviews of the health research literature are widely acknowledged as the best way to evaluate all the evidence relevant to a particular healthcare issue. Reviewers need to understand exactly how the original studies were carried out. However, this assessment is frequently severely hampered by inadequate reporting of the primary studies. If research is not well reported it is difficult for clinicians to take best evidence and translate it into best practice. In some cases inadequate research reporting can have detrimental effects on patient care. Whittington and colleagues9 carried out a systematic review of published and unpublished data on the risks and benefits of selective serotonin reuptake inhibitors (SSRIs) in the treatment of depression in children. Published data suggested a favourable risk–benefit profile for some SSRIs; however, the addition of unpublished data indicated that the risks could outweigh the benefits of these drugs (except fluoxetine) to treating depression in children and young people. This situation could have been avoided by the adoption of a more responsible and transparent attitude to the reporting of clinical research.

Over the last decade or more there have been considerable efforts to try to improve the quality of reporting of individual research studies. One of the key developments has been the introduction of reporting guidelines which provide structured advice on what minimum information needs to be included in a research article to allow readers to assess the study methodology, relevance and validity of the research findings. Most of the internationally recognised reporting guidelines reflect the consensus opinion of experts in a particular field, including methodologists and journal editors and also draw on relevant empirical evidence to support the inclusion of particular checklist items.10 In this article we introduce three of the most commonly used reporting guidelines: the Consolidated Standards for Reporting Trials (CONSORT), Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) and Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) Statements.

Reporting of randomised ­controlled trials

In 1993 an international group of researchers and medical journal editors met in Ottawa, Canada to develop a way of improving the reporting of randomised trials. The Standardised Reporting of Trials (SORT) guidelines were published in 1994 as were similar recommendations by another group. Additional meetings led to a ‘consolidated’ guideline in 1996, known as the CONSORT Statement.11 Subsequent research and further meetings led to a revised CONSORT Statement in 2001,12 which was, without precedent, published simultaneously in three prestigious international medical journals. In 2001 the group also published a detailed explanation and elaboration document to accompany the Statement.13 This publication was recognised as an important innovation and the idea has since been taken up by other guidelines groups. The CONSORT Statement was last updated in 2010,14 and was published simultaneously in 10 leading medical journals and the accompanying revised and expanded explanatory document15 was also published in two leading journals. The CONSORT 2010 Statement has already been translated into three languages and at least eight more are currently in progress.

The CONSORT Statement (http://www.consort-statement.org) comprises a checklist (table 1) of essential (minimum) items that should be included in reports of randomised trials and a diagram (figure 1) for documenting the flow of participants through a trial. It is aimed at primary reports of randomised trials with two-group parallel designs. However, most of CONSORT is also relevant to a wider class of trial designs, such as non-inferiority, equivalence, factorial, cluster and crossover trials. Extensions to the CONSORT checklist for reporting trials with some of these designs have been published,16 ,17 ,18 as have those for reporting harms,19 types of interventions (non-pharmacological treatments,20 herbal interventions)21 and abstracts.8 CONSORT is an ongoing initiative and as the evidence base to inform CONSORT continues to grow, the Statement will be updated to reflect the new evidence and changes in opinion and the research environment.

• Download figure
• Open in new tab
• Download powerpoint

Figure 1

Flow diagram of the process through the phases of a parallel randomised trial of two groups (ie, enrolment, intervention allocation, follow-up and data analysis).14 ,15

Early efforts to improve the reporting of randomised trials were not particularly successful in changing practice (ie, improving the quality of reporting) or in influencing the policies of journals. The original SORT statement, developed by methodologists and trialists, met with only limited success even though it was methodologically sound. The CONSORT Statement has been more successful, which is likely to be due to many factors. High on the list are its international membership, including clinical trialists, methodologists and editors of influential medical journals.

Reporting of systematic reviews and meta-analyses

A number of other related initiatives have since followed a similar model to that of the CONSORT Statement, with the aim of improving the quality of reporting of other types of health research. The Quality of Reporting of Meta-analyses (QUOROM) Statement, developed in 1996 and published in 1999,22 was conceived as reporting guidance for authors reporting meta-analyses of randomised trials. In 2009 the QUOROM Statement was substantially revised and replaced by the PRISMA Statement as a guideline for facilitating the reporting of systematic reviews that assess the benefits and harms of health interventions. The PRISMA Statement23 (http://www.prisma-statement.org) consists of a 27-item checklist and flow diagram for documenting the number of studies identified, included and excluded from a systematic review. PRISMA was published in nine leading medical journals together with the accompanying explanatory document, which provides examples of good reporting and the rationale for required checklist items. While the primary focus is on systematic reviews of randomised trials, PRISMA can also be used as a basis for reporting systematic reviews of other types of research, for example, evaluations of non-randomised study interventions.

Reporting of observational ­studies

Observational studies represent a large proportion of published health research. Much of the research into the cause of diseases relies on cohort, case–control, or cross-sectional studies. Observational studies also have a role in research into the benefits and harms of medical interventions which cannot be addressed by randomised trials, for example, in detecting rare or late adverse effects of treatments.24 The STROBE Statement was developed in 2004 and published in 2008 to provide recommendations on what should be included when publishing a report of an observational study. It provides recommendations for three main study designs: cohort, case–control and cross-sectional studies. The STROBE Statement24 (http://www.strobe-statement.org) consists of a 22-item checklist, of which 18 items are common to all three study designs and 4 are design specific. As with CONSORT and PRISMA, the STROBE Statement has been published in a number of journals and also has an accompanying explanatory document providing examples of good reporting and the rationale and importance of each checklist item.

Impact of reporting guidelines

The introduction of reporting guidelines has seen considerable international recognition. Perhaps the most influential has been the CONSORT Statement which has received support from the World Association of Medical Editors, the Council of Science Editors, the International Committee of Medical Journal Editors and over 600 journals worldwide. However, despite this endorsement, only 38% of 165 leading medical journals explicitly recommend or require authors to adhere to CONSORT when preparing articles for submission.25 A number of studies have examined the impact of the CONSORT Statement on the reporting quality of published randomised trials. A systematic review involving 50 quasi-experimental studies, containing 16 222 reports of randomised trials, showed that journal adoption of the checklist was associated with increases in reporting quality.26 For example, one such study compared the quality of reports of randomised trials indexed in PubMed in 2000 (n = 519) and 2006 (n = 616) and assessed whether the quality of reporting had improved after publication of the CONSORT Statement in 2001. The findings demonstrated an increase in the proportion of trial reports that included details of the primary outcome (risk ratio (RR) 1.18, 95% CI 1.04 to 1.33), sample size calculation (RR 1.66, 95% CI 1.40 to 1.95), and the methods of random sequence generation (RR 1.62, 95% CI 1.32 to 1.97) and allocation concealment (RR 1.40, 95% CI 1.11 to 1.76) after publication of the 2001 CONSORT Statement. There was no difference in the proportion of trials that provided specific details on who was blinded (RR 0.91, 95% CI 0.75 to 1.10).27 Although methodologically challenging, similar studies are needed to evaluate the impact of using the PRISMA and STROBE Statements over time.

Reporting of other types of health research

The Enhancing the Quality and Transparency of Health Research (EQUATOR) Network (http://www.equator-network.org) is an international initiative that works to improve the reliability and value of medical research literature by promoting accurate and transparent reporting of health research studies. EQUATOR is unique in that it brings together all parties involved in research publication, including researchers, journal editors, publishers and scientists involved in developing reporting guidelines, educators and research funders.10 In addition to the three reporting guidelines mentioned here, the EQUATOR website provides an extensive catalogue of other reporting guidelines and other resources to support and encourage high standards in research publications. Examples of other key reporting guidelines are listed in table 2.

The clear, transparent and timely reporting of research studies is an integral part of responsible research conduct. Adherence to reporting guidelines such as the CONSORT Statement for reporting randomised trials should improve the accuracy and completeness of research reporting and facilitate its better interpretation. This will lead to widespread incremental improvements in the quality of the information available to clinicians, patients and policy makers.