The journal Bandolier1 asked three questions as part of its title logo: What do we think? What do we know? What can we prove?

All three questions are important, and the role of evidence-based practice is to move us on from the first through the second to what we can prove. Much of what happens in healthcare is based on what we think. The opinions we own are often derived from our mentors, our own reading or experience, and may be coloured by a bad experience where a mistake was made. One of us recalls a patient in a wheelchair who was given a nonsteroidal anti-inflammatory drug (NSAID) for muscle pain. His kidneys shut down and he required dialysis for several weeks. That experience raises caution when thinking about paraplegics and NSAIDs!

Asking the right questions

General questions are about knowledge to understand a subject better to gain knowledge foundation to provide clinical care. Mostly, these can be answered by general texts and are outside the scope of this chapter. Specific questions seek knowledge to make sound clinical decisions, usually for a particular patient case or to devise treatment guidelines.

The following table gives some comparison of different types of questions.

General questions are useful for pharmacy students or trainee pharmacists to learn broad knowledge about topics, or for gaining an understanding of a new clinical area. Practicing clinical pharmacists directly involved in patient care should ask specific questions to answer queries and resolve a patient's problem. Question formulation is a key skill required for practising EBM.

Questions need to be directly relevant to a patient's problem or defined issue, and formed to help a search to find useful answers.

Many EBM practitioners use the acronym PICO which stands for P (patients or problem) I (intervention), C (comparison–if relevant) and O (outcomes). Taking a clinical question such as ‘is gabapentin effective in neuropathic pain?’ The PICO may look like this:

P. Adults with neuropathic pain of at least 12 weeks duration who report their pain as moderate or worse

I. Gabapentin given in any dose for any length of time

C. Other treatments for neuropathic pain or possibly placebo

O. Patient-reported pain relief using validated scales for either pain intensity or pain relief recorded over time. Adverse effects would also be part of the outcome assessment.

There are a few variations to PICO. Some add ‘S’ for studies to define the study type being sought. In the example above, randomised controlled trials would be a good study type. Others add T for time, so again using the example above, it would be sensible to assess the effect of gabapentin over a minimum of 8 weeks as this is a chronic condition.

The following shows how questions can be formulated for treatments, prognosis, diagnosis or causation, for example, for adverse effects. Adapted from Melnyk and Fineout-Overholt.2

Treatment: In__________(P), how does __________(I) compared to__________(C) affect __________(O) within __________(T)?

Prognosis/Prediction: In __________(P) how does __________(I) compared to _________(C) influence/predict__________(O) over__________(T)?

Diagnosis or Diagnostic Test: In__________(P) are/is __________(I) compared with __________(C) more accurate in diagnosing __________(O)?

Aetiology/Causation: Are __________(P) who have __________(I) compared with those without __________(C) at __________risk for/of __________(O) over __________(T)?

Critically appraised topics

A critically appraised topic (CAT) is a short summary of evidence on a topic of interest, usually focussed around a clinical question. The Centre for Evidence-based medicine website has some helpful advice and even a piece of software to help you develop a CAT.3 CATs are designed to help you develop a clinical answer without having to embark on a full systematic review. They can be a very useful way of developing issues that come up in clinical pharmacy work or medicines information, and many develop them into deposit of CATs sometimes called a CAT bank.

Gathering knowledge

Many maintain a file (paper or electronic) of useful articles gathered over a period of time. These can be a useful starting point but suffer from various shortcomings. The file will certainly be incomplete, what is in the file has probably come from at least some biased sources, and the information is likely to be pretty random in terms of its accumulation. At the very least, clinical pharmacists should be able to search for evidence quickly and efficiently. They should also be familiar with databases focusing on systematic reviews (eg, Cochrane), international, national and local guidelines and recommendations, as well as databases to identify individual studies and other publications (eg, PubMed). Google Scholar is now also becoming a useful resource.

MEDLINE

Some pharmacists will regularly acquire information by use of one or more bibliographic databases, but many suspect that this facility is only available to specialist librarians or is the responsibility of medicines information units. Knowledge can be a powerful tool in building clinical pharmacy credibility. Entrez PubMed (PubMed) can be accessed without charge for anyone with internet access. MEDLINE (the parent database of PubMed) refers to the electronic version of Index Medicus, International Nursing Index and Index to Dental Literature. This version is often available on subscription to academic libraries. While PubMed is extremely useful, users need to be aware that PubMed builds in a lot of assumptions into any search which are not always appreciated by the experts in knowledge management or understood by the casual user. It is well worth spending time getting to know PubMed—there is a free training link and also some useful features such as identifying full text articles that are free of any charges.

The following may help those who are not familiar with using MEDLINE or other databases.

Other databases

Although MEDLINE/PubMed is pretty comprehensive for clinical enquiries, it has a strong US bias and may not be the best source of references when looking for wider topics or other aspects of healthcare. There are other more appropriate databases for different areas. CINAHL (Cumulative Index to Nursing and Allied Literature) for nursing and the allied professions has some journals relevant to pharmacy and clinical pharmacy services. EMBASE (electronic version of Excerpta Medica) is a medical database which has a better coverage of European journals; it also includes more references to drugs and therapeutics. International Pharmaceutical Abstracts covers a wide range of pharmacy and pharmacology, but is not easy to access outside the USA. There is a great deal of overlap between these databases, but as no single one can deliver a complete picture, several often need to be used.

Searching skills

The most important stage of any search is deciding and defining exactly what needs to be found—it is important, therefore, to avoid just typing in the first word that comes to mind. This is where the PICO can really help. The query needs to be formulated in terms of a question and then broken down into the component concepts. This helps to clarify the search and improve the yield of the search. Medical librarians are a great resource to help you develop skills if you cannot access these through pharmacy colleagues. The following is a brief introduction to some important issues around searching.

‘Boolean operators’

Although all commercial providers of MEDLINE cater for `natural language’ queries to a greater or lesser extent, a more effective search can be carried out by using the MeSH terms that have been selected, and combining these with the logical connectors OR and AND. Thus, in undertaking a search around nutrition of the elderly, the terms that would need to be chosen are nutrition and the related terms diet, food, food habits. By combining these terms with the connector OR, all the papers indexed under nutrition as well as all those that have been indexed under diet and food and food habits will be identified. It is then important to limit the search to only those papers which concern old people by using the connector AND, and the term ‘aged’. In some senses, the terms are counter-intuitive. ‘OR’ increase the numbers of hits and ‘AND’ reduces the numbers of hits.

Using searching for finding evidence at the appropriate level

The evidence-based world seems to be one where information is often readily shared thus making life considerably easier for everyone. Methodological search filters are available to ensure that the hit rate is the best possible when searching for different types of evidence.

The range of filters covers:

• Systematic review search strategies

• Randomised controlled trial search strategies

• Diagnosis methodological filter

• Prognosis methodological filter

• Therapy methodological filter

• Aetiology, causation or harm methodological filter

• Guidelines methodological filter

• Treatment outcome methodological filter

• Evidence-based healthcare methods filter.

An internet search will identify relevant filters hosted on many university websites.

Other useful sources

The Cochrane Library4 is now a valuable source of knowledge with some 6000 systematic reviews and over 700 000 randomised controlled trials. Additionally, there 26 000 summaries of other systematic reviews from the DARE database (Database of reviews of Effectiveness) produced by the NHS Centre for Reviews and Dissemination in York (numbers accurate at end of 2013). All countries can access the Library and see abstracts of Cochrane reviews, however, full access is only available where national or local library licences exist. At the time of writing, this covered the UK, some parts of Europe, some parts of the USA and developing countries under the Hinari initiative.

Dealing with a mountain of evidence

In Chapter 2, there was a hierarchy of evidence presented. When seeking to answer a clinical question, then it makes sense to look for synthesised evidence such as in a systematic review. This should give a useful appraisal of relevant evidence. We are in a situation where even systematic reviews are being produced in incredible numbers. A paper by Bastian et al5 suggest there are 11 new systematic reviews published every day on average, along with some 75 new randomised controlled (or clinical) trials (RCTs). The authors argue that this growth has not yet reached a steady state.

The development of the overview of systematic reviews

One response to these large numbers of systematic reviews has been the development of the overview of reviews. This is an attempt to describe all systematic reviews relevant to a single topic or intervention aimed at helping practitioners and commissioners to make high-quality decisions about healthcare. For an example, see ‘Single dose oral analgesics for acute postoperative pain in adults’ by Moore et al.6

Understanding of how to present data in the most helpful way is emerging, as is the development of so-called network meta-analyses. Network meta-analyses are beyond the scope of this chapter, but pharmacists need to be aware of this complex statistical analysis and of the potential benefits and pitfalls.

The rise of the systematic review

Reviews have been popular for a long time because, like fast food, they provide instant satiation of a need. Equally, like fast food, they can be of extremely variable quality.

Systematic reviews need to be appraised before applying the results to practice as they can be misleading for a number of reasons:

• They may represent only a proportion of the literature on a subject.

This is often because the search for articles may have been inadequate; these include poor search terms, a failure to use a wide range of databases, by restricting the search to English language or to a short chronological time period.

• Data is abstracted in a subjective way with no assessment of the quality of the data source.

• Poor analytical techniques may have been used to abstract any data, or worse, no attempt is made to bring data together.

The systematic review should exhibit the following characteristics:

• A clearly defined search strategy, outlining the search terms, the breadth of the search, the databases searched, and the date of the last search.

• Follow-up of relevant cited articles in identified papers or any hand searching of other relevant literature.

• Explicit criteria to evaluate the quality of the papers reviewed. Various scoring systems exist, however, methods that are readily reproducible across reviewers are more difficult.

• A description of how the findings were analysed using validated methods.

Tools exist to assess the quality of included studies in a systematic review. For an example, see the Cochrane risk of bias tool.7

A systematic review should seek to bring together the world literature on a subject irrespective of language of publication or date of publication.

Systematic reviews of randomised controlled trials provide the highest level of evidence of efficacy of treatments—though in other circumstances, such as adverse events or diagnostic tests or service reconfigurations, randomised trials may not always provide the best evidence.

Output from systematic reviews

The evidence provided in systematic reviews can take various forms. Often it is statistical—an OR, relative risk, HR or effect size. These show statistical superiority of one treatment over another, or over no treatment, but it is often difficult to relate them to clinical practice.

Because of this, the concept of the number-needed-to-treat is increasingly being used as a useful way of looking at results of reviews or trials for at least two reasons. It is easy to calculate, and provides the treatment-specific result in a form that is understandable.

Meta-analysis

Meta-analysis is a technique used to pool the data reported in clinical trials. This method has several advantages.

It illustrates the overall pooled effect of a number of trials that individually may show a trend that is inconclusive. The meta-analysis weights the trial size to effect (based on the variance) so that the pooled result gives a clear indication of whether an intervention is effective or not. A section on forest plots is available in Chapter 4.

A pooled analysis has the effect of increasing the precision of the conclusions of a systematic review. It does this by making comparisons between several studies more objective, and provides a means of dealing with results that seem to conflict.

All this assumes that systematic reviews and meta-analyses are conducted using rigorous scientific techniques, otherwise bias can be introduced at the trial selection stage.

The meta-analysis relies on trials that have been published. The issue of unpublished trials is often raised, arguing that negative trials often remain unpublished, and that if these could be included they would somehow reverse the answer delivered by the meta-analysis. A recent amnesty on unpublished data failed to produce any useful answers to this, but in many cases the amount of unpublished data would need to be enormous to counteract much of the existing evidence that has undergone careful meta-analytic examination. Authors of good meta-analyses have normally made attempts to identify any unpublished material, however the task is often time consuming and largely without reward. This issue is further considered in a section on funnel plots in Chapter 4.

Why controlled and randomised trials?

Performing and evaluating medical research needs a high level of skills. The fundamentals to consider are:

• is it biologically plausible?

• does it make sense?

• is it reproducible?

• is there a time association? and

• what is the strength of the association?

The strength of the association can be statistical and/or clinical significance. But beware of using statistical significance where there is little clinical benefit. For example, a large study might show a very small decrease in blood pressure from a particular antihypertensive; however, the reduction may not be of any clinical significance.

Dose-response associations and of design is of major importance for the evaluation.

A descriptive study (case report, case series, ecological studies) and an observational study (case-control, cohort) can observe an association, but normally not as a causality. For this, a controlled, randomised and blind study is needed, and the problems of associations, as described in figure X, can be minimised.

The RCT is considered the most reliable way to assess the effect of an intervention. The principle of randomisation is simply that a subject has an equal probability of being assigned to any particular group within the study and that allocation to the group is purely chance. Randomisation eliminates selection bias by removing any influence of the investigators to assign a subject to one group or another. It is important that the likelihood of being assigned to one group or another purely by chance is maintained. This could be achieved by tossing of a coin for example, but is more normally delivered by either random number tables or computer generated randomisation. Any feature that predetermines which group a subject is assigned to should be avoided. These include the use of hospital numbers; date of birth or even the order patients were seen in the clinic list.

RCTs by their nature require that the question being asked is at a state of equipoise, that is, of two or more treatments it is not known whether one is better than the other and if there is a difference, which is better. It is not ethical to assign patients to treatment arms that are known to be inferior.

In addition to randomisation, bias is further eliminated by the use of blinding or masking of treatments and of the outcomes assessment. The standard approach is to double blind interventions so that neither the investigator nor the subject knows which treatment the subject is receiving. It is important, therefore, that treatments are identical in appearance. Where treatments are physically different then a double-dummy technique is also required. For example, in a comparison between an injectable and an oral treatment, subjects would receive either an active injection with an oral placebo or a placebo injection with an active oral treatment. It has been shown that RCTs that do not use a double-blind design are more likely to overestimate the results, similarly, a lack of blinding can cause the observer recording outcomes to overestimate or underestimate the effect of the treatment group based on preconceived ideas about that treatment.8 The precision of the RCT increases as the number of patients in the trial increases. In fact it is likely that studies with small samples may miss an important clinical difference that actually does exist equally. These studies may, by chance, report a difference which, in fact, is not true.

Summary

Dealing with clinical queries has to be a key skill for pharmacists. Defining the question using a tool, such as the PICO facilitates searching for relevant evidence and should lead to a more reliable answer. All pharmacists should have some expertise in searching for suitable evidence. For some, this skill should be developed to an advanced level. Finally, an understanding of levels of evidence and of related evidence products should streamline any search for suitable evidence and the identification of suitable evidence, if there is any.

Footnotes

• Phil Wiffen is editor-in-chief of EJHP and also teaches methodology for Evidence Based Medicine and systematic reviews.

• Tommy Eriksson is Professor in Clinical Pharmacy and Program Director of the MSc pharmacy programme at Lund University in Sweden.

• Hao Lu is a clinical pharmacist based at the Beijing United Family Hospital in China.