Article Text
PDF
Abstract
In developing the European Statements of Hospital Pharmacy, a method was required that afforded participants the opportunity to contribute to the wording of the statements, and provided a way to measure the level of agreement. Given the diversity of hospital pharmacy service delivery across Europe, it was envisaged that an explicit, objective and inclusive approach was required to achieve consensus. A two stage approach was therefore designed: a Delphi process followed by a World Café workshop. The sequential use of these two processes in a highly complex context with a requirement for significant international collaboration was highly successful as a consensus building approach. We recommend the sequential use of these two techniques for achieving consensus in similarly complex decision making processes.
- CLINICAL PHARMACY
- Evidence based medicine
- HEALTH SERVICES ADMINISTRATION & MANAGEMENT
- MEDICAL EDUCATION & TRAINING
- PHARMACY MANAGEMENT (ORGANISATION, FINANCIAL)
Statistics from Altmetric.com
- CLINICAL PHARMACY
- Evidence based medicine
- HEALTH SERVICES ADMINISTRATION & MANAGEMENT
- MEDICAL EDUCATION & TRAINING
- PHARMACY MANAGEMENT (ORGANISATION, FINANCIAL)
Introduction
Quantitative methods such as meta-analysis have been developed to improve the statistical agreement of data and improve its validity. This is more difficult when conducting qualitative research or when seeking to produce consensus statements. Consensus methods can help deal with conflicting evidence or views by determining the extent of agreement about a given issue. They can address some of the disadvantages normally found with decision making in groups or committees, which runs the risk of bias due to domination by one or more powerful individuals, or by coalitions representing vested interests. In open committees, individuals are often not ready to retract long-held and publicly stated opinions, even when these have been proved to be false.1
In developing the European Statements of Hospital Pharmacy, a method was required that afforded participants the opportunity to contribute to the wording of the statements, and to provide a method to measure the level of agreement. Given the diversity of hospital pharmacy service delivery across Europe, it was envisaged that an explicit, objective and inclusive approach was required to achieve consensus. A two stage approach was therefore designed: a Delphi process followed by a World Café workshop.
Stage 1: the Delphi process
What is the Delphi method?
The term Delphi originates from Greek mythology, Delphi being the residence of a wise oracle—‘the Delphic oracle’. The Delphi method or process was developed in the early 1950s by Olaf Helmer and Norman Dalkey of the RAND Corporation. It was developed to solicit the views of experts related to national defence and other controversial sociopolitical areas of discourse. The original intent was to enhance strategic planning by improving the accuracy of predictions concerning future trends and priorities. While largely discounted for its original purpose, it remains a useful technique in improving decision making by enhancing consensus or agreement among experts or stakeholders.2
Evidence shows that using the Delphi method enables a structured communication technique to maximise the benefits of group decision making while minimising the negative aspects of group dynamics such as ‘groupthink’.2
How the process is applied
The general principles of the Delphi method involve2:
-
Anonymised collation of responses from participants
-
Oversight of responses by an anonymous facilitator
-
Analysis of responses and objective measurement of agreement
-
Participants being asked to explain a problem or predict a future state of affairs
-
Facilitators controlling interactions among participants by processing the information and filtering out irrelevant content
-
Replies being gathered, summarised, and then fed back to all group members
-
Iteration of the process until the responses converge satisfactorily and an agreed level of consensus is achieved.
To initiate the process, an expert workshop was held where 48 initial statements were drafted, using as a base the 2008 International Pharmaceutical Federation (FIP) ‘Basel’ statements on hospital pharmacy practice.
These statements were categorised under six headings:
-
Introductory statements and governance
-
Selection, procurement and distribution
-
Production and compounding
-
Clinical services
-
Patient safety and quality assurance
-
Education and research.
The Synmind software and platform (http://www.synmind.com/) was utilised for an online, two round, Delphi process. An anonymous facilitator was assigned to each category. European Association of Hospital Pharmacists (EAHP) members together with representatives from patient and other healthcare professional groups were invited to read the draft statements and indicate their level of agreement on a scale of 0–3 (where 0 is strongly disagree and 3 is strongly agree). Participants were also asked to give a justification for their response in the free text boxes under each statement. Round 1 took place in November and December 2013. Statements were revised based on the results of that round and were subjected to further online discussion and voting in round 2 which took place in January and February 2014.
Key results from rounds 1 and 2 of the Delphi process
The level of agreement with the initial draft statements improved during both rounds of the Delphi process. Over the two rounds, the mean score for 25 statements improved by more than 0.1 and no statement had a score which fell by more than 0.1. Sixteen statements remained unchanged from round 1, while 22 statements were unchanged in round 2. During round 1, no statements were removed but one statement moved from section 3 to 4.
In round 2, three statements were amalgamated and section 3 was reordered with one further statement proposed for amalgamation to be discussed at the next stage of the process. Of the 45 revised statements, 33 had an overall score for agreement at the end of the Delphi process of more than 2.75.
Following the second round of the Delphi process, further refinements were made to the statements based on the outcomes of that second round. Those statements then proceeded to the second stage of consensus building—the World Café.
See online supplementary appendix 1 for details of the review of the statements during the Delphi process.
Stage 2: the World Café method
The World Café process3 was devised serendipitously in 1995 by a small group of business and academic leaders in Mill Valley, California. During a group discussion, the 24 participants spontaneously formed into small, intimate table conversations on the issues that had drawn them together. Recording the important points of agreement on makeshift paper tablecloths, they periodically switched tables so the insights and ideas that had real power might circulate, deepen and connect. A record of the important insights enabled them to discern the emerging patterns in their thinking, which then enriched subsequent rounds of conversation. This improvised process produced a collective intelligence that transformed the depth, scope and innovative quality of their collaboration.
The World Café process relies on a number of factors:
-
Set the context: Participants must be clear on the purpose of the meeting in order to consider and choose the most important elements.
-
Create a hospitable space: The environment in which the discussions take place must feel safe and inviting in order to stimulate creative thinking, speaking and listening.
-
Explore questions that matter: Knowledge emerges in response to compelling questions, so facilitators should be prepared to ask questions that are relevant to the most pressing concerns of the group.
-
Encourage everyone’'s contribution: Some participants may be reluctant to contribute while some will be overly keen. Finding the balance is a key component of the facilitator's role.
-
Connect diverse perspectives: The opportunity to move between tables and meet new people thereby linking the conversations to ever-widening circles of thought is one of the distinguishing characteristics of the World Café. As participants carry key ideas or themes to new tables, they exchange perspectives, greatly enriching the possibility for surprising new insights.
-
Listening for patterns and insights: The quality of the listening is perhaps the most important factor determining the success of a Café.
-
Share collective discoveries: The last phase of the World Café is the ‘harvest’ where the collective discussions held throughout the day are discussed in a smaller group to capture the key elements and reach consensus.
In applying these principles at the EAHP Summit in Brussels, six ‘tables’ were created, each devoted to one set of statements. Two facilitators were assigned to each table, plus a note taker. Participants at the summit were each given a unique timetable for the day that meant each of them spent time at each table for 40 min of facilitated discussion on the set of statements for that table. Individuals then moved to a new table with a different group of participants. This enabled each participant to discuss each set of statements while not allowing ‘cliques’ to form or ‘groupthink’ to develop.
Within each group, the set of statements was explored in turn and the facilitators recorded key points of agreement or disagreement. Agreed changes to the statements were written on the tablecloths to inform the ‘Harvest’ discussions in producing revised statements at the end of the World Café. These revised statements were then subject to a vote on day 2 of the summit. See page 267 for details of the voting process.
Key changes to the statements from the World Café workshop
Online supplementary appendix 1 gives details of how the statements evolved throughout the Delphi and World Café stages to produce the draft statements on which participants voted on day 2 of the summit.
Most of the discussions during the World Café involved simplifying the wording of some statements, and improving the flow within groups of statements. This often involved reordering statements within each section so that the important statements were listed first to give them greater emphasis. There was also general agreement around the use of the words ‘must’ where participants felt this was a minimum requirement and ‘should’ where it was recognised that adopting the statement may be currently be aspirational for some countries. A strongly emergent theme, continuing a trend from the Delphi process, was the importance of emphasising collaboration with other healthcare professionals and the role for pharmacists in actively engaging in clinical decision making as part of a multidisciplinary healthcare team, and with patients.
Section 3 on production and compounding was subject to the largest iterative change, continuing another trend in the Delphi process. This was a section which was not included in the 2008 FIP statements, and perhaps the volume of discussion on this section was a reflection of having to devise original statements for the whole section.
The order of the statements in Section 3 was changed to reflect the sequence of how participants felt production and compounding should take place. This finally flowed from encouraging use of commercially available products where available, to outsourcing under the responsibility of a pharmacist, then undertaking risk assessment where production or compounding was deemed appropriate, and finally ensuring quality assurance and hazard prevention measures were in place.
The end product of the Delphi and World Café processes was a revised set of 44 statements that were presented for the final voting stage on day 2 of the Summit.
Discussion
We are unable to find other descriptions of the sequential use of a Delphi process followed by a World Café workshop as a consensus building approach in healthcare. Both processes delivered increasing consensus, and they worked synergistically. No unfavourable comments were made about the Synmind platform utility by participants, and it was favourably received by the Delphi facilitators. The World Café process was nothing less than a delight, with productive and considered input from all participants who were fully engaged throughout the day. Harvesting the end product at the end of the day took several hours, a consideration for others who may consider emulating this approach.
In ideal circumstances the Delphi facilitators would have received input from all participants early in each of the rounds. This would have enabled more discussion between participants and facilitators to explain the positions and views expressed, and between the participants themselves. It is, however, entirely understandable that a number of busy healthcare professionals, often in senior positions within their organisations, were only able to take part close to the end of a Delphi round.
Conclusion
The sequential use of a Delphi process followed by a World Café workshop in a highly complex context with a requirement for significant international collaboration was very successful as a consensus building approach. We recommend the sequential use of these two techniques in achieving consensus in similarly complex decision making processes.
Acknowledgments
We wish to thank all participants and facilitators in both the Delphi and World Café processes for their boundless enthusiasm, commitment and consideration.
Supplementary materials
Supplementary Data
This web only file has been produced by the BMJ Publishing Group from an electronic file supplied by the author(s) and has not been edited for content.
Files in this Data Supplement:
- Data supplement 1 - Online supplement
Footnotes
-
Contributors NM moderated the two rounds of the Delphi process, leading the collation of comments and revisions of statements. JU facilitated one category of statements in the Delphi process and observed the remaining categories as they progressed through the Delphi process. JU suggested the synergistic use of the World Cafe workshop. NM led the team facilitating the World Cafe process, and JU facilitated one category of statements. JU wrote the first draft of this paper, which was then revised by NM. Both authors agreed on the final version of the paper.
-
Competing interests Both authors are employed part-time by the National Institute for Health and Care Excellence.
-
Provenance and peer review Commissioned; internally peer reviewed.