Article Text

Download PDFPDF
Original article
Strategy for development and pre-implementation validation of effective clinical decision support
  1. Anne-Marie J Scheepers-Hoeks1,
  2. Rene J Grouls1,
  3. Cees Neef2,
  4. Eric W Ackerman1,
  5. Erik H Korsten3
  1. 1Department of Pharmacy, Catharina Hospital, Eindhoven, The Netherlands
  2. 2Department of Clinical Pharmacy and Toxicology, Maastricht University Medical Center, CAPHRI, Maastricht, The Netherlands
  3. 3Department of Anaesthesiology, Intensive Care and Pain Relief, Catharina Hospital, Eindhoven, The Netherlands
  1. Correspondence to Anne-Marie J Scheepers-Hoeks, Department of Pharmacy, Catharina Hospital Eindhoven, Post office 1350, Eindhoven 5602 ZA, The Netherlands; anne-marie.scheepers{at}


Objective Well-designed clinical decision support systems (CDSS) can reduce the problem of alert fatigue by generating patient-specific alerts. This paper describes a strategy for the development and pre-implementation validation of specific and relevant clinical rules in order to reduce alert fatigue.

Methods A four-step development and validation strategy of clinical rules is presented. As an example, from March to September 2006 the ‘lithium therapy rule’ was developed with this strategy based on the Plan-Do-Check-Act cycle. 15 368 patients were retrospectively screened and 2503 patients were prospectively screened while the positive and negative predictive values (PPV/NPV) were continuously monitored. The first step is to confirm that the parameters used in the definitions are linked to the correct data in the electronic health record; the second step involves an expert team in the review process to assure that alerts generated are clinically relevant; in the third step the rule is adjusted to generate the right alerts in daily practice; and the fourth step ensures technical and therapeutic maintenance after implementation in practice.

Results From September 2006 to July 2010 nine other rules were developed following exactly the same strategy. The 10 clinical rules developed showed a progression during the development and all resulted in a final therapeutic PPV of ≥89% before implementation, based on expert opinion. NPV was determined for five clinical rules and was always 100%.

Conclusions The proposed strategy is effective for creating specific and reliable clinical rules that generate relevant recommendations. The inclusion of an expert team in the development process is an essential success factor. It is hoped that it will accelerate the widespread use of these promising decision support systems in practice.

  • Medical Errors
  • Clinical Decision Support Systems
  • Validation

Statistics from

Request Permissions

If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.