Table 3

Characteristics and outcomes of the studies (n=8) included in the systematic review

Reference; country; setting	Study design; grade	Objective	Methods	Intervention; unit dose dispensing robot used	Outcomes
Medication administration error assessment
Cousein et al 17; France; short stay geriatric unit (40 beds) within a general hospital	Before-after observational study; moderate	To assess the effect of automated UDDS on MAEs	Observation periods of medication administration rounds were 3 months for WSS, 2 months for UUDS1 and 1 month for USSD2. Within the rounds administered drugs to prescribed drugs were compared. Outwith the operating hours of the UD robot, drugs were available for the nurses via ADC. MAEs were calculated, classified and compared between the study periods. Additionally, error gravity and risk reduction for the patients were defined. Statistical analysis: Student’s t-test (quantitative variables) and χ² test or Fisher’s exact (qualitative variables); 95% CI and two-tailed α level of 0.05.	Implementation of patient specific UDDS including UD dispensing robot, CPOE, pharmaceutical prescription control and ADC in medication process. UDDS had two variants; UDDS1 contained CPOE without eMAR and UDDS2 contained both CPOE and eMAR. versus Traditional WSS with manual drug dispensing by nurses. No control group. UD dispensing robot: PillPick, Swisslog	MAEs 53% reduction of MAEs with UDDS versus WSS All MAE types reduced with UDDS versus WSS; wrong dose by 79.1% and wrong drug by 93.7% Decrease of MAEs by 45% and 62% comparing UDDS1 and UDDS2 versus WSS, respectively No significant difference between UDDS1 and UDDS2 Patient risk for being exposed to one or more MAEs Reduction of absolute and relative risks by 10.5% and 34.6%, respectively, when WSS was switched to UDDS One of every 10 patients avoided a ME or for every 15 medications administered a ME was prevented Medication error gravity Statistically lower prevalence of errors requiring monitoring, therapy or intervention during the UDDS compared with WSS
Risør et al 18; Denmark; two haematological wards (21 and 22 beds, respectively) within a university hospital	Prospective, controlled before-after study; moderate	To evaluate the impact of psAMS on MAEs and MAE subtypes	Study was conducted in two wards having comparable workflow, medication profile and flow of patients. The observation periods were performed 3 weeks before and after implementation in both intervention and control wards. MAEs were identified and calculated by direct observation of the medication administration process. MAEs were further divided into clinical errors (ie, patient did not receive the medication as prescribed) and procedural errors (ie, deviations from written procedures or guidelines that may lead to a clinical error). Statistical analysis: logistic regression; p≤0.05, OR and 95% CI. Sub-analyses and sensitivity analyses	Implementation of psAMS including ADD, CPOE, pharmaceutical prescription control, eMAR and BCMA in medication process. versus Traditional WSS with manual drug dispensing by nurses, CPOE and eMAR in control group. UD dispensing robot: PillPick, Swisslog	MAEs Overall risk of MAEs reduced significantly by 57% with psAMS compared with WSS Clinical errors were reduced significantly by 94% Procedural errors were reduced non-significantly by 30% No clinical errors occurred in doses for which PDA scanning was used correctly during BCMA
Risør et al 23; Denmark; two acute medical units (17 and 14 beds, respectively) within a university hospital	Prospective, controlled before-after study; moderate	To evaluate the effectiveness of cAMS and npsAMS on MAEs and MAE subtypes	Study was performed in two units having similar medication handling process and comparable medication profile. Measurement of MAEs was performed by direct observations of the medication administration process in intervention and control units during a 3 week period as initial baseline and two subsequent follow-up periods after implementation. Any discrepancy between administered and prescribed medication or deviations from written procedures or guidelines was considered as an error and categorised by error types (clinical errors or procedural errors). Statistical analysis: logistic regression; p≤0.05, OR and 95% CI. Sub-analyses	Implementation of cAMS and npsAMS sequentially: first cAMS consisting of ADD, CPOE, pharmaceutical prescription control, eMAR, BCMA and ADC was implemented and subsequently npsAMS consisting of ADD, CPOE, eMAR and BCMA. versus Traditional WSS with manual drug dispensing by nurses, CPOE and eMAR in control group. UD dispensing robot: PillPick, Swisslog	MAEs cAMS reduced significantly overall risk of MAEs by 47% and procedural errors by 56% compared with WSS cAMS reduced clinical errors by 20% compared with WSS but reduction was not significant npsAMS reduced significantly clinical errors by 62% compared with WSS npsAMS decreased overall risk of MAEs by 27% and procedural errors by 11% compared with WSS, however non-significantly No significant difference between cAMS and npsAMS
Dispensing error assessment
Le Gonidec et al 20; France; medical ward of a jailhouse (350 patients)	Observational study; very low	To evaluate the performance of ADD – work flow and drug dispensing errors	Drugs were dispensed as UDs patient specifically for 7 days at the time (average 3 UDs per prescription) by ADD robot. UD packing and dispensing rates, dispensing errors and security of medication circuit were observed for 3 months. Study was conducted 2.5 years after implementation of the ADD robot. No statistical analysis	Implementation of ADD integrated with CPOE and dispensing software. No comparison with previous drug dispensing system. UD dispensing robot: PillPick, Swisslog	UD packing and dispensing rates 377 UDs packed per hour and 537 UDs dispensed per hour Dispensing errors Dispensing error rate 0.5%; too many or missing doses due to wrong delivery orders mainly generated by computer order entry software Secondary outcome ADD saved the working time required for manual collection of medicines from technical staff but did not reduce any work from pharmacists
Sutra et al 21; France; multiple types of geriatric wards (total 307 beds) within university hospital	Observational study; very low	To assess the impact of automated UDDS on drug dispensing errors – for appropriate corrective operations before implementing the UDDS broader	Drug dispensing errors were observed systematically by nurses in wards supplied by automated UDDS over 13 month period and classified into four categories: errors related to the UDDS, software related errors, errors in the declarations or prescriptions and errors related to pharmacy (non-automation errors). Results were presented as errors over the 13 month period, with monthly monitoring of error rates. Study was conducted 1 year after the implementation of the automated UDDS. No statistical analysis	Implementation of automated UDDS with integrated prescription, warehouse management and automation management software and pharmaceutical analysis of the prescription. No comparison with previous drug dispensing system. UD dispensing robot: Athena, Sinteco	Dispensing errors (including errors not related to UDDS) Total amount of drug dispensing errors 0.25%; 96% of errors caused by missing dose 41.5% of errors were related to UDDS process; caused by UD robot or user interference 35.6% of errors in declaration or prescription 21.2% of errors were software related errors Dispensing errors (related to UDDS) The average error rate 0.10% with a maximum rate of 0.16% Monitoring of error rates resulted in identification and correction of various problems in the UDDS
Economic evaluation
Risør et al 19; Denmark; two haematological wards (21 and 22 beds, respectively) within a university hospital	Original effectiveness data obtained from prospective, controlled before-after study by Risør et al 18; moderate	To evaluate the costs and cost-effectiveness of psAMS and to identify the cost factors related to psAMS	The economic evaluation compared psAMS with the traditional WSS in which medicines were delivered in their original packages to wards and dispensed by nurses. The cost analysis was performed in the hospital setting using two 6 month study periods (baseline and follow-up). The analysis used a short-term incremental costing approach and the assumption that only the costs related to medicine delivery and handling would change. Total costs were divided into costs of handling, waste, pharmaceutical services, PDAs and unit dose bags. Costs included labour and annual running costs, purchase costs of automated dispensing robot, development costs of interfaces between eMAR and PDAs, and costs of facilities. In cost-effectiveness evaluation the cost analysis and effects of psAMS to MAE rates were used. Calculation and statistical analysis: calculation of the total running and implementation costs, the annual amount of MAEs, the number of avoided MAEs and costs per avoided errors. Sensitivity analyses	Interventions are presented by Risør et al 18 UD dispensing robot: PillPick, Swisslog	Running and implementation costs psAMS increased the costs of medication dispensing and administration process with €16 843 per every 6 months Wastage costs and costs of pharmaceutical services were higher with psAMS Medication handling costs were lower with psAMS PDAs and unit dose bags caused costs only when the psAMS were used Implementation costs (planning, developing and implementing) of psAMS were €31 789 Cost-effectiveness Cost-effectiveness ratio (cost per avoided error) was: €2.01 per MAE €2.91 per procedural error €19.38 per clinical error Cost increase using psAMS was €502 per 100 patient-days and resulted in 258 avoided MAEs, 178 procedural errors and 27 clinical errors compared with traditional WSS
Risør et al 24; Denmark; two haematological wards (21 and 22 beds, respectively) and two acute medical units (17 and 14 beds, respectively) within a university hospital	Original effectiveness data obtained from two prospective, controlled before-after studies by Risør et al 18 23 ; moderate	To evaluate the costs and cost-effectiveness of psAMS, cAMS and npsAMS by using a model-based indirect cost-effectiveness comparison of three different, real world AMSs	The economic evaluation compared psAMS, cAMS and npsAMS with traditional WSS, respectively. The cost analysis was performed and contained the same cost factors as described previously by Risør et al 19 . In cost-effectiveness analysis the costs of AMSs defined in the cost analysis and effects of AMSs to MAE rates were used. Calculation and statistical analysis: Calculation of the total running costs and costs per dose. The effect of consumption and costs were standardised for an indirect comparative analysis. Calculation of the estimated number of MAEs and the number of avoided MAEs for each consumption scenario. ICERs for different drug consumption scenarios. Non-parametric simulations based on the mean/OR and SE of the mean on the observed parameters. The probability of cost-effectiveness related to different assumed values per avoided error interpreted as CEACs.	Interventions for psAMS are presented by Risør et al 18 Interventions for cAMS and npsAMS are presented by Risør et al 23 UD dispensing robot: PillPick, Swisslog	Running costs The total incremental costs with assumed consumption scenario of 30 000 doses per every 6 months/corresponding incremental costs per administered dose: €19 477/€0.65 for psAMS €17 132/€0.57 for npsAMS €102 653/€3.42 for cAMS The incremental costs per dose decreased with higher number of administered doses for all AMS, primarily for the cAMS Cost-effectiveness ICER with assumed consumption scenario of 30 000 doses per every 6 months: For MAEs €5 with psAMS, €14 with npsAMS and €46 with cAMS For procedural errors €8 with psAMS, €42 with npsAMS and €59 with cAMS For clinical errors €24 with psAMS, €26 with npsAMS and €386 with cAMS CEAC showed that psAMS has a 100% probability of being cost-effective at a valuation of €24 per avoided error with all error types CEAC for clinical errors indicated that npsAMS reached 100% cost-effectiveness probability at valuation over €26, whereas cAMS only reached 25% change to be cost-effective with value over €150 CEAC for administration and procedural errors indicated that the cost-effectiveness between cAMS and npsAMS was dependent on the valuation of the avoided error; npsAMS was more cost-effective than cAMS at lower value of avoided error (€50) but with higher value of avoided error (€100) cAMS was found to be more cost-effective than npsAMS
Lappalainen et al 22; Finland; 21 somatic adult wards within university hospital	BIA; low	To assess the net cost impact of psAMS implementation	BIA included theoretical salary costs, investments (purchase of automated dispensing robot, constitution costs of cleanroom, development costs of software interphases) as well as space and overhead costs related to AMS over a 10 year period. The hospital’s own databases and experts, data provided by the robot supplier and publicly available reliable statistics and databases were used as data sources for costs. In addition, the effect of increasing the number of hospital wards using the AMS was evaluated. Statistical analysis: one-way sensitivity analyses to assess the robustness of the results over a 10 year period	Implementation of an imaginary psAMS including ADD, CPOE, pharmaceutical prescription verification, eMAR and BCMA in medication process. versus Traditional WSS with manual drug dispensing. UD dispensing robot not specified	Running costs Initially the total costs of psAMS and the current scenario (manual drug dispensing process) were similar After 8 years use and the repayment of the system-related investments, costs of psAMS were around 37% lower Expected cost savings could be predicted to increase as a function of a number of hospital wards applying AMS Salary, maintenance and unit dose package costs are significant cost factors in AMS

Grade: high, moderate, low, very low.
ADC, automated dispensing cabinet; ADD, automated drug dispensing; AMS, automated medication system; BCMA, barcode-assisted medication administration; BIA, budget impact analysis; cAMS, complex automated medication system; CEAC, cost-effectiveness acceptability curve; CPOE, computerised physician order entry; eMAR, electronic medication administration record; ICER, incremental cost per avoided error; MAE, medication administration error; npsAMS, non-patient specific automated medication system; PDA, personal digital assistant; psAMS, patient specific automated medication system; UD, unit dose; UDDS, unit dose dispensing system; WSS, ward stock system.