Background and Importance In order to fully capture the contribution of clinical pharmacists to pharmacotherapy, a standardised and validated classification tool for drug-related problems (DRP) and pharmacist interventions (PI) is essential for both research purposes and management tasks. Such an instrument is not yet available in Austria. Therefore, the documentation system, ‘DokuTool’, has been developed by an Upper Austria hospital trust following the expansion of its clinical pharmacy services.

Aim and Objectives This study aimed to assess the reliability, validity and user-practicability of the classification system, ‘DokuTool’, within an Upper Austrian hospital trust.

Material and Methods Two-phase quantitative methodology: 1) Twenty-nine clinical pharmacists classified 24 sample cases with ‘DokuTool’. Inter-rater and test-retest reliability was determined using the kappa statistic. Validity was determined by correlating the individual ratings with a gold standard (majority vote of experts) using contingency coefficient. 2) User-practicability was assessed by an online survey using a 5-point Likert scale.

Results ‘DokuTool’ achieved moderate agreement in the inter-rater reliability test of the two main categories ‘Type of DRP’ (k = 0.528 [95% confidence interval (CI): 0.514 – 0.541]) and ‘Cause of DRP’ (k = 0.594 [95% CI: 0.587 – 0.601]). The category ‘Planned PI’ showed substantial agreement (k = 0.638 [95% CI: 0.629 – 0.647]). Test-retest reliability achieved an almost perfect agreement for all three main categories: ‘Type of DRP’ (k = 0.825 [95% CI: 0.734 – 0.915]), ‘Cause of DRP’ (k = 0.896 [95% CI: 0.825 – 0.967]) and ‘Planned PI’ (k = 0.891 [95% CI: 0.819 – 0.964]). The median rater-specific contingency coefficient was 0.84 [range: 0.75 – 0.89], 0.95 [0.94 – 0.96] and 0.93 [0.91 – 0.94]. ‘DokuTool’ was rated comprehensive (median: 2 [interquartile range: 1.75]), user-friendly (2 [1]) and practical (2 [1]). Time expenditure was considered adequate (3 [1]), but the completeness and clarity of the categories were rated negatively (3 [2]).

Conclusion and Relevance Moderate to substantial inter-rater reliability, almost perfect test-retest reliability, good criterion validity and acceptable user-practicability demonstrated that ‘DokuTool’ is a valid and reliable classification instrument of DRPs and PIs, that is well-suited for Austria.

However, the evaluation of usability led to suggestions for improvement for future versions.

Conflict of Interest No conflict of interest