You are here

  • Home
  • Online First
  • Effect of a smart temperature logger on correctly storing biological disease-modifying antirheumatic drugs at home: a pre-post study

Article Text

Article menu
Original research
Effect of a smart temperature logger on correctly storing biological disease-modifying antirheumatic drugs at home: a pre-post study
  1. Lex L Haegens1,
  2. Victor J B Huiskes2,3,
  3. Charlotte L Bekker3,
  4. Bart J F van den Bemt2,3
  1. 1Department of Research, Sint Maartenskliniek, Nijmegen, The Netherlands
  2. 2Department of Pharmacy, Sint Maartenskliniek, Nijmegen, Gelderland, The Netherlands
  3. 3Department of Pharmacy, Radboudumc, Nijmegen, Gelderland, The Netherlands
  1. Correspondence to Lex L Haegens, Research, Sint Maartenskliniek, Nijmegen, Netherlands; l.haegens{at}maartenskliniek.nl

Abstract

Objectives Biological disease-modifying antirheumatic drugs (bDMARDs) require specific storage temperatures, but are frequently stored outside the recommended range of 2–8°C. As incorrect storage may affect therapy effectiveness and consequently lead to higher disease activity, compliance with recommended storage temperatures should be improved. eHealth interventions can provide insight into storage temperatures and alerts in case of deviations from recommended temperatures. Therefore, this study aims to assess the effect of a smart temperature logger on correctly storing bDMARDs at home by patients with rheumatic diseases.

Methods A pre-post study was performed in a hospital in the Netherlands. The baseline period consisted of 12 weeks of storage temperature measurement with a passive temperature logger, and the intervention period consisted of 12 weeks of storage temperature measurement with a smart temperature logger. This smart logger included a smartphone application which provided insight into storage temperatures and real-time alerts when exceeding recommended temperatures. The main outcome measure was the difference in the number of patients who stored their bDMARDs correctly between baseline and intervention. Secondary outcomes were the difference in the proportion of measurement time within 2–8°C between baseline and intervention, the distribution of measurement time among temperature categories, and the patient’s acceptance measured using a questionnaire based on the Technology Acceptance Model.

Results In total, 48 participants (median age 55 years (IQR 47–64), 53% male) were analysed. The proportion of participants correctly storing bDMARDs increased from 18.8% (n=9) during baseline to 39.6% (n=19) during intervention (p=0.004). The median proportion of measurement time between 2–8°C improved by 6% (IQR 0–34%) (p<0.0001). Technology acceptance was scored as moderate.

Conclusions Temperature monitoring and real-time feedback with a smart temperature logger shows potential to improve at-home storage of bDMARDs, provided that continuous connection is realised to ensure real-time alerts and data collection.

  • Rheumatology
  • PHARMACY SERVICE, HOSPITAL
  • DRUG-RELATED SIDE EFFECTS AND ADVERSE REACTIONS
  • PREVENTIVE MEDICINE

Data availability statement

Data are available upon reasonable request.

http://creativecommons.org/licenses/by-nc/4.0/

This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, an indication of whether changes were made, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.

https://doi.org/10.1136/ejhpharm-2023-004028

Statistics from Altmetric.com

    Request Permissions

    If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.

    WHAT IS ALREADY KNOWN ON THIS TOPIC

    • The majority of patients with inflammatory rheumatic diseases store their refrigerated biological disease-modifying antirheumatic drugs incorrectly at home. Incorrect storage may affect therapy effectiveness and safety and should therefore be prevented.

    WHAT THIS STUDY ADDS

    • A smart temperature logger is able to improve correct at-home storage significantly. Smart temperature loggers should possess several (technical) features before real-world application, such as connection stability between temperature logger and the user’s smartphone, as well as ensuring data are continuously collected.

    HOW THIS STUDY MIGHT AFFECT RESEARCH, PRACTICE OR POLICY

    • Application of smart temperature loggers in temperature-sensitive medication has the potential to improve correct storage of this medication, possibly benefitting individual patients. Future application of this technology in specific therapy phases and application regarding redispensing temperature-sensitive medication should be researched.

    Introduction

    Biological disease-modifying antirheumatic drugs (bDMARDs) are safe and effective drugs for the treatment of inflammatory rheumatic diseases if the treatment target is not achieved with conventional synthetic DMARDs alone.1 bDMARDs require storage between 2–8°C, as described in the Summary of Product Characteristics (SmPC) recommendations.2–8 Via a strictly monitored cold chain, bDMARDs are produced, stored and transported within this temperature range. But from the moment drugs are dispensed to the patient, storage conditions are uncontrolled and out of sight of healthcare professionals. Despite instruction to the patient regarding storage conditions by the pharmacy staff on dispensing, research has shown that over 90% of patients using bDMARDs do not store their drugs according to the SmPC instructions.9–11 Moreover, one study found that 26% of patients with rheumatoid arthritis stored their bDMARDs below 0°C or above 25°C for more than 2 hours consecutively, and 6% below 0°C for at least 24 hours consecutively.9 Storing bDMARDs outside the recommended temperature range can lead to formation of protein aggregates,12 13 which in turn may lead to formation of anti-drug antibodies.14 Anti-drug antibodies are immunogenic and result in reduced drug responses for bDMARDs.15 16 In short, incorrect storage of bDMARDs may affect therapy effectiveness and lead to higher disease activity.17

    Compliance with recommended storage temperatures of bDMARDs should thus be improved. eHealth technologies have previously been applied for at-home monitoring of body temperature as a vital sign18 19 or foot temperature in diabetes, for example,20 21 and options for at-home temperature monitoring of medication storage have been outlined previously.22

    eHealth can therefore be a valuable instrument to support patients in correctly storing drugs at home by giving patients insight into the actual storage conditions within their refrigerator. MedAngel One is a wireless temperature logger connected with a smartphone application that continuously monitors storage temperature and alerts users in real-time when actual storage temperatures exceed recommended storage temperatures. Temperature monitoring with real-time alerts might help patients to store bDMARDS adequately within the recommended temperature range. It is currently unknown if a temperature logger with a smartphone application is able to positively affect storage conditions of bDMARDs at home, and whether patients find such a device acceptable and easy to use.

    Therefore, the primary objective of this study was to assess the effect of using a smart temperature logger on storing of bDMARDs at home according to the recommended temperature range by patients with inflammatory rheumatic diseases. Additionally, user experience was assessed using a questionnaire based on the Technology Acceptance Model.

    Methods

    Design and population

    This one-group pre-post-test study was performed between February 2021 and July 2022 in the outpatient pharmacy of the Sint Maartenskliniek, which is the largest rheumatology clinic in the Netherlands with 18 850 patients treated in 2021. Patients were eligible for participation if all the following criteria were met: (1) receive treatment by a rheumatologist for rheumatoid arthritis, psoriatic arthritis or spondyloarthritis (based on International Classification of Diseases, 10th Revision (ICD-10) codes in the patient’s medical record); (2) receive a bDMARD from the outpatient pharmacy prescribed by their rheumatologist; (3) possess a smartphone or tablet capable of running the software application accompanying the smart temperature logger; and (4) sufficiently write and understand the Dutch language.

    Sample size

    Prior research has shown that only 7% of patients with rheumatoid arthritis stored their drugs within the recommended 2–8°C.9 We hypothesise that our intervention will be able to improve this to 20%. With a power of 0.8 and a two-sided α of 0.05, a sample size of 124 was calculated. Including 20% loss to follow-up, we aimed for a sample size of 150 participants.

    Procedure and data collection

    Eligible patients received study information by email and were asked to provide digital informed consent. Participants were enrolled at their next bDMARDs delivery by the pharmacy, following the study procedure depicted in figure 1.

    Figure 1
    Figure 1

    Overview of study procedure. TAM, Technology Acceptance Model.

    After inclusion, participants received bDMARDs for exactly 12 weeks from the outpatient pharmacy, delivered at home or dispensed in the pharmacy. A sealed passive temperature logger (TempyTag23) was placed on top of one drug package to measure storage temperature continuously, with instructions to use the package with the logger last, to ensure 12 weeks of temperature registration. Participants were instructed to store their bDMARDs as usual.

    After 12 weeks, participants received their subsequent delivery of bDMARDs. During this intervention period, participants received a ‘smart’ intervention temperature logger. A passive temperature logger was also provided to ensure temperature registrations in the baseline and intervention periods were measured by the same device. During the intervention period, participants were instructed to instal the smartphone application accompanying the intervention temperature logger, register their temperature logger, and actively monitor their storage condition using the smartphone application. To ensure both data were gathered and past storage conditions were viewed, participants received weekly reminders via email to connect to and synchronise with their temperature logger. No specific instructions on how to handle alerts were provided other than participants should alter the way they store their bDMARDs in order to comply with the recommended temperature range.

    After the intervention period, participants received a return envelope to return the passive temperature logger. Participants were allowed to keep the smart temperature logger for future use free of charge. Additionally, participants received an email with an invite to complete a digital questionnaire regarding user experience.

    Intervention

    The intervention used in this study, the MedAngel One, is a Bluetooth temperature logger that allows users to (retrospectively) monitor storage temperature of temperature-sensitive drugs 24/7 using the accompanying application for mobile phones. Additionally, the application provides the user with real-time alerts on their smartphone when temperature exceeds pre-defined temperature limits (set at 2–8°C for all participants), assuming an active Bluetooth connection. Although several temperature loggers exist, the MedAngel One was chosen as this temperature logger features both a smartphone and a tablet application giving insight into current and past recorded temperatures, and additionally can give a real-time alert in case of excursions outside 2–8°C.

    Primary outcome

    The primary outcome was the difference in the proportion of patients who correctly stored their bDMARDs within 2–8°C between baseline and the intervention period. Correct storage was defined as storing bDMARDs within 2–8°C for the total duration of measurements (1) without excursions outside this temperature range for more than 48 hours cumulatively, and (2) without excursions below 0°C or above 25°C for a duration of at least 2 hours consecutively, in accordance with a previous study performed by Vlieland et al.9

    Secondary outcomes

    Secondary outcomes were (1) the difference in proportion of total measurement time between 2–8°C between baseline and the intervention period, (2) the difference in distribution of measurement time across multiple temperature groups (<0°C, 0–2°C, 2–8°C, 8–15°C, 15–25°C and >25°C) between baseline and the intervention period, and (3) user experience with the software application.

    Questionnaire

    The questionnaire to assess individual participants’ acceptance of the intervention was based on the Technology Acceptance Model.24 This model states that technology use is directly dependent on the intention to use a technology. This intention is in turn dependent on the attitude towards using a technology, which is a product of the perceived usefulness and perceived ease of use of a technology.

    Questions were derived from Davis24 (Perceived usefulness), Brooke’s System Usability Scale (SUS)25 (perceived ease of use), and the user version of the smartphone application Rating Scale (uMARS) by Stoyanov et al.26 Items from the uMARS questionnaire were screened and selected for applicability to this study by LH and CB. Selected questions were initially applied to the smartphone application and translated by LH, checked by CB, and adjusted by LH and CB until all discrepancies were resolved (English translation in online supplemental material A).

    Data analysis

    The start and end of measurement periods were determined for individual temperature loggers. Within participants, the shortest measurement time from either baseline or intervention temperature logger was applied to the other temperature logger measured from the start of measurements to ensure corresponding lengths of measurement times within participants (online supplemental material B). For the primary outcome measure, both baseline and intervention periods from individual participants were assessed binarily as correct or incorrect, according to the criteria outlined in the methods section. The difference in the proportion of participants who correctly stored bDMARDs between baseline and intervention was calculated by McNemar’s test for paired categorical data.

    Proportions of measurement time were compared between baseline and intervention period by either paired t-test or Wilcoxon Signed Rank test, depending on normality of the data. Technology acceptance was assessed using the scoring methods of the incorporated questionnaires, if applicable, and reported using mean and SD or median and IQR, depending on the normality of the data.

    Results

    Participants

    A total of 640 eligible patients were invited to participate, of whom 151 agreed (24% response rate) and 140 (93%) started the baseline period. Of these participants, 89 (59%) resulted in retrievable baseline temperature measurements, after which 48 (32%) participants resulted in both retrievable baseline and intervention temperature measurements (figure 2). The questionnaire was completed by 90 participants (64%), as this was sent to all participants completing intervention measurements, regardless of retrievability of temperature data.

    Figure 2
    Figure 2

    Participant flow chart. bDMARDs, biological disease-modifying antirheumatic drugs.

    The participants’ median age was 55 (IQR 47–64) years with just over half the population being male (53%). The participants’ median time since diagnosis was 7 (IQR 2–17) years, and 47% had rheumatoid arthritis (table 1). Participants excluded from per-protocol analysis did not differ significantly from included participants (data not shown). After matching measurement times of baseline and intervention periods within participants, the mean (SD) total measurement time per patient per study phase was 71.8 (19.4) days.

    View this table:
    Table 1

    Baseline characteristics of participants

    Primary outcome

    The proportion of participants who correctly stored bDMARDs increased significantly from 18.8% (n=9) during baseline to 39.6% (n=19) during the intervention period (p=0.004). The main improvement was seen in the number of participants who complied with the criterion of a maximum of 48 hours outside 2–8°C in total, with nine participants (18.8%) complying in the baseline period and 20 (41.7%) in the intervention period. Data on numbers of patients who stored bDMARDs in accordance with individual requirements for correct storage can be found in the online supplemental material C.

    Secondary outcomes

    Proportion within 2–8°C

    The median proportion of measurement time per patient between 2–8°C improved statistically significantly with 6% (IQR 0–34%) (p<0.0001) (figure 3) from median 69% (IQR 19–96%) during baseline to 96% (IQR 70–99%) during intervention. Thirty-seven participants (77%) improved in the proportion of total measurement time within 2–8°C with a median of 24% (IQR 3–47%) between baseline and intervention. Nine participants (19%) worsened in proportion to the total measurement time within 2–8°C with a median of 4% (IQR 1–9%) between baseline and intervention. Two participants (4%) did not improve or worsen, of whom one participant had 100% and the other had 0% of total measurement time within 2–8°C during both baseline and intervention. A figure with individual baseline and intervention measurements can be found in the online supplemental material D.

    Figure 3
    Figure 3

    Individual difference in proportion of total measurement time within 2–8°C between baseline and intervention period. Dots represent individual participants.

    Temperature distribution

    The proportion of total measurement time within 2–8°C increased significantly from 59% to 79% (p<0.0001) and the proportion of total measurement time within 8–15°C decreased significantly from 36% to 19% (p<0.0001) (figure 4).

    Figure 4
    Figure 4

    Bar chart showing distribution of measurement time over temperature categories. *Statistically significant difference (p<0.05).

    Technology acceptance questionnaire

    Of the 90 participants who completed the intervention, 71% (n=64) completed at least one section and 49% (n=44) fully completed the questionnaire. Results per questionnaire section below are based on participants who completed that particular section.

    Perceived ease of use

    All respondents completed the perceived ease of use section of the questionnaire. The mean (SD) SUS score was 61 (17), with 40% rating ease of use as good or excellent (SUS >71.4).27 The mean (SD) score on the selected items from uMARS was 2.9 (1.0) out of 5.

    Perceived usefulness

    Sixty-two respondents (97%) completed this section of the questionnaire. Overall, participants’ perceived usefulness of the smartphone application was neutral to slightly positive (figure 5). Only usefulness of the application for disease management (‘By using this application, I think my rheumatic disease is doing better’) and engagement with their rheumatic disease (‘The application makes me more engaged with my rheumatic disease’) were perceived lower, with both statements scoring a median of ‘slightly disagree’.

    Figure 5
    Figure 5

    Median and IQR scores for statements regarding usefulness of the application accompanying the intervention temperature logger.

    Attitude towards using

    Sixty respondents (94%) completed this section of the questionnaire. The application received a median of 3 (IQR 2–3) out of 5 stars: 27 of these respondents (45%) would use the application at least monthly; 33 respondents (55%) indicated they would use the application once per year or not at all; 47 (78%) respondents would recommend the application to other people who could benefit from it; and eight (13%) participants were willing to pay for the app.

    Discussion

    This study showed that a smart temperature logger was able to increase significantly the proportion of participants who stored their bDMARDs correctly. Furthermore, participants stored bDMARDs significantly longer within the recommended temperature range of 2–8°C using the intervention when compared with baseline. Most of this improvement was found in a shift from storage between 8–15°C to between 2–8°C. Technology acceptance was scored as moderate. However, technical defects with both baseline and intervention loggers impacted the number of participants included in the analysis and intended intervention exposure. Defects included baseline temperature loggers with (partly) irretrievable data and intervention temperature loggers with insufficient connection stability, possibly resulting in absence of real-time alerts and retroactively viewable temperature data.

    This study investigated the effect of a smart temperature logger on the proportion of participants who stored bDMARDs correctly. We found that 18.8% of participants correctly stored bDMARDs during the baseline period. Although this number is higher than the 6.7% found by Vlieland et al, at-home storage of bDMARDs is still suboptimal without intervention.9 The proportion of measurement time within 2–8°C of 59% found in this study is comparable with the proportion of correct bDMARD storage time of 54.8% by Vlieland et al9 and other studies reporting on correct storage of bDMARDs.10 28 No studies have investigated an intervention to improve this suboptimal at-home storage of bDMARDs.

    The intervention used in this study significantly improved the proportion of participants who correctly stored bDMARDs and the proportion of storage time within 2–8°C. However, the criteria for correct storage in this study were stricter than storage requirements stated in the SmPCs which state that bDMARDs can be stored at temperatures of up to 30°C for prolonged periods of time, ranging from 5 to 28 days.2–8 Therefore, the question remains whether this improvement has a clinically relevant effect (eg, on therapy effectiveness and safety, or disease activity) as improvement could have taken place in participants who already fell within the SmPC stated storage requirements.

    In spite of the significant increase of storage time at the right conditions between baseline and intervention, in 19% of the participants the proportion of measurement time between 2–8°C decreased from baseline to intervention. This might be explained by the fact that patients inadvertently may have adjusted the way of storage wrongly but were insufficiently alerted by the smart temperature logger due to technical defects.

    This study showed the potential of a smart temperature logger in improving at-home storage conditions of bDMARDs. However, technical problems encountered during the study should be resolved before the intervention is applicable in a real-world setting. Therefore, based on the experiences in this study, a smart temperature logger should possess several (technical) features. Connection stability between the smart temperature logger and the user’s smartphone should be guaranteed to ensure real-time feedback on storage temperature. Due to the limited range of Bluetooth, the connection between smartphone and temperature logger could be improved by using an internet connection. Additionally, data collection should be ensured to enable users to monitor storage temperature retrospectively. Presuming Bluetooth signals might have difficulty penetrating certain refrigerators and is limited in connectivity range, other connections such as WiFi could be explored.

    Although a smart temperature logger seems promising for improving at-home storage of refrigerated drugs under the right conditions, one could argue whether continuous monitoring is needed or whether this can be applied for specific therapy phases. For example, continuous monitoring and real-time feedback when temperature limits are exceeded may be useful during the first dispensing of refrigerated drugs, as patients get to understand how to store their drugs properly in combination with their refrigerator at therapy initiation. This may sufficiently improve at-home storage of bDMARDs at the right conditions in the long term and obviate the need for long-term continuous intervention use. On the other hand, as bDMARDs are expensive drugs, with prices ranging from €8000 to €12 000 per patient per year depending on the bDMARD and dosing interval, disposal of medication due to incorrect storage can lead to economic losses.29 Furthermore, a recent study has shown that patients are willing to participate in medication redispensing of oral anticancer drugs.30 In this context, monitoring of at-home storage temperature of drugs can increase patient accountability in the process of assuring that returned (expensive) drugs are suitable for redispensing, in addition to providing a tool to monitor storage temperature retrospectively.

    Limitations

    A few limitations should be acknowledged. First, technical defects with temperature loggers might have had an impact on the outcomes. Although excluded participants did not differ from included participants, one cannot assume that these participants would have had similar storage conditions. Furthermore, intervention exposure (ie, number of alerts generated) was not measurable due to technical defects, and participants possibly did not receive real-time feedback on storage temperature and might therefore be underexposed to the intervention. Therefore, it remains uncertain whether improvements in storage conditions are related to real-time feedback or to (retrospective) insight in storage temperature. Second, generalisability might be limited as participants were recruited digitally. Furthermore, participants were obliged to possess a smartphone. This potentially attracted a more digitally-inclined population when compared with the general rheumatic diseases population. Additionally, part of the effect of the intervention may be due to the written instructions patients received at the beginning of the study on how to adjust their storage behaviour when they received an alert of temperatures exceeding 2–8°C. Third, due to the pre-post design of this study with a single group, participants may have been aware of being observed and as a result altered their bDMARDs storage. This risk was minimised by not providing feedback on storage temperature during the baseline period. Moreover, participants received no instructions regarding bDMARDs storage at the start of the baseline period, only at the start of the intervention period. Lastly, seasonal weather changes may have influenced storage conditions within refrigerators, but since data collection for both the baseline and intervention period took place across all seasons, we assume this risk was minimal.

    Conclusion

    Temperature monitoring and real-time feedback with a smart temperature logger shows potential to improve at-home storage of refrigerated medication, provided that the smart temperature logger has adequate connection stability and ensured data collection.

    Data availability statement

    Data are available upon reasonable request.

    Ethics statements

    Patient consent for publication

    Ethics approval

    This study was performed in line with the principles of the Declaration of Helsinki (version 2013). The Medical Ethics Research Committee of Arnhem-Nijmegen waived official ethical approval (case-number 2020–7127) as this study was deemed not subject to the Medical Research Involving Humans Act. Informed consent was obtained from all individual participants before inclusion in the study.

    Acknowledgments

    The smart temperature loggers used in this study were provided free of charge by MedAngel BV.

    References

    1. ↵
      1. Smolen JS,
      2. Landewé RBM,
      3. Bergstra SA, et al
      . EULAR recommendations for the management of rheumatoid arthritis with synthetic and biological disease-modifying antirheumatic drugs: 2022 update. Ann Rheum Dis 2023;82:3–18. doi:10.1136/ard-2022-223356
      OpenUrlAbstract/FREE Full Text
    2. ↵
    3. ↵
    4. ↵
    5. ↵
    6. ↵
    7. ↵
    8. ↵
    9. ↵
      1. Vlieland ND,
      2. Gardarsdottir H,
      3. Bouvy ML, et al
      . The majority of patients do not store their biologic disease-modifying antirheumatic drugs within the recommended temperature range. Rheumatology (Oxford) 2016;55:704–9. doi:10.1093/rheumatology/kev394
      OpenUrlCrossRefPubMed
    10. ↵
      1. de Jong MJ,
      2. Pierik MJ,
      3. Peters A, et al
      . Exploring conditions for redistribution of anti-tumor necrosis factors to reduce spillage: a study on the quality of anti-tumor necrosis factor home storage. J Gastroenterol Hepatol 2018;33:426–30. doi:10.1111/jgh.13920
      OpenUrl
    11. ↵
      1. Santin G,
      2. da Silva MMM,
      3. Villarreal VA, et al
      . Home storage of biological medications administered to patients with rheumatic diseases. Adv Rheumatol 2020;60:30. doi:10.1186/s42358-020-00131-x
      OpenUrl
    12. ↵
      1. Liu L,
      2. Braun LJ,
      3. Wang W, et al
      . Freezing-induced perturbation of tertiary structure of a Monoclonal antibody. J Pharm Sci 2014;103:1979–86. doi:10.1002/jps.24013
      OpenUrl
    13. ↵
      1. Vlieland ND,
      2. Nejadnik MR,
      3. Gardarsdottir H, et al
      . The impact of inadequate temperature storage conditions on aggregate and particle formation in drugs containing tumor necrosis factor-alpha inhibitors. Pharm Res 2018;35:42. doi:10.1007/s11095-017-2341-x
    14. ↵
      1. Ratanji KD,
      2. Derrick JP,
      3. Dearman RJ, et al
      . Immunogenicity of therapeutic proteins: influence of aggregation [Internet]. J Immunotoxicol 2014;11:99–109. doi:10.3109/1547691X.2013.821564
      OpenUrlCrossRefPubMedWeb of Science
    15. ↵
      1. Garcês S,
      2. Demengeot J,
      3. Benito-Garcia E
      . The immunogenicity of anti-TNF therapy in immune-mediated inflammatory diseases: a systematic review of the literature with a meta-analysis. Ann Rheum Dis 2013;72:1947–55. doi:10.1136/annrheumdis-2012-202220
      OpenUrlAbstract/FREE Full Text
    16. ↵
      1. Garcês S,
      2. Demengeot J
      . The Immunogenicity of biologic therapies. Curr Probl Dermatology 2017;53:37–48. doi:10.1159/isbn.978-3-318-06101-7
      OpenUrl
    17. ↵
      1. de Assis Damasce TM,
      2. Fernand V,
      3. Leite da Silva CA, et al
      . Storage conditions of immunobiologicals and their influence on the efficacy and safety in the treatment of autoimmune rheumatic diseases. The Open Rheumatology Journal 2020;14:1–6. doi:10.2174/1874312902014010001
      OpenUrl
    18. ↵
      1. Kagiyama N,
      2. Hiki M,
      3. Matsue Y, et al
      . Validation of telemedicine-based self-assessment of vital signs for patients with COVID-19: a pilot study. J Telemed Telecare 2023;29:600–6. doi:10.1177/1357633X211011825
      OpenUrl
    19. ↵
      1. Boano CA,
      2. Lasagni M,
      3. Romer K, et al
      . . 2011 IEEE International Symposium on Object/Component/Service-Oriented Real-Time Distributed Computing Workshops; Newport Beach, CA, USA.2011:2011; 10.1109/ISORCW.2011.28
    20. ↵
      1. Tekale S,
      2. Varma A,
      3. Tekale S, et al
      . A review on newer interventions for the prevention of diabetic foot disease [Cureus [Internet]]. Cureus 2022;14:e30591. doi:10.7759/cureus.30591
    21. ↵
      1. Brooks E,
      2. Burns M,
      3. Ma R, et al
      . Remote diabetic foot temperature monitoring for early detection of diabetic foot ulcers: a cost-effectiveness analysis. Clinicoecon Outcomes Res 2021;13:873–81. doi:10.2147/CEOR.S322424
      OpenUrl
    22. ↵
      1. National Homecare Medicines Committee
      . ‘RPS Handbook for Homecare services - Appendix 23: maintaining ‘the cold chain’ in patients’ homes - guidance for temperature control, storage &amp; handling of homecare medicines by patients’. 2022. Available: https://www.rpharms.com/Portals/0/RPS
    23. ↵
      1. Confrerie Clinique
      . TempyTag-NFC. 2023. Available: https://www.confrerie-clinique.com/products/tempytag-nfc/
    24. ↵
      1. Davis FD
      . A technology acceptance model for empirically testing new end-user information systems: theory and results management. 1985.
    25. ↵
      1. Brooke J
      . ‘SUS: A ‘quick and dirty’ usability scale’. Usability Eval Ind 1995:207–12. doi:10.1201/9781498710411
    26. ↵
      1. Stoyanov SR,
      2. Hides L,
      3. Kavanagh DJ, et al
      . n.d. Development and validation of the user version of the mobile application rating scale (uMARS). JMIR Mhealth Uhealth;4:e72. doi:10.2196/mhealth.5849
    27. ↵
      1. Bangor A,
      2. Kortum P,
      3. Miller J
      . Determining what individual SUS scores mean; adding an adjective rating. J Usability Stud 2009;4:114–23.
      OpenUrl
    28. ↵
      1. Cuéllar MJ,
      2. Marco JL,
      3. Pérez-Castelló I, et al
      . Calidad en La Conservación de Los Medicamentos Termolábiles en El Ámbito Domiciliario. Rev Calid Asist 2010;25:64–9. doi:10.1016/j.cali.2009.09.001
      OpenUrlCrossRefPubMed
    29. ↵
      Medicijnkosten.nl. 2023. Available: https://www.medicijnkosten.nl/
    30. ↵
      1. Smale EM,
      2. Egberts TCG,
      3. Heerdink ER, et al
      . Key factors underlying the willingness of patients with cancer to participate in medication redispensing. Res Social Adm Pharm 2022;18:3329–37. doi:10.1016/j.sapharm.2021.12.004
      OpenUrl

    Supplementary materials

    • Supplementary Data

      This web only file has been produced by the BMJ Publishing Group from an electronic file supplied by the author(s) and has not been edited for content.

    Footnotes

    • EAHP Statement 6: Education and Research.

    • Contributors LH, VB, CB and BB designed the study; LH collected and analysed the data; LH, VB, CB and BB drafted the manuscript; BB is guarantor of the study.

    • Funding This study was funded by MedAngel BV.

    • Competing interests None declared.

    • Provenance and peer review Not commissioned; externally peer reviewed.

    • Supplemental material This content has been supplied by the author(s). It has not been vetted by BMJ Publishing Group Limited (BMJ) and may not have been peer-reviewed. Any opinions or recommendations discussed are solely those of the author(s) and are not endorsed by BMJ. BMJ disclaims all liability and responsibility arising from any reliance placed on the content. Where the content includes any translated material, BMJ does not warrant the accuracy and reliability of the translations (including but not limited to local regulations, clinical guidelines, terminology, drug names and drug dosages), and is not responsible for any error and/or omissions arising from translation and adaptation or otherwise.