Hassana Akhloufi

Quality indicators are an element of practice performance for which there is evidence or consensus that it can be used to assess the quality, and hence change in the quality, of care provided. Quality indicators to measure the appropriateness of antibiotic use in the treatment of all bacterial infections in hospitalized patients were not available yet at the moment of our survey. However, at this moment such quality indicators have been developed and validated in a clinical setting. These indicators describe/define appropriate antibiotic use from start to discontinuation of antibiotics, and appear to be associated with a shorter length of hospital stay. These quality indicators can be used to measure performance retrospectively, but can also be used prospectively as the basis for developing different types of clinical decision aids. The CDSS that we have developed (Chapter 4, 5 and 6) aims to improve/optimize these indicators (4 of 9 indicators), namely: 1. Systemic antibiotic therapy should be switched from iv to oral antibiotic therapy within 48-72 hours on the basis of the clinical condition and when oral treatment is adequate. 2. Empirical systemic antibiotic therapy should be prescribed according to the local guideline. 3. An antibiotic plan should be documented in the case notes at the start of systemic antibiotic therapy. 4. Dose and dosing interval of systemic antibiotics should be adapted to renal function. The other quality indicators are: - before starting systemic antibiotic therapy at least two sets of blood cultures should be taken; - take cultures from suspected sites of infection as soon as possible, preferably before antibiotics are started; -change empirical to pathogen-directed therapy if culture results become available; - perform therapeutic drug monitoring at steady state; - discontinue antibiotic therapy if infection is not confirmed. The maximum duration of empirical systemic antibiotic treatment should be 7 days. These quality indicators should also be used in antibiotic stewardship to measure appropriateness of antibiotic use and are ideally also a target for other CDSS in the future.

Consensus on iv to oral antibiotic switch criteria.
Although robust scientific evidence about which iv to oral antibiotic switch criteria have to be minimally met for a safe iv to oral switch in hospitalized patients is missing, physicians must nonetheless make the decision to switch on a regular basis. The operationalized criteria in Chapter 3 are developed using a modified-RAND Delphi procedure, in which a systematic literature search and input from an international multidisciplinary panel composed of experts from involved medical disciplines is combined. These criteria can be used in daily clinical practice, by antibiotic stewardship teams and by attending physicians and are an important step towards improving the (national) uptake of an early switch from iv to oral antibiotics. It is recommended that these consensus criteria are being disseminated. Especially since a striking lack of awareness of iv to oral switch guidance is one of the barriers to iv to oral switch programs. When introducing such programs it is recommended to annually audit them to measure success or areas of improvement. The developed criteria may facilitate auditing iv-to-oral antibiotic switch practices on a specific ward or hospital, and enable comparisons between hospitals or regions. To improve iv-to-oral antibiotic switch practices in an effective and sustainable manner, more is needed than instructions and guidelines. A CDSS that reminds physicians to switch their patients to oral antibiotics has great potential. Earlier developed CDSS for this purpose have been developed using local criteria. Other CDSS for this purpose use very general rules, such as a certain duration of iv therapy or/and an active order for scheduled oral medications or an oral diet. This limits their general applicability, acceptance and specificity. The developed consensus criteria can be used to build a system, which is general applicable.

Iv to oral antibiotic switch alerts in clinical practice
A CDSS, based on consensus criteria (Chapter 3), is effective in facilitating antibiotic stewardship teams by offering a preselection of iv to oral switch candidates (Chapter 4). It is plausible to assume that this saves time and work in comparison with a situation in which antibiotic stewardship teams have to assess all admitted patients with an iv antibiotic on the possibility to switch from iv to oral antibiotics. About one third of admitted patients in a hospital receive antibiotic drugs (Chapter 2), which in our hospital were in total 337 patients (on both May 4 and May 16 2013) (Chapter 2). A daily assessment by an antibiotic stewardship team of all these patients is not feasible. A CDSS has the potential to preselect patients which qualify for iv-oral switch. For instance, our CDSS algorithm generated 840 alerts in 773 different patients during a period of 4 months, which is a number of patients that is feasible to assess for the antibiotic stewardship team; an average of 10 per day (standard deviation: 4). The number of clinical relevant and useful alerts can be increased by further optimization of the CDSS algorithm, for example by introducing the possibility to switch off/delay alerts for a certain duration for patients with infectious diseases for which a relatively long duration of iv therapy is needed (for example endocarditis). Our CDSS algorithm was not able to exclude or delay an alert for these patients, because criteria that contain a diagnosis could not be translated into a computer interpretable format (Chapter 3). Improving this is important, because over alerting can lead to alert-fatigue, a well-known problem related to an active CDSS.

Since in the Netherlands, in approximately 60% of the hospitals funding for antibiotic stewardship programs is lacking and if a budget is provided less than indicated by the staffing standard, a CDSS that saves work seems ideal. However, only 9% of Dutch hospitals have dedicated IT support for stewardship teams. Improving this will create more possibilities for the development and improvement of several CDSSs.

A CDSS for empirical antibiotic therapy

In addition to the active CDSS algorithm for iv to oral switch, we developed a passive CDSS for empirical antibiotic therapy. It is recommended to assess usability before implementation of a CDSS, especially with a lot of user system interaction, to improve its adoption, effectiveness and safety. For this, the augmented UAF (systematic framework developed by Khajouei et al.) can be used, as it is a simple, but effective way to identify usability problems and prioritize system redesign effort (Chapter 5). For ultimate system usability, iterative usability evaluation during the development and implementation of CDSS is important. Residents do not always follow advice of the CDSS without considering the correctness and applicability of the recommendation (Chapter 5), which is positive, because these systems should always be seen as an aid. The definite decision is always at the physician’s discretion and too much reliance on a CDSS may have a negative impact on the skills of the user. There may be good reasons to deviate from a guideline, so also from a CDSS based on a guideline. In this light it is also important to note that the uptake of CDSS in hospitals is hindered by the concern of clinicians that their professional autonomy or critical thinking may be reduced by the system. Related to this concern is the fear that the advice of a CDSS can be used against them in medico-legal procedures. This goes both ways; on the one hand the CDSS clearly reveals the discrepancy between (several) guidelines and contextualized decisions. Thus not following or not using the CDSS can be used against the physicians for which this discrepancy is important in the decision making (for example when the kidney function or BMI of a specific patient should be taken into account). On the other hand introduction of technology can cause new types of error. Thus, blindly following the CDSS advices can be used against the physicians. For this reason it is important that the legal consequences of (not) following CDSS advices become clear in context of liability.

One of the elements that have to be considered to avoid usability problems is to retrieve as much information as possible automatically from the hospital information system (Chapter 5). Several challenges exist in achieving this, such as required data not being available in the EHR (Chapter 2) or data being unreliable. An example of unreliable data in the EHR is the (alleged) presence/absence of an allergy. Another challenge is data not always being coded in a standardized terminology in the EHR. For example, we found that we could not retrieve the diagnosis automatically from our EHR, because clinical users use free text to record the diagnosis. For this reason the diagnosis is one of the data that had to be entered manually. This however could also be seen as an advantage in light of maintaining physicians ‘autonomy’ as this gives control over the CDSS.

Since not all antibiotic guidelines were present in the CDSS, we assessed in how many patients the system could be used (data not published). During the implementation period of the CDSS, 3,349 patients received at least one antibiotic for systemic use. From these 3,349 patients we randomly selected 248 patients to manually check whether the patients had one of the diagnosis included in the CDSS. In this proportional stratified random sample of 248 patients all departments were reflected. From these 248 patients, 100 patients received at least one antibacterial drug as empirical antibiotic therapy for one of the diagnosis included in the CDSS, which is equivalent to 40.3% (95% CI 34.3-46.3) of patients. That means that of the 3,349 patients that received at least one antibiotic for systemic use, 1,349 patients received this antibiotic(s) as empirical antibiotic therapy for one of the included diagnosis in the CDSS. The CDSS was used 184 times, of which 15 times for patients who did not have any signs of infection or were not admitted to the hospital (trying out/testing the system). Thus the system was used for 12.5% (184-15)/1,349) of patients for which it could be used. This low use existed despite the development by a multidisciplinary team, usability testing (Chapter 5), followed by improving the system and promotion of use on a regular basis in clinical practice. A low level of CDSS use is found in several studies. In one study, the study design was adjusted because of potential CDSS underutilization. In this study the researchers performed preintervention interviews, which showed that clinicians perceived excessive time would be required for the use of the CDSS. For this reason an antimicrobial management team was organized that used the CDSS and made the clinicians aware of CDSS recommendations. This clearly illustrates the problem with the use of CDSSs. In this study an Internet-based decision support tool was used for empirical antibiotic advice for community acquired pneumonia. A detailed description of the tool is missing in the study publication. The system was not connected to an EHR. Therefore automatic extraction of relevant information from the EHR was not possible and all relevant information for an antibiotic advice had to be entered in the system. This may have contributed to the perception of the clinician that excessive time would be required for the use of the CDSS. A relatively high rate of adoption (57.5%) was demonstrated in a study of CDSS/clinical prediction rules (the Heckerling Clinical Decision Rule for pneumonia and the Walsh clinical prediction for streptococcal pharyngitis) in primary care. The authors speculate that this high rate is because of their development process which was comprehensive and user-centered. Like us, they performed usability testing and collaborated with a multidisciplinary team. Beside this, they performed focused user training on clinical decision support and their multidisciplinary team included also clinical decision support specialists. Our CDSS was developed by a multidisciplinary team, composed of an infectious disease physician, a clinical pharmacist with infectious diseases training experienced in clinical decision support, a clinical microbiologist, an infection control professional and a hospital epidemiologist. To improve use, we recommend to add an expert on implementation of CDSS to the development team. This way implementation issues can be taken into account in an early stage of CDSS development. Another important aspect that has probably contributed in a positive way to the higher adoption rates of the earlier mentioned CDSS/ prediction rules in primary care, is the blending of the system in the clinical workflow. The clinical prediction tool appeared on the screen when the provider entered one or more relevant keywords in certain fields during a clinical encounter. We found a low use of our CDSS for empirical antibiotic therapy, which may be explained by it being a passive and new system. At the moment of empirical antibiotic prescribing, physicians had to realize that a CDSS could assist them and find the link to the system in our EHR. A reminder in the CPOE and direct access from the CPOE (fitting in the work process) could be an option to improve CDSS use. A system not being integrated into workflows is one of the barriers that is often described in literature. To improve integration into workflow we recommend making CDSS also accessible at the point of care as smartphone application. Especially since smartphone use among clinicians is increasing.

Multiple other barriers to use a CDSS are described in literature, such as the earlier mentioned fear that such a system will compromise professional autonomy. Other barriers which are mentioned in the literature, include: a poor usability, and absence of technical support and training. Although CDSSs have high potential in improving guideline adherent therapy their low use is a major barrier in reaching their potential. In addition, because of this low use, measuring clinical outcome is challenging, given the fact that a large sample size is needed to find significant differences in important outcome measures.

Our CDSS and other CDSS for antibiotic therapy: differences and similarities

CDSSs to support appropriate use of antibiotics have been developed since 1980. These systems have targeted a variety of aspects, such as optimizing antimicrobial dosing or supporting antimicrobial de-escalation. Most of these systems however focus on antimicrobial prescribing. CDSSs to promote an early iv to oral antibiotic switch have been developed before. However, none of the CDSS for an early iv to oral switch are based on international consensus criteria. Earlier developed CDSSs to promote an early iv to oral antibiotic switch have been developed using local criteria or very general rules, such as a certain duration of iv therapy or/and an active order for scheduled oral medications or an oral diet. This limits their general applicability, acceptance and specificity.

Many CDSSs that are developed for more general antibiotic prescription focus on a specific infectious disease, mostly respiratory tract infections or are developed/ implemented/tested only in or for a specific department, such as the emergency department or the intensive care, or in a limited number of departments. In our CDSS for empirical antibiotic therapy, the most common infections were included and our CDSS was implemented in a tertiary hospital with a wide variation of departments. Several CDSSs have been specifically developed to improve empirical antibiotic therapy for hospitalized patients. The CDSSs, specifically developed for empirical antibiotic therapy differ on varying aspects. Some systems use expert rules to predict the pathogen’s susceptibility to antibiotics, using antibiotic susceptibility profiles from patients with similar characteristics, but don’t take into account for example the antibiotic resistance history of the patients of interest, or presence of neutropenia, like our system does. Other systems use causal probabilistic networks to predict the probability of a bacterial infection, site of infection and pathogens and their susceptibility to antibiotics. The CDSS we developed generates antibiotic advices based on relevant guidelines. Like many other CDSS for empirical antibiotic therapy, input of the physicians was needed in our system for the generation of an antibiotic advice.

It is important to note however that a good comparison between the different CDSSs is difficult, because of a heterogeneous and disjointed approach to reporting CDSS interventions. Following Rawson’s framework ensures a structured overview of many important aspects of a CDSS intervention. Use by others is recommended and will enable a more easy comparison between the different CDSSs. Several important aspects are not included in this framework, such as: the composition of the team that developed the CDSS, type of CDSS (active or a passive), guidelines on which the CDSS is based, rationale for using these guidelines, commercial or noncommercial CDSS, setting for which the CDSS was developed. We propose that these key components should also be considered when reporting on CDSS interventions.

Implications for future research

Iv to oral switch algorithm
The iv to oral switch algorithm proved to be a valid instrument to identify IVOS candidates. With this algorithm a report containing all eligible patients for IVOS was automatically generated on a daily basis and directed to the ID specialist of the Antibiotic Stewardship Team. The ID specialist then assessed whether the patient could switch to oral therapy and contacted the treating physician if this was possible. It is important and interesting to use future research to assess the number of patients that have been switched to oral antibiotics by the treating physician after this contact. Future research should focus on the effect of this algorithm on outcomes, such as length of stay, readmission rates and costs.

A report with iv to oral switch candidates saves work in comparison with a situation in which the ID specialist has to assess all patients with an iv antibiotic on the possibility to switch to oral antibiotics. However, alert fatigue may still be a problem. For this reason future research should take this aspect into account. With advanced coding of data in EHR the efficacy of the CDSS can be improved, thereby also reducing this risk of alert fatigue. This also provides the opportunity to direct the alerts generated by this algorithm directly to the treating physician in the future. We expect the additional value of the algorithm to be even more in a setting with no or a less active ID service. Research is needed in these settings to evaluate this assumption.

CDSS for empirical antibiotic therapy
The 6-month time period may have been too short to improve use/uptake of the CDSS for empirical antibiotic therapy, since it has been shown that continued use of these systems improves their acceptance. Future research should aim to assess use and uptake of CDSS for a longer duration. To gain more insight in the specific barriers to CDSS use in our hospital further research is useful. With an understanding of these specific barriers a specific action plan may be developed to generate interest and improve use of this CDSS and the uptake of its advice. Improving CDSS use and uptake is important to achieve its full potential and also to justify the costs that are associated with development and maintenance of these systems. With more use of the CDSS (with a longer implementation/study period and tackling of specific barriers) a larger sample size can be created for future studies that measure differences in important outcome measures, such as the susceptibility of cultured micro-organism for the prescribed antibiotic before and after implementation of the CDSS. A multicenter approach is recommended to facilitate appropriate sample size. Besides an appropriate sample size this also provides the option to assess differences of implementation in different settings. We expect that the developed CDSSs will be of more value in a hospital with less active ID consultancy systems. In addition, given the fact that first-line care and long-term care include the highest antibiotic prescription rates the implementation and research of CDSS to improve antibiotic use should ideally also be extended to these settings. Because new resistance mechanisms emerge and spread globally, implementing and assessing CDSS in low- and middle-income countries (where antimicrobial resistance is high) is an important future target.

To help increase the use of a passive CDSS, such as our CDSS for empirical antibiotic therapy, enabling access at the point of care as a smart phone application is recommended. Especially since smartphone use among clinicians is increasing.

Future research should also focus on smartphone based CDSS, because smartphones are increasingly being used by physicians. It is interesting to study the utilization, acceptability and impact of such a CDSS. The Erasmus Medical Center will participate in an international randomized, multicenter clinical trial evaluate the impact of an antimicrobial stewardship smartphone application for the hospital setting.

Artificial intelligence / Machine learning CDSS
The CDSSs described in this thesis, like most current CDSSs, use algorithms/rules to generate alerts (iv to oral switch algorithm) or an advice. Changes in guidelines have to be followed by manually changing the algorithms/rules on which the CDSS is based. In the world of health technology another field, that has drawn increasing interest, are CDSSs using artificial intelligence/machine learning. With artificial intelligence/ human learning, systems are able to create/add algorithms/rules themselves. These systems learn automatically from data, and their performance is therefore depending on the quantity and quality of available data. For this reason most of these systems in infectious diseases target domains with high quality and large databases, such as certain patient populations in the ICU. With the availability of more of such databases this is an important future development to focus on and to study. It is important that attention is paid to the transparency of algorithms/rules on which these systems are based. Especially since an important barrier to use a CDSS is the concern of clinicians that their professional autonomy or critical thinking may be reduced by the system. With systems that are able to manage and act relatively fast on large amount of information, it is imaginable that these systems may be equal or better in decision-making than clinicians (in the future), and this concern becomes even more pertinent.

Final remarks
The studies described in this thesis resulted in valuable insights in relevant aspects involved in the development, validation and implementation of a clinical decision support system to improve antibiotic prescribing. The developed (active) iv to oral switch algorithm proved to have good test performance and the added value of this algorithm in identifying iv to oral switch candidates was shown. Important general usability problems that have to be taken into account when developing a passive CDSS for empirical antibiotic therapy have been described. Finally the usefulness of a systematic framework for reporting CDSS interventions was shown.