Elizabeth Van Veen

General Conclusions

The general aim of this thesis was to find an answer to the question whether a nationwide long-term benchmarking collaborative of the operating room (OR) departments of all eight University Medical Centers (UMCs) in the Netherlands could lead to improvements in overall OR management. For this purpose, several studies were conducted:
 one exploratory study combining qualitative and quantitative research methods;
 three descriptive studies based on a substantial amount of multicenter data;
 and six quasi-experimental studies to determine the effect of specific interventions in different OR processes.

Benchmarking OR Departments in the Netherlands

To investigate whether the collaborative and long-term approach of the Dutch OR benchmarking initiative has led to benefits in OR management, an evaluation frame based on literature 1, 2 was applied in a mixed-methods study design (Chapter 1). Collaborative benchmarking has benefits different from mainly performance improvement and identification of performance gaps. It is interesting to note that, since 2004, the OR benchmarking initiative still endures after already existing for more than ten years. A key benefit was pointed out by all respondents as ‘the purpose of networking’. The networking events organized by the collaborative were found to make it easier for participants to contact and also visit one another in the OR departments of the eight university hospitals. Apparently, such informal contacts are helpful in spreading knowledge, sharing policy documents and initiating improvements in overall OR management. One reason for this is that they could be used to discuss the tacit components of best practices, that are hard to share in more formal communication media. Respondents were satisfied with the content of these meetings and with the exchange of knowledge in an informal manner, the exchange of experiences including sharing best practices as well as discussing worries and today’s challenges in OR management. It enables understanding and learning from each other. These findings corroborate the idea of De Korne et al. 1, 2 that participating in benchmarking offers other advantages, such as generating discussions about how to deliver services and increasing the interaction between participants.

During the initiation phase of the benchmark collaborative, a considerable amount of time (two years) and effort was undertaken by the steering committee to develop a collaboration agreement. This agreement created the foundation for trust and confidentiality between the eight participating partners, because confidentiality and ownership of benchmarking data are two delicate and important parts of the agreement. These first years were also seized by the development and harmonization of definitions of performance indicators. Common definitions are an essential base for external benchmarking 3, 4. The long-term commitment of the eight centers to the OR benchmark collaborative is exceptional, yet might also be necessary to build and maintain trust between the centers, and also be necessary for uniform data registration and harmonization of indicator definitions.

Benchmarking is defined as a ‘continuous process’ and encourages the use of a continuous quality improvement model (the PDSA cycle). Although this OR benchmark initiative, as many benchmark initiatives, started with a stated aim to improve, actual (measurable) quality or performance improvements are not necessary for this initiative to endure. These findings further support the idea of De Korne et al. 1, 2 that benchmarking is relying on iterative and social processes in combination with structured and rational process of performance comparison. The relatively limited focus on OR utilization in this benchmark seems to be a starting point for exchanging a variety of information and experiences considering the structure, process and performance of OR departments. More attention needs to be given to the relation between benchmarking as instrument and the actual performance improvements realized through benchmarking in the local UMC’s. A collaborative approach in benchmarking can be effective because participants use its knowledge-sharing infrastructure which enables operational, tactical and strategic learning. Organizational learning is to the advantage of overall operating room management. Benchmarking seems a useful instrument in enabling hospitals to learn from each other, to initiate performance improvements and catalyze knowledge-sharing.

DESCRIPTIVE STUDIES

The findings presented in Chapter 2 are important for hospital management and surgical teams, since they clearly suggest that improving the utilization of OR time should be focused on reducing the amount of underused (empty) OR time at the end of the day. This performance indicator has the strongest influence on raw utilization (%), followed by late start and turnover time. The relationships between the three ‘nonoperative’ time indicators were negligible. The impact of the partial indirect effects of ‘nonoperative’ time on utilization were statistically significant, but relatively small.

Based on this study, late start, turnover time and underused time were ‘stand-alone’ aspects with an important direct influence on raw utilization and only a minor influence on each other. We were unable to verify the earlier reported ‘trickle down’ effect, caused by late start and resulting in an increased delay as the day progresses. Potential solutions and interventions to address the issue of underused OR time are: improving the prediction of the total procedure time of surgical cases; altering the sequencing of scheduled operations and altering patient cancellation policies.

Chapter 3 identified that in a university hospital environment a quarter of Total Procedure Time (TPT) is Anesthesia-Controlled Time (ACT). As a result, it was concluded that grossing up the predicted Surgeon-Controlled Time (SCT) by 33% to account for the expected ACT, can improve the prediction of TPT if this methodology is adopted. This confirms that employing a fixed time period for ACT (e.g. 20 minutes), is unsuitable because like SCT, ACT is subject to variability. The results affirm that ACT is a considerable part of TPT, which should be scheduled just as realistically as SCT. Robust OR schedules need to anticipate SCT as well as ACT. ACT should be predicted apart from SCT, as a separate time period instead of one combined predicted time period for TPT. More accurate prediction rules may lead to less under- and over-utilized OR time and a reduction of case cancellations.

Thirty-three percent is a higher proportion than reported in earlier research. This recommendation will improve OR scheduling, which might result in the reduction of under- and over-utilized OR time as well as a reduced amount of case cancellations, and therefore in more efficient use of limited OR resources. Recently one Dutch UMC, the Academic Medical Center (AMC) in Amsterdam, adopted a system of scheduling ACT based on predetermined time frames per anesthetic technique. These time frames were differentiated according to the quantity of anesthesia monitoring needed and the complexity of the patient. The implementation of this new scheduling method started at the end of 2012 and we have currently conducted further research to assess the value and effects of this methodology in practice (Chapter 6).

Chapter 4 assessed the effect of surgeons and anesthesiologists on the prediction of OR time. Previous work showed the importance of the surgeons and the surgical team for prediction of OR time 11-13. However, this study is the first to show the actual effect sizes of surgeons and anesthesiologists on OR time in a multivariate model corrected for various known predictors. The actual effect of individual surgeons and anesthesiologists is rather small. The overall effect of the first surgeon could explain only 2.7% of the total variation in OR time. Nonetheless, including the individual members of the surgical team in the prediction model improved its accuracy and reduced the over- and underestimation of OR time.

INTERVENTIONAL STUDIES

To identify whether the Dutch OR benchmarking collaborative has led to sustainable improvements in OR management in time, several quasi-experimental studies were performed on specific relevant subjects. To begin, a ‘late start’ or first-case tardiness (Chapter 5) is still a common source of frustration in the OR department. On an overall level of eight UMCs in the Netherlands, 43% of all first operations start at least 5 minutes later than scheduled and 425,612 minutes are lost due to this annually, which has a respectable economic impact.

On the other hand, the results show that on an overall level of all UMCs first-case tardiness has decreased since 2005 and four centers implemented successful interventions to reduce tardiness. These UMCs showed a stepwise reduction in variation of first-case tardiness, in other words a decrease in IQR during the years, which indicates an organizational learning effect. ANOVA with contrast analysis shows that a marked change occurred at the time of the intervention, which indicates the success of their interventions. First-case tardiness occurs on a daily basis in Dutch UMCs and this has a sizeable impact on OR efficiency. Yet, this study shows that benchmarking can help to overcome this by exchanging best practices and prevent ‘reinventing the wheel’ through organized learning and networking. In accordance with De Korne et al. this research further supports the idea that benchmarking is highly dependent on social processes and a learning environment parallel to a structured and rational process of sharing performance data. Transfer of knowledge is one of the major targets of the Dutch OR Benchmarking collaborative. During the two-monthly organized multidisciplinary focus-group study meetings and the yearly national invitational conference, targets and goal setting are a matter of discussion and presentation. The overall data presentation is complemented by best practices from different hospitals. Thus, knowledge transfer is performed according to two routes: data analysis and best practice sharing. Overall, this study shows that benchmarking can be applied to identify and measure the effectiveness of interventions to reduce first-case tardiness in a university hospital OR environment.

In 2012, in AMC Amsterdam the OR management decided to implement a new strategy regarding realistic scheduling (Chapter 6). This new method comprised of developing predetermined time frames per anesthetic technique based on historical data of the actual time needed for anesthesia induction and emergence. In total seven so-called ‘anesthesia scheduling packages’ (0 – 6) were established. Several options based on the quantity of anesthesia monitoring (e.g. intubation, arterial line, central line) and the complexity of the patient were differentiated in time within each package: e.g. general anesthesia with tube 30 minutes or awake fiber-optic intubation with epidural 80 minutes.

The most prominent results to emerge from this study are the reductions in prediction errors as well as in the number of case cancellations since the implementation of this new scheduling method specifically for anesthesia-controlled time. Simultaneously, the number of cases performed, increased along with an increase of mean total procedure time. These findings provide important implications with respect to OR scheduling in a university hospital setting, since they affirm that anesthesia time is a considerable component of total procedure time and should be scheduled just as realistically as and separate from surgeon-controlled time. Scheduling the two major components of a procedure (ACT as well as SCT) more accurately, results in less case cancellations and lower prediction errors. This may lead to more patient satisfaction and a more efficient use of limited and expensive OR resources.

In recent years, there has been an interesting development in one of the Dutch UMCs, i.e. Radboud UMC in Nijmegen, regarding operating room scheduling, which received a lot of attention in the OR benchmarking network. This intervention comprised of the implementation of ‘cross-functional OR scheduling teams’ (CFTs) for every surgical department. Every CFT is headed by an anesthesiologist and also includes a surgeon, a scheduler, an OR nurse, an anesthesia nurse, a recovery room nurse, and a nurse from the ward. One a week there is a team meeting to discuss the OR schedule of the next week and to evaluate the OR performance of the previous week, in terms of utilization, case cancellations and other factors interfering with ‘smooth’ planning. The CFT examines the complete OR program, day by day and members inform their colleagues regarding all relevant issues needed for optimal planning and safety. The CFT was given full ‘mandate’ (or ‘authorization’) by the Head of the Department of Operating Rooms and by the Head of the Department of Anesthesiology, to make operational decisions regarding the OR schedule and to make alterations to the submitted OR schedule (e.g. change the order of cases or to not approve of a submitted schedule when the scheduled time exceeds the 8h OR block time allocated to a specific surgical department).

Three studies were conducted on the subject of CFTs: a single-center qualitative case study (Chapter 7), a single-center study with a longitudinal quantitative research design (Chapter 8), and a multi-center study with a quasi-experimental time-series design (Chapter 9). The findings of these three studies highlight the importance of team-based approaches and the need to improve multidisciplinary collaboration between healthcare professionals. The best-performing teams could identify bottlenecks at an earlier stage and were able to solve these bottlenecks. Reduction of uncertainties – by means of optimizing multidisciplinary collaboration – will improve OR scheduling (Harders et al., 2006). In other words, CFTs are assumed to have a self-regulating capacity to identify bottlenecks and to improve continuity. The teams gained insight into their performance using several performance indicators. Consequently, through collaboration, these teams could both control and learn (Chapter 7).

Moreover, our research identified a gradual improvement in OR utilization (2005 – 2011) for two specific surgical departments in Radboud UMC. Results showed a significant reduction in variation – a decrease of interquartile range during the years – of utilization since the implementation of CFTs and a significant increase in mean raw utilization every year. The stepwise reduction of variation indicates an organizational learning effect and more consistency in OR scheduling (‘in control’). The increase of mean raw utilization during the years and the reduction of uncertainty are indicators of more efficient utilization of scarce OR time (Chapter 8). Furthermore, the multicenter study strengthened the idea that multidisciplinary collaboration in CFTs during the perioperative phase has a positive influence on OR scheduling and the use of OR time. Radboud UMC had the highest median raw utilization – 94% versus 85% group median of six other UMCs – during the years 2005 up to and including 2013. An interesting additional finding is that other national databases, concerning mortality rates, support the idea that introducing CFTs can be important for improving the quality and safety of care, since Radboud UMC showed the lowest mortality and lowest complication rates (Chapter 9).

The last interventional study (Chapter 10) in this thesis actually covers two important topics: OR capacity for emergency surgery and the value of computer simulation modeling in the complex environment of the OR department. The study showed that in daily practice a dedicated OR for emergency cases is preferred over the approach of evenly reserving capacity for emergency surgery in all elective ORs, in performance terms of raw utilization (%), ‘overtime’ and the number of ORs running late. Moreover, the additional data concerning case cancellations indicates that a dedicated emergency OR has the benefit of less case cancellations. The study assessed the change in efficiency parameters of the problem that was simulated by Wullink et al. 15. The results of their specific simulation study led to closing of the emergency OR in the Erasmus MC and evenly allocating emergency capacity in all elective ORs. However, the results of this recent study are partially in contrast with the simulated results: in retrospect, overtime significantly increased after closing the emergency OR. These recent results were based on the empirical data. OR utilization did increase, nevertheless, this increase was lower than the increase in utilization found in both control UMCs without the specific intervention of closing the emergency ORs. Therefore, these recent results do not support the earlier published conclusions 5, 6, 11, 12 that distributing free OR capacity for emergency surgery evenly over all elective ORs performs better than dedicated emergency ORs on measures regarding the efficient use of scarce OR time.

Furthermore, the results corroborate the idea of Fone et al. 16 that computational modeling experiments are important to support evidence-based policy making in hospital care but they are not able to reflect all complexities of organizations, such as OR departments, and the people working there. In other words, the theory is right but practice is different. A reasonable approach for future interventions in the multifaceted OR environment could be to follow-up policy decisions that are based on simulation modeling results with a robust controlled time-series research design using empirical data, in which the change in parameters is carefully assessed. To further support ‘Evidence-Based OR Management’, every intervention should be followed-up by an evaluation (‘check’) and, if needed, followed by adjustments and new actions (‘act’), in line with the well-known Deming’s cycle.