Rutger Quispel

The aim of this thesis is to explore the use and quality of care delivered by endoscopic ultrasonography in clinical practice.

In Chapter 1 the introduction of endoscopic ultrasound (EUS), dissemination of the technique and developments in the use of EUS over the past 40 years are described. The field of endoscopic ultrasonography has grown tremendously over the 4 decades of its existence. Concerns about interobserver-variability, practice variation, and quality of training in EUS have been raised in several publications [1-5]. To date these topics have received little attention in the framework of scientific studies. As a matter of fact, until recently, quality in EUS was not clearly defined. In recent years, quality improvement programs have been initiated by national and international endoscopy societies [5-8]. These have provided practitioners with key performance indicators (KPI), and performance targets. In this thesis, we explored the use of performance indicators to measure performance in both EUS-guided tissue acquisition (TA) of solid pancreatic lesions, and EUS in patients suspected to have bile duct stones in clinical practice.

Chapter 2 describes the use and yield of linear endosonography in patients with suspected bile duct stones in community hospital practice. In line with previous publications, EUS serves as an excellent tool to select patients for endoscopic therapy. EUS rules out bile duct stones in up to 63% of cases, thereby preventing unnecessary endoscopic retrograde cholangiography and endoscopic sphincterotomy (ERC+ES), and avoiding its complications. However, in some patients, EUS detects bile duct sludge, or microlithiasis, i.e. stones <3 mm. Both sludge and microlithiasis may spontaneously pass into the small bowel, but might also lead to complications such as pancreatitis. The clinical relevance of microlithiasis and bile duct sludge remains to be elucidated. This might lead to a further reduction of the need for endoscopic therapy and its related complications. In Chapter 3, we investigate the interobserver variability among 41 endosonographers evaluating EUS videos of bile duct stones, microlithiasis (i.e. stones < 3mm), and bile duct sludge. We also reported whether the EUS diagnosis would prompt the endosonographer to advise to perform endoscopic therapy (ERC + ES) or refrain from it. Furthermore, we investigated whether the degree of experience of the endosonographer influences diagnosis and treatment decisions. For bile duct stones and a “clean” bile duct, the interobserver agreement was moderate. For both microlithiasis and bile duct sludge, there was only slight agreement. There was an almost perfect agreement regarding the need for endoscopic therapy in cases with bile duct stones or cases with a normal “clean” bile duct. In cases with sludge or microlithiasis, there was no consensus regarding the need for endoscopic therapy. Practitioners’ experience appeared not to influence neither EUS diagnoses nor treatment decisions. The only moderate and slight interobserver agreements on stones and sludge likely reflect both the limited but variable definitions of bile duct sludge in literature, and variations in education and training of contributing endosonographers. In order to get more insight in the clinical relevance of bile duct sludge, we studied a large prospective database including over 40.000 EUS procedures recorded in a time-frame of 20 years from a tertiary referral center, in Montreal, Canada. In Chapter 4, the prevalence of bile duct sludge is described in patients from this database who had intermediate or high probability of bile duct stones, according to the clinical prediction tool of the American Society of Gastro-Enterology (ASGE). In approximately one out of 25 EUS procedures (4%) performed for this indication, bile duct sludge is diagnosed. The clinical relevance of this finding warrants further study. In the second section of this thesis, the use and performance of EUS-guided tissue acquisition (EUS-guided TA) of solid pancreatic lesions is investigated. Chapter 5 starts with a retrospective study of the yield of 20 EUS-guided TA procedures from four community hospitals, followed by the initiation of a prospective registry of these procedures, and the founding of a regional EUS collaborative aiming to improve the outcome of these procedures. From 2015 onwards, this collaborative quality in endosonography team (QUEST) organized three meetings annually. Topics discussed at these meetings are: guidelines and relevant literature, challenging cases, and feedback on KPI of EUS-guided TA of solid pancreatic lesions per center. Prior to meetings, feedback regarding KPI of EUS-TA is provided to each collaborating practitioner. At meetings, performance data are discussed in an anonymized fashion, in order to create and maintain a safe learning environment for quality improvement. This deliberately leaves the responsibility for the quality delivered with the practitioners from the individual hospitals. Initial retrospective data revealed significant practice variation regarding KPI among contributing hospitals. The first report on the proceedings of QUEST describes prospectively collected data from January 2015 to September 2016. It concludes that in relatively low-volume community hospitals, regular meetings with feedback on KPI leads to a reduction of practice variation and an improvement of quality up to the desired level. In Chapter 6, the diagnostic yield and agreement on fine needle specimens (FNA) from solid pancreatic lesions were assessed, and a comparison of the traditional “smear technique”, and two separate liquid based cytology (LBC) techniques was made. The diagnostic value of two LBC techniques combined (Cellblock and Thinprep), proved superior to traditional “smears” when assessing the yield of a single pass EUS-guided FNA of solid pancreatic lesions. This indicates that LBC adds to the diagnostic value of the traditional “smear technique”, and should be considered especially when rapid on-site evaluation of samples is unavailable to ensure sample adequacy. Since LBC also allows for additional testing, such as immunohistochemistry or next generation sequencing (NGS) we recommend the use of LBC in all cases of EUS-guided FNA of solid pancreatic lesions. In Chapter 7, the proceedings of the regional EUS collaborative QUEST, concerning EUS-guided TA of solid pancreatic lesions from 2015-2018, are reported. This chapter introduces close collaboration with the pathology departments in the contributing hospitals, as well as the use of cumulative sum (CUSUM) analysis, a graphical method to plot learning curves. This study used CUSUM curves to assess trends in quality over time, and explored potential benefits of CUSUM curves as a feedback-tool. Throughout the 4 years of this study, all KPI improved. However, not all ASGE-defined KPI are consistently met in each center yet. Therefore, feedback on performance as well as feedback on potential ways to improve the yield of EUS-guided TA should be an ongoing process. Learning curves were proven a valuable adjunct to tables with numbers for providing such feedback. They allow determination of best practices and comparison amongst peers at a glance and provide additional learning opportunities. Previous studies using learning curves for monitoring development and training of advanced endoscopy trainees, already taught us that numbers of procedures performed are a poor measure of competency [4,9-13]. Similarly, our study shows that numbers of procedures performed are a poor measure of quality. Our study results also indicate that performance of EUS-guided TA differs considerably amongst centers and even individual operators. It therefore is the responsibility of each operator to be informed about the quality of his/her own performance and CUSUM curves are an excellent tool to facilitate this. Chapter 8 summarizes quality and practice variation regarding EUS-TA of solid pancreatic lesions nationwide in the Netherlands. For this purpose, we retrospectively analyzed all resected pancreatic ductal adenocarcinoma cases from the national pathology database (PALGA), and all pathology reports describing the evaluation of tissue samples from EUS-TA procedures performed in these patients. Again, significant practice variation regarding KPI of EUS-TA was seen. Although the sample adequacy rate (RAS) meets the predefined performance target of 85% in all dedicated pancreatic surgery centers, the performance target ‘sensitivity for malignancy >85%’, was met in only 9 out of 17 of these centers. This study demonstrates opportunities for improvement in quality of each stage of these multi-step procedures. Consorted action from endosonographers and pathologists should be prioritized, especially since the emergence of neo-adjuvant therapies and future “targeted therapy”, are expected to further increase the importance of a “first-time right” diagnosis in these patients. The data from the current study may serve as a reference for a future multidisciplinary audit aiming for continuous improvement of quality of care in patients with suspected pancreatic malignancies.

Discussion and future perspectives

Quality is doing the right thing in the right way [14]. This means a procedure is performed for the right indication, by a trained operator using the appropriate methodology aiming for the best possible outcome: “first time right”, including the lowest possible chance of adverse events, and a decreased need for additional care.

Quality of EUS is important to patients since it can limit the burden of being submitted to additional care, including the management of complications, (repeated) endoscopic procedures and/or prolonged hospitalization. This was illustrated in chapter 2, where EUS prevented the unnecessary performance of endoscopic retrograde cholangiography and endoscopic sphincterotomy (ERC+ES) and it’s complications in 63%. Another example is the improved quality of EUS guided TA in patients with suspected pancreatic cancer reducing the need for second procedures to establish a pathological diagnosis from 18% to 7% in Chapter 5.

What is EUS performed in “the right way”? How can we measure performance in EUS?

Quality measurements
Volume (number of procedures performed annually) has long been the only measure of quality for many procedures, first because it’s easily measured, and second because other (real) quality measures were unavailable. Many publications have highlighted the correlation between outcomes of surgical or endoscopic procedures and case volumes of centers and individual practitioners. The inverse relation between center volume and mortality, established for a multitude of surgical procedures in the United States in 2002, is an example of this [15]. For ERCP it was confirmed that low-volume providers (performing less than 25 ERCPs annually) have a significantly higher failure rates [16].

Although volume is obviously related to outcome, it is at best, a surrogate marker of performance with significant limitations. Main limitation of volume as a performance indicator is the fact that changes in volume will not directly lead to changes in outcome. Therefore, volume is unfit for repeated or continuous quality measures through time. Ideally, performance indicators connect specific procedural characteristics to procedural outcome and related patient burden. Examples of more adequate performance indicators are the adenoma detection rate (ADR) for quality in colonoscopy, and bile duct cannulation rate at ERCP. The rate of adenomas detected by a single endoscopist at colonoscopy (adenoma detection rate: ADR), was proven an independent predictor for a patient’s risk of developing colorectal carcinoma 6-36 months after colonoscopy [17]. A low bile duct cannulation rate at ERCP will likely to lead to an increased need for repeated procedures.

The American Society of Gastrointestinal Endoscopy (ASGE) divided quality indicators for EUS into three categories: 1. structural measures, 2. process measures, and 3. outcome measures. Structural measures mainly relate to the environment in which the procedures are performed. Process measures assess performance during the procedure (for example training and experience of the endoscopists involved, or rate of EUS procedures performed for proper indications). Outcome measures assess the results of care provided and are therefore truly relevant for patients. The ASGE proposed nine specific quality indicators for EUS, of which three outcome measures were designated priority performance indicators: diagnostic yield of malignancy (DYM>70%), and sensitivity for malignancy (SFM>85%), as well as incidence of adverse events after EUS guided TA [8].

The European Society of Gastrointestinal Endoscopy defined two key performance measures for EUS: 1. Proportion of antibiotic prophylaxis prior to EUS guided TA of cystic pancreatic lesions, with target >95%, 2. Proportion of adequate tissue samples obtained by EUS guided TA, target >85%. Adequate documentation of EUS landmarks was designated a minor performance measure [6].

The quality measures/performance indicators reported on in this thesis (in chapters 5, 7 and 8) have in common that they are outcome measures likely to have a direct impact on patient burden. When EUS-TA is performed aiming to diagnose pancreatic cancer prior to the start of neoadjuvant therapy, a low adequate sample rate will likely increase the need for repeated EUS-TA procedures. This will be similar for diagnostic yield of malignancy (DYM) and sensitivity for malignancy (SFM).

Why should quality be measured?
Patients, obviously, deserve the best possible care. Therefore, each EUS practitioner should know the quality of care he/she is providing. This cannot be done without measurements. If you do not measure you cannot properly inform your patient. If you do not measure you will never know whether improvement is necessary. If you do not measure you cannot benchmark your performance with your peers. If you do not measure you cannot improve. If you do not measure you miss out on opportunities for teaching advanced endoscopy trainees. If you do not measure you miss learning opportunities and the satisfaction of improving quality of care.

For these reasons endosonographers in the 21st century, both in high- and low(er) volume centers, should feel obliged to measure key performance indicators. KPI measurements may serve as the basis for regional quality improvement initiatives, including post-graduate teaching and learning opportunities. Measuring different KPI may also allow to pinpoint ‘vulnerabilities’ in complex multi-step procedures, such as EUS guided TA, and target quality improvement steps, as was illustrated in chapters 7 and 8.

What is keeping us from measuring KPI?
What is preventing endoscopists throughout the world to start measuring KPI to see how they are doing in comparison to the proposed performance targets and their peers? To answer this question, the ESGE quality improvement committee conducted surveys among its member societies in 2017 and 2019. The first barrier identified concerns motivation of endoscopists. Endoscopists fear additional administrative burden next to their already demanding daily practice. They also fear additional costs and prolonged endoscopy reporting times. Another source fueling resistance to implementation is lack of trust in organizations performing audits and the fear of being punished for underperformance [18].

Auditing quality of endoscopy in the Netherlands
With the implementation of the national colorectal cancer screening program, automated performance indicator measurements for colonoscopies on a national level were introduced in 2011[19]. The Dutch gastrointestinal endoscopy audit (DGEA), was initiated in 2016, and is based on the initial colorectal cancer screening framework. DGEA is currently recording performance measures from colonoscopies for all indications performed nationwide. DGEA uses an automated system integrating data from the national pathology database, and endoscopy reporting systems from individual hospitals. DGEA is started by the Dutch Society of Gastroenterology (NVMDL), governed by the Clinical Audit Board of the NVMDL, and executed by the Dutch Institute of Clinical Auditing (DICA). Ownership of data in DGEA remains with the contributing hospitals. DGEA aims to include a variety of endoscopic procedures in the near future, including ERCP and EUS [20].

The Dutch registration of complications in endoscopy (DRCE) was also initiated in 2016 and comprises all complications of endoscopic procedures performed in the Netherlands. The registration of endoscopic complications in DRCE is performed manually by individual endoscopists, following their institutions local evaluation of complications of endoscopic procedures.

From 2012 onwards, quality metrics of all ERCP procedures performed nationwide are prospectively recorded. Feedback on recorded data is provided to contributing endoscopists. This has led to both a nationwide reduction in the number of gastroenterologists performing ERCPs, as well as improvement in performance compared to initial measurements [21-23].

Future directions
Measuring KPI and improving quality of endoscopic procedures should be incorporated into clinical practice as well as into the training curriculum of future endoscopists. Regardless of the continued focus on volume and “production” in hospitals, endoscopists should feel obliged to start measuring and improving. Moreover, measuring KPI of endoscopy should be integrated in a structured curriculum for lifelong learning [24]. Policy makers and hospital management may endorse and facilitate such initiatives by stimulating multidisciplinary collaboration in and amongst hospitals, measuring costs (and cost-reductions likely achieved), and by supporting the construction of necessary information technology platforms.

Based on experience with performance indicators of EUS-TA described in this thesis and other publications on this topic, we suggest as a core data set to audit EUS performance: age and sex of the subjects, indication of the EUS procedure, and RAS and DYM in cases of EUS-TA of solid pancreatic lesions, and complications [6,8]. RAS and DYM are both performance indicators reflecting outcome of EUS-TA procedures, having a direct effect on patient burden. They both are available within a week following the procedure, and therefore suitable for “real-time” monitoring of performance. Moreover, they can be extracted automatically from the Dutch national pathology database (PALGA), preventing administrative burden, and can be presented as CUSUM-curves, allowing for instant comparison with benchmarks and performance of peers.

Prior to initializing such an audit standardization of pathology reports is required. We would suggest implementation of the classification suggested for evaluation of cytology of solid pancreatic lesions by the Papanicolaou society for all EUS-TA procedures of solid lesions, regardless of the type of needles (FNA or FNB) used [25]. Implementation and governance of an EUS audit should be organized and led by the Dutch Society of Gastroenterology and Hepatology (Nederlandse Vereniging van Maag-, Darm-, en Leverartsen: NVMDL) in collaboration with the Dutch Society of Pathologists (Nederlandse vereniging voor pathologie, NVVP).

With regards to the use of EUS for selecting patients with suspected bile duct stones for ERC+ES, we would suggest to add a performance indicator to the list of ERCP performance indicators. Since MRCP is also used for this purpose, we suggest to introduce the proportion of patients with proven bile duct lithiasis prior to ERCP as a performance indicator. Although this indicator reflects the quality of the diagnostic process leading to ERCP, instead of reflecting actual ERCP performance, creating a separate EUS performance indicator on this topic seems less practical.

In our opinion, the provision of feedback on performance to individual endosonographers should be organized regionally. By introducing regional quality improvement teams with regular meetings amongst a limited number of professionals familiar with one another, a safe environment is created in which feedback data and improvement strategies can be discussed. Presenting feedback data at these meetings in an anonymized fashion, will further add to this feeling of safety, and leaves the responsibility for the quality delivered with the contributors from individual centers. As described in the proceedings of QUEST (chapters 5 and 7), this will lead to improvement of quality overall and in each individual center, without the need for “top-down” interventions. This method allows practitioners the responsiblity to take their own action in case of underperformance, supervised and guided by their regional peers.

We would encourage all practitioners involved in EUS-TA procedures, not to wait for an audit to “come their way”, but to proactively start to measure and improve their individual performance instead. We would advise all endosonographers to review the last 20 EUS-TA procedures of both solid pancreatic lesions, and solid non-pancreatic lesions performed in their endoscopy unit. The proportion of non-diagnostic procedures should be <15% (equal to RAS > 85%) and the diagnostic yield of malignancy in case of suspected pancreatic malignancies should be >70%. If these performance targets are not met, we suggest to re-evaluate the protocols and techniques used, and discuss the subject with your peers and local pathologist, devise a strategy for quality improvement, and continue your measurements.

Endosonographers throughout the world are invited to share the results of their measurements with the Dutch Quality in Endosonography Team (QUEST) by e-mail (r.quispel@rdgg.nl). We intend to provide a sequel of the benchmark for quality performance measurements study performed by Savides et al in 2007 [26].