The impact of improved surgical safety checklist participation on OR efficiencies : A pretest-post test analysis Does improved use of a surgical safety checklist influence OR efficiency ?

Objective: To describe changes in day of surgery (DOS) cancellations and procedural delays following introduction of a practice improvement intervention to improve team members’ participation in the surgical safety checklist (SSC). Methods: Pretest—posttest electronic audit of secondary data collected 12 months before and 12 months after implementation. A consecutive sample of patients who underwent elective surgeries were included. Elective surgeries over two periods (November 2014 to September 2015, and November 2015 to October 2016) were included in the audit and data was collected retrospectively. The practice improvement intervention coined ‘pass the baton’ was implemented over four weeks in October 2015. Results: Across audit periods 33 017 surgical procedures (16 262 pretest and 16 755 posttest) were performed. DOS cancellations between phases totalled 826 with an increase of 112 in the posttest phase with the largest posttest increase being in suite cancellation (increase of 97). Across phases, there were 1508 procedural delays (pretest n=737, posttest n =771), with the most frequent delay being due to staff availability (p=0.577). Pretest procedural delays averaged 38.7 minutes (SD 52.4) and posttest averaged 36.8 minutes (SD 43.2) (p=0.428). Conclusions: These results suggest no change in clinical efficiencies when the SSC is fully utilised. That is, increased participation in the checklist does not increase delays in surgery. When considering ways to improve clinical efficiency, hospital administrators need to consider skill mix, physical layout of the OR and additional staffing, factors not captured in routine clinical audit data collected.


Introduction
Perioperative services are typically comprised of three phases: preoperative, intra-operative, and post-operative.As a department, perioperative services is one of the most dynamic and complex in a hospital system and generates up to 60 per cent of the total gross revenue 1,2 .Nevertheless, US estimates suggest that they are also one of the costliest departments in any hospital, contributing to more than 40 per cent of its total running costs 1,3 , with costs as high as USD $40 per minute 1,2 (2018 AUD estimates $55 per minute).Therefore, efficient management of the service is necessary to minimise increased costs.Loss of information during the patient journey through the department may negatively affect patient flow and reduce clinical efficiency.
Peer-reviewed article 'Efficiency' is broadly defined as performance that leads to cost reduction without compromising quality.Thus, efficiency relates to both productivity and quality.In the operating room (OR) context, definitions of efficiency usually focus on time, whereas reductions in time related to a level of output translates into efficiency 4,5 .Efficiency in the OR depends on minimising wasted and unused time to meet projected surgical targets 1 .Numerous factors influence OR efficiencies e.g.surgical scheduling accuracy, on time starts, minimising case cancellations and case turnover times 4 .
Research suggests that improved service efficiency depends on the synchronisation of interprofessional communications in the OR department which has a resultant impact on patient flow 6,7 .The intent of the World Health Organization (WHO) surgical safety checklist (SSC) is to improve several 'must do' critical clinical tasks and hence improve the fluency of processes, team communications and operations throughout the patient's perioperative journey.Although not intended to directly improve OR efficiencies, the checklist acts as a memory aid for passing on key information or actions that may otherwise be overlooked or forgotten ensuring timely and consistent communications among surgical teams 8 .Thus, the SSC aids interdisciplinary team communications and coordination of clinical activities.The checklist divides the operation up into three phases -the period before anaesthetic induction (sign-in), the period after induction and before surgical incision (timeout), and the period during and immediately after wound closure but before transferring the patient out of the OR (sign-out) 8 .Despite the WHO SSC having been implemented in over 132 countries world-wide 9 , compliance remains a challenge [10][11][12] .We hypothesised that a theory-based practice improvement intervention aimed at changing clinician behaviour would increase checklist participation and item use and influence OR efficiencies relative to day of surgery (DOS) cancellations and procedural delays.We chose these efficiencies because communication processes may affect them, particularly during the sign-in and sign-out phases of the WHO SSC.To date, few studies have evaluated improvements in WHO SSC use relative to longitudinal changes in these OR efficiencies.

Method
We conducted a pretest-posttest audit of electronic secondary data to describe changes in the numbers of procedural delays and DOS cancellations following implementation of an intervention to improve participation in the WHO SSC.DOS cancellations and delays, regardless of the underlying cause(s), negatively impact on use and consequently on costs 13 .Retrospective audits of an electronic database of surgical information maintained by the hospital occurred over two 12-month periods.

Setting and sample
The study setting was a 750-bed tertiary hospital in Queensland specialising in all surgeries except transplantation.The department has 18 commissioned ORs and performs approximately 16 000 surgeries per year.A consecutive sample of patients undergoing elective surgeries during the periods November 2014 to September 2015 and November 2015 to October 2016, and drawn from the Operating Room Information Management System (ORMIS) database was included.Data for the month of October 2015 was excluded as at this time the process improvement strategy was being implemented across the OR department.Over a four-week period, key stakeholders implemented a process improvement strategy intended to increase staffs' participation in the safety checks of the WHO SSC.

Process improvement strategy
In October 2015, a process improvement intervention coined 'pass the baton' (PTB) was rolled out department-wide with the goal of improving team participation in the locally modified WHO SSC.PTB was nurse-led and developed with input from key stakeholders across nursing, surgery and anaesthetics.Process strategies to promote behaviour changes in WHO SSC participation were delivered over four weeks and included audit and feedback, opinion leaders and change champions, reminders and prompts and formal and informal education.A process evaluation of these strategies is presented elsewhere 14 .The phases in which it was most difficult to maximise staff participation were the sign-in and sign-out phases.Therefore, the PTB intervention specifically involved the allocation of nursing staff to lead the sign-in and sign-out using a deliberate call-and-response format.Implementing changes that address team-based delivery of care have demonstrated not only increases in OR efficiencies [15][16][17] but also improvements in patient safety 18,19 .

Data collection and coding
Electronic data from the ORMIS database of operative times inclusive of in-suite to out of OR times (i.e.in-suite, in anaesthetic, in OR, procedure start, procedure finish, out of OR), procedural delays (type and reason), surgical specialty, and month and year were extracted for cases of elective surgeries.The original ORMIS data files were given to the lead author as an encrypted Excel file.In the original database, DOS cancellations and delays had multiple codes for similar types and reasons.
We recoded DOS cancellations and procedural delays according to their primary origin, i.e. whether they were related to the organisation/ department or to the patient.In the analysis, we excluded DOS cancellations and procedural delays that were patient-related as these were usually out of the control of health care professionals and not influenced by process improvements associated with the use of the WHO SSC.For instance, in relation to DOS cancellations 'failure to attend surgery', 'patient cancelled booking' and 'unfit for surgery' were Reasons for delays relate to the availability of bed, equipment or documents; staffing; and previous case over-run.
Coded according to the primary reason/origin.
Categorical variable, numbers summed in each category.
In OR time 5 Time the patient enters the OR, often referred to as 'wheels in' to OR.
Time recorded in ORMIS.

Procedure start time 2
The earlier time of either the specific positioning of the patient for surgery or commencement of the skin preparation.
Time recorded in ORMIS.
In OR time ('wheels in') to procedure start time 4,5 Time the patient enters the OR from either the induction room or main reception area until the time the patient is either positioned or has been prepped and draped for surgery.This period includes anaesthetic induction process.

Measured in minutes.
Procedure finish time 5 Time when all the instruments and sponge counts are completed and verified as correct, all post-operative radiological studies to be done in the OR are completed, all dressings and drains are secured, and the surgeon(s) have completed all procedurerelated activities on the patient.
Time recorded in ORMIS.
Out of OR time 5 Time the patient leaves the OR, often referred to as 'wheels out' of OR.
Time recorded in ORMIS.
Procedure finish time to out of OR time ('wheels out') 4,5 Time from application of the final incision dressing, to when the patient leaves the OR for transfer to the PACU.
Measured in minutes.

Elective day of surgery cancellation 4
Unanticipated cancellation of elective surgery due to either patient or hospital-initiated factors.
Coded according to the primary reason/origin.

Analysis
We cleaned and analysed the data using the Statistical Package for Social Sciences (SPSS; V.24, IBM, NY, New York, USA), and checked a random sample of 20 per cent for accuracy.Descriptive statistics using absolute (n) and relative frequencies (per cent) or means and standard deviations (SD) were used appropriate to the level of data.
For categorical data, comparisons between phases relative to type and reason for DOS cancellation and procedural delay, and surgical specialty were analysed using the Chi squared (χ 2 ) statistic.Independent sample t-tests were used to compare overall time differences (in minutes) for each surgical specialty over pretest and posttest phases.We used 95 per cent confidence intervals (CI) and considered p-values of < 0.05 significant.

Ethics
Ethics approval was given by Griffith University (NRS/06/14/HREC) and the Gold Coast University (HREC/13/ QGC/154) Human Research Ethics committees.Following ethics approval for the main study, we sought permission to obtain  3 shows longitudinally, the pretest and posttest means (in minutes) for all specialties combined relative to time from procedure finish to out of OR.In the pre-implementation phase there were drops in February, June and September.

Discussion
Few studies have used longitudinal efficiency indicators to measure the impact of theory-based process improvement strategies on DOS cancellations and procedural delays across an entire OR department.
The benefit of the checklist on patient outcomes, safety related practices and clinical processes are well researched [20][21][22][23] .There were no significant differences in clinical efficiencies despite observed improvements in checklist items coverage and participation post-implementation of PTB (acknowledging that the SCC was not fully utilised) 24 .Clearly, improvements in using the checklist do not translate into increased efficiencies.Still, our results suggest that increased participation in the WHO SSC does not negatively impact on OR efficiency.That is, active team participation does not increase the time taken to complete clinical activities.Many staff were concerned that implementation of PTB needed extra time and would reduce their ability to complete elective case lists on time 25 .Previous research suggests that improvements in interdisciplinary communication reduces procedural delays 7,26,27 .Nonetheless, some of these studies used self-reported survey data or had short follow-up periods 26,27 .Month (Error bars +1-2 Std error) Mean time from in OR to procedure start (minutes) Figure 2: Time from in OR to procedure start (in minutes) pre-and post-implementation periods over month Therefore their findings need to be considered relative to these limitations.
Our results indicate increases across most specialties for total DOS cancellations (Table 2).The increase in ' in-suite' cancellations during the posttest period suggest that clinical/case-related discrepancies may not have been identified until after the patient was received into the department.The main reason for DOS cancellation related to bed or equipment availability.We suggest there are a couple of contributing factors.Firstly, for obstetric procedures, the availability of a 'dedicated' emergency obstetric theatre during weekdays (8.00 am to 5.00 pm) is not always guaranteed at the study hospital.Priority is always given to emergency Caesarean sections (categories 2-4), resulting in the cancellation and rescheduling of DOS elective (booked) C-sections.Second, maxillary facial/ENT/plastics and orthopaedic cases involving implantable prosthetic components (e.g. total hip/knee replacement surgeries) relies on having the appropriate range and sizes of prosthetics available.The check-in phase of the WHO SCC has an item covering equipment and instrument availability.It may be that increased communication at this time identified a problem with availability and averted a situation when patients were anaesthetised without having the equipment on hand.Plausibly this may demonstrate that team members are communicating the necessary pre-checks and lessening the risk of unnecessary or prolonged anaesthesia time thereby increasing patient safety.
The duration of procedural delays actually decreased despite an increase in the number of surgical procedures performed during the posttest period.The results of other research in this area also suggests modest to moderate improvements in procedural delays following teamwork initiatives 17,25,26 .For instance, Wolf et al. 26 and Nundy et al. 27 reported reductions of 13 per cent to 31 per cent in procedural delays following the implementation of briefings and debriefings.Clearly, improvements in communication, teamwork and planning are the Figure 3: Time from procedure finish to out of OR (in minutes) in pre-and post-implementation periods over month drivers behind how checklist briefings reduce procedural delays 27 .Paradoxically in our study, four out of ten specialties showed increases in time delays (Table 3).Generally, procedures in these specialties had shorter operative times, were less technically complex and involved younger patient cohorts.
Our results suggest that staff availability was the most common cause of procedural delays across both periods (Figure 1).This result is somewhat concerning.This type of delay is potentially disruptive to workflow and impinges on the quality and work environment of surgery.
Staffing issues are often associated with safety because improved efficiency and capacity mean that more operations are performed during the daytime when back up personnel are readily available.Fewer surgeries are performed at night when skeleton teams who may be unfamiliar with each other are more likely to work together 26,27 .Changes to staffing over time are inevitable in any health care setting.Over the twoyear audit period there were changes in staffing with seasonal influxes or attrition of staff occurring throughout the year.Further, increases in the number and complexity of surgical cases in the posttest period meant that staff workloads necessarily increased leading to additional staff being hired.Many of these new staff needed training and upskilling in unfamiliar surgical specialties and so were often on a steep learning curve.
Saving time (as a measure of efficiency) in the OR does not necessarily lead to increased efficiency 28 .PTB was implemented as a driver to enable change in practice and process when executing the checklist 14,24

Limitations
We acknowledge some limitations, so there are caveats in the interpretation of these results.Firstly, the use of a single hospital site may limit the extent to which results can be generalised.Secondly, ORMIS data may be subject to errors in coding, leading to misclassification.Where there were discrepancies, the lead author followed up with coding staff to clarify.Also, the accuracy of the times entered depends on the ability of staff to enter these times in the ORMIS system as they occur.Clearly there will be occasions where clinical activities take priority, potentially reducing the accuracy of these data.Thirdly, these analyses are based on selected factors identified at the departmental level, thus patient-related factors were not included and may have contributed to OR efficiencies.Nonetheless, these factors were largely outside the control of the department or organisation, hence their exclusion.Fourthly, departmental factors (e.g.staff turnover and training requirements, increased workload and the addition of new procedures) could not be accounted for.Such factors may also influence performance but could not be captured in the audit data.Finally, while PTB was implemented department-wide, not all teams consistently participated.
Prior to analysis, it was impossible to delineate particular cases (and exclude them) where there was patchy or limited use of PTB.Despite these limitations, these longitudinal analyses showed trends relative to the types of delays that occurred (i.e.bed, equipment or documentation availability; staff availability, case over-run) and seasonal variations in wheels-in and wheels-out times across surgical specialties.Thus, these results may help to identify areas of process efficiency and areas for improvement.

Implications for perioperative nursing
Our study shows no change in health services performance when the surgical safety checklist is fully utilised.The primary intent of the checklist is to improve team performance visà-vis communication among surgical teams rather than clinical efficiencies.Contrary to long-held beliefs, performing the checks as a team-based activity does not decrease clinical efficiencies.Clearly contextual factors have a bearing on performance.Therefore, hospital administrators need to also consider the interplay of environmental and operational factors not currently measured as part of clinical efficiencies.
Month (Error bars +1-2 Std error)Mean time from procedure finish to out of OR (minutes)

Table 1 :
OR efficacy indicators, their definitions and measures (where applicable)

Table 2 :
DOS cancellations pre-and post-implementation Note: ^ covers facio/maxillary, ear, nose and throat, dentistry and plastic surgery.Figure 1: Types of delays relative to bed/equipment/documentation, staffing and prior case over-runs in preand post-implementation periods over month

Table 3 :
Pretest-posttest results for times from in OR to procedure start and procedure finish to out of OR Notes: Time difference is displayed in h:mm:ss.Some degrees of freedom (df) have decimals because Levene's test was violated so 'equal variances not assumed' data used.^ covers facio/maxillary, ear, nose and throat, dentistry and plastic surgery.