Use of the consolidated framework for implementation research to guide dissemination and implementation of new technologies in surgery

Introduction

Improving surgical outcomes is important for patients, providers, and healthcare systems. To facilitate high quality outcomes, the utilization of evidence based interventions, practices, procedures, and policies (EBP) that are successfully implemented into various clinical environments is imperative. Yet, barriers can occur at various points in the implementation process, leading to less effective interventions and gaps in care. Multiple factors known to prevent successful adoption and implementation of an EBP include: resistance of providers; lack of belief or knowledge of the EBP; lack of skills to implement the EBP, and lack of organizational management support and resources (1). Therefore, understanding specific theories and processes to promote the uptake of EBPs into surgical care is imperative. Dissemination and implementation (D&I) science address these needs.

Implementation research consists of “scientific investigations that support movement of evidence-based, effective health care approaches from the clinical knowledge base into routine use” (2). Dissemination research, as defined by the National Institutes of Health, is the “targeted distribution of information and intervention materials to a specific public health or clinical practice audience” (2). Together, D&I is used to guide the process from initial design to implementation and dissemination of EBP, utilizing specific theories, models, frameworks, and processes to promote the uptake of EBP into surgical care.

Conceptual frameworks

Conceptual frameworks are utilized in D&I to increase the “generalizability and interpretability of research findings” (3). They are used to better understand and explain how and why the D&I of evidence based innovations succeed or fail, may guide the assessment of the process, and identify factors that might influence the implementation and effectiveness (3-5). Consolidated framework for implementation research (CFIR) is a pragmatic meta-theoretical framework that encompasses a repository of standardized D&I constructs (3,5). Damschroder et al. integrated the previously published implementation science theories to identify constructs that either conceptually or empirically influenced implementation and compiled a single, common language, consolidated framework with defined domains and constructs (3,5). CFIR has five major domains and 39 individual constructs within the domains.

Intervention characteristics address the characteristics of the intervention/innovation being implemented and their potential influence on implementation. The eight constructs within the domain address specific factors such as: source of the innovation; evidence strength and quality; relative advantage; adaptability of the innovation; trialability; complexity; design quality and packaging; and finally its costs (3,4).

Inner and outer settings include the social, political, and economic contexts though which the implementation process occurs. These domains are dependent on the context of the implementation process. For example, an EBP may be implemented within a hospital unit (inner setting) but implementation efforts may be affected by factors outside the hospital (outer setting). The outer setting has four constructs: patient needs and resources; cosmopolitanism—networked with other organizations; peer pressure—mimetic/competitive; external policy; and incentives. The inner setting has five broad constructs, and two of the broad constructs have sub-constructs. The constructs include: structural characteristics; networks and communications; culture; implementation climate (includes six sub-constructs); and readiness for implementation (includes three sub-constructs) (3,4).

Characteristics of individuals contain five constructs: knowledge and beliefs about the intervention; self-efficacy to execute the innovation; individual stage of change; individual identification with organization; and other personal attributes. Finally, the process domain has four broad constructs and one has sub-constructs: planning; executing; reflecting and evaluating; and engaging (includes five sub-constructs) (3,4).

CFIR was designed to be used across all phases of implementation to systematically identify factors that may influence D&I efforts (3,4). CFIR can be used to help design a roadmap or implementation protocols, by guiding data collection and analyses. In the pre-implementation phase, CFIR can be used to assess contextual capacity and needs, by providing defined constructs that identify potential barriers and facilitators to implementing an EBP in specific environments (5). During the implementation process CFIR can be used to monitor various processes and progress, leading into post-implementation where it can be used to determine which constructs influenced outcomes and effectiveness (5).

An example using CFIR to assess D&I of the SUrgical Risk Preoperative Assessment System (SURPAS)

We utilized CFIR to facilitate the translational process from design to implementation, broad utilization, and sustainability of the SURPAS tool. SURPAS creates data-driven individualized risk assessments of common adverse postoperative outcomes that may enhance the informed consent process, improve shared decision making, and lead to improved surgical outcomes.

Risk assessment in surgery is essential to guide patient-centered treatment decisions but is variable in practice (6,7). Presently, risk assessment of perioperative complications is based on accepted or previously reported values and influenced by subjective provider assessment of individual patient comorbidities (8,9). Formal risk assessment tools exist, such as the Veterans Health Administration’s Veterans Affairs Surgical Quality Improvement Program (VASQIP) risk calculator and the American College of Surgeons’ (ACS) National Surgical Quality Improvement Program (NSQIP) risk calculator. However, these, and other surgical risk assessment tools, are time intensive to use, may not provide clinically useful information, and are not regularly integrated into clinical workflow (8,9).

SURPAS aims to address these issues. It was developed from the ACS NSQIP database and the detailed design and statistical methodologies of SURPAS have been previously described (8,10,11). The SURPAS tool is user-friendly and is based on a more parsimonious set of eight easily accessible preoperative risk variables. It calculates accurate risk predictions of 11 meaningful adverse surgical outcomes across nine adult surgical specialties and is integrated into the electronic health record (EHR) (Figure 1) (8-11). SURPAS is utilized at the preoperative patient visit and is designed to improve clinical surgical outcomes, patient communication, and health literacy, while better informing the provider of patient-specific needs (8-11). A pictorial and numeric display of the results is printed out and provided to the patient (Figure 2).

Figure 1 SURPAS screen for data input and risk display for the surgical team. This figure shows the EHR screen: on the left providers enter patients’ surgical procedure, age, functional health status, ASA class, planned hospital status following procedure, surgical specialty, and if it was an emergency procedure. On the right side of the screen are presented the results of the individual risk for the 11 clinically-meaningful adverse surgical outcomes. Additionally, the results are compared to the national risks in the ACS NSQIP database, shown by the blue line. SURPAS, SUrgical Risk Preoperative Assessment System; EHR, electronic health record; ACS, American College of Surgeons; NSQIP, National Surgical Quality Improvement Program.

Figure 2 SURPAS’ individualized pictograph of the output for postoperative risk. The SURPAS individualized pictograph provides patients with their results for 30-day postoperative adverse outcomes: mortality; overall morbidity; infectious; urinary tract infection; transfusion; cardiac; renal; pulmonary; venous thromboembolic; and neurological complications. The document contains two pictographs for each outcome: (I) the first (on the left) represents their individualized risk for the postoperative outcome; and (II) the second (on the right) represents the ACS NSQIP national average for the outcome. SURPAS, SUrgical Risk Preoperative Assessment System; ACS, American College of Surgeons; NSQIP, National Surgical Quality Improvement Program.

In preparation for the broad D&I of SURPAS to all University of Colorado (UC) Hospital surgical clinics and to prepare for further spread of SURPAS to all six UCHealth System hospitals, we conducted a formative evaluation, “designed to identify potential and actual influences on the progress and effectiveness of implementation efforts. Formative Evaluation enables researchers to explicitly study the complexity of implementation projects and suggests ways to answer questions about context, adaptations, and response to change.” (12). Findings from the formative evaluation are guiding design of a multi-component strategy for future scale-up. In this formative evaluation, we used CFIR to guide the assessment of the process and identify factors that might influence SURPAS implementation and thus, its effectiveness (3-5).

The objective of this paper is to review the need for D&I, introduce one D&I framework, and provide an example of how it is used to facilitate the implementation and sustainability of an intervention into multiple clinical environments. In this article, we describe how we used CFIR to identify D&I factors that emerged from the formative evaluation. We also describe the identified CFIR domains and constructs to design and guide a pragmatic D&I protocol for further implementation and scaling-up of SURPAS.

Methods

The formative evaluation included two phases: contextual baseline assessment and trial implementation. The contextual baseline assessment was performed at the UC Hospital and utilized a qualitative methodology. Stakeholders were engaged through focus groups with surgical patients and providers, and individual interviews with senior administrators, which provided suggestions about improving SURPAS’ use and utility as a tool and provided guidance to facilitate the implementation such as: seeking early wins; identification of local champions; and performing iterative improvements. Detailed findings of the Contextual Baseline Assessment are published (13).

We then utilized these findings to guide the trial implementation to study the feasibility of the uptake of SURPAS and the contextual baseline assessment of five UC Hospital surgical clinics. These included thoracic, endocrine and general, orthopedic, vascular surgery, and urology. Specifically, we obtained opinions and personal experiences of using SURPAS, identified adaptations to SURPAS for improvement, assessed the context of the clinical environment for SURPAS, and were provided suggestions to improve the implementation process.

Data collection

Contextual baseline assessment consisted of focus groups and individual interviews with key stakeholders: surgical patients; surgical providers; and hospital administrators. These participants were shown a presentation of the SURPAS tool. Opinions, suggestions for adaptations to improve the SURPAS tool, and suggestions to facilitate the eventual implementation of SURPAS were elicited.

Patient recruitment included surgical patients who underwent an operation within the previous 12 months of the study start date at UC Hospital, were enrolled in NSQIP data collection, and lived within 30 miles of UC Hospital. Recruitment letters describing the study were mailed to 200 patients randomly selected from the above cohort, inviting them to contact us. Those who contacted us sharing interest in participation were scheduled for participation in a focus group. All patients who attended one of the original focus groups were invited to attend a follow-up focus group approximately one year later. All patient participants received $75 for their participation in each focus group.

A convenience sample of surgical provider and clinical researcher participants was recruited from the University of Colorado School of Medicine Department of Surgery’s Surgical Outcomes and Applied Research program monthly meeting. A follow-up focus group was held approximately one year later. Individual interviews were performed with administrative officials. Individual postcard informed consent was provided at the time of all focus groups and interviews.

During the Trial Implementation phase, surgical providers from the involved clinics, and their respective clinic administrators were recruited to participate in the study. Providers were asked to use the SURPAS tool with consented patients, complete a quantitative survey on each patient, and agree to be subsequently interviewed. The clinic administrators were individually interviewed on their opinions about SURPAS and its impact on work flow and patient clinical care.

Surgical patients of the participating providers, who attended a pre-operative clinic appointment, were invited to participate in the study. Recruited patients were consented and provided a copy of the signed informed consent. Participation included utilization of the SURPAS tool in their risk discussion during the preoperative clinical visit, completion of a quantitative survey after the visit, and possible observation during the clinic visit and interview afterward to obtain their opinions and experience with SURPAS.

The qualitative inquiry elicited opinions about SURPAS, suggestions for improvement, opinions for optimizing the implementation process based on experiences of implementation within the contextual climate, culture, and its proclivity for change. The 30–45 minute interviews and focus groups were conducted by Master- and PhD-educated members of the team trained in qualitative research, were audiotaped and transcribed verbatim. This study was approved by the Colorado Multiple Institutional Review Board.

Data analysis

CFIR coding

A matrix analysis qualitative approach was used to code, analyze, and organize the qualitative data (14,15). We used all of the 39 CFIR constructs and their definitions (4) as our a priori codebook to code qualitative segments that influenced effectiveness of the SURPAS tool, its processes and process adaptations, and barriers and facilitators to the D&I of SURPAS. Important domains and constructs were identified based on the frequency they were mentioned, the degree of importance articulated by the participants or researchers, or both. This iterative process involved three qualified analysts (AC Lambert-Kerzner, DM Aasen, DM Overbey). Each coded the transcribed text segments from interviews of individuals to specific CFIR constructs. We then compared coding and used a consensus process when disagreements occurred. Upon thematic saturation (not eliciting any new themes), the team concluded the analysis and identified the final domains, placing them into the matrix for comparative analysis.

The matrix allowed simultaneous comparison of a large volume of data and contrasting findings between the different groups: patients, providers, and administrators (14,15). These findings identified factors related to SURPAS characteristics, assessed the readiness of the clinical environments to implement SURPAS, assessed the culture of the environment, and identified barriers and facilitators to the implementation processes. All analyses and findings were integrated and documented with an audit trail (14-17). Illustrative quotes were selected by consensus of all members of the analytic team for each CFIR construct.

Results

The contextual baseline assessment consisted of interviews with three focus groups of 18 patients (Table 1); two focus groups with 13 surgical providers, one researcher, two medical students, two clinic administrators; and five individual interviews with administrative officials (Table 2). During the trial implementation, nine surgical providers and five clinic administrators agreed to participate after reviewing post-card consents. This resulted in recruitment of 199 patients enrolled through the participating surgical clinics.

Table 1 Demographics of patient participants
Full table

Table 2 Demographics of providers
Full table

We interviewed the seven surgeons and two nurse practitioners who implemented SURPAS, five clinic administrators (Table 2), as well as a convenience sample of 27 of the 199 patients (Table 1) who used SURPAS during their pre-surgical risk assessment.

Of the 31 constructs within the CFIR, factors affecting D&I of SURPAS were spread across three of five CFIR domains. Four CFIR constructs strongly influenced the effectiveness of the implementation of SURPAS based on the degree of importance articulated by the participants or researchers, or both. These domains and constructs were affirmed by data analyzed from the trial implementation. Five additional constructs, identified as being strongly influential across the remaining two CFIR domains, emerged through the course of the trial implementation. Specific quotes for each CFIR domain and construct from each phase of the study are presented in Table S1. The following sections describe each construct.

Table S1 Specific quotes for each CFIR domain and construct from each phase of the study
Full table

Outer setting domain

Patients’ needs and resources

This construct emerged as a central focus of the formative evaluation. Providers and administrators identified SURPAS as a mechanism to provide individualized, patient-centered care to patients. SURPAS was thought to engage and empower patients with personalized information to participate in a conversation that supports a shared decision regarding their treatment plan.

“Any type of personalized care is gonna be better than generalization for people. Having that kind of information on paper is extremely helpful because you could imagine you get bombarded with a lot of information leading up to the surgery.” (Patient #3).

Optimization of pre-surgical care was identified by the administrators and providers. Surgeons may change their preoperative workup, medical optimization, or potentially select different surgical or therapeutic treatment options. Recommendation for patients to change some of their lifestyle habits, in light of their increased surgical risks, could be made.

Patients perceived that SURPAS provided desired information, increased engagement with providers, and enhanced patients’ surgical experience. When specifically asked about the SURPAS tool, all but one patient was impressed with SURPAS.

“Pretty informative”; “Simple to understand”; “It did a great job of delineating likely risk to unlikely risk”; “More realistic”; and “Better understanding”. (Multiple patients).

Intervention characteristic domain

Evidence strength and quality

The empiric evidence that SURPAS was grounded in the ACS NSQIP database and statistical methodologies supported participants’ strong view of its ability to provide accurate data to providers and patients. The participants were confident the data would improve the risk assessment process and could ultimately improve outcomes of morbidity and mortality. Providers found SURPAS to be helpful in accurately presenting the surgical risk to patient’s families, especially in the more complex cases. Most providers indicated it enabled a more in-depth discussion of risk than the current standard of care and ensured that patients were educated about their operation.

“I think, [SURPAS] is informing our providers and our faculty based on science and the robust database that we have, then one could say that, “This is evidence-based, and we have all the data back here to back this, and we should listen to it,” because if we can predict with X percent accuracy, why wouldn’t you use this? Then, why wouldn’t it improve outcomes?” (Administrator #7).

Adaptability

During the contextual baseline assessment phase, the adaptability construct emerged as a facilitator of tool usage by the local providers. Suggestions to improve the SURPAS tool included providing definitions for the American Society of Anesthesiology physical status classification (ASA class), emergent/elective operation status, and functional health status of the patient on the input screen, provide drop-down menus for the input of the SURPAS predictor variables, current procedural terminology (CPT) codes, and names for operations frequently performed by the provider. In addition, due to the variability in each clinical environment, multiple suggestions were provided to guide the implementation process in these various contexts. Setting up protocols for implementing SURPAS in addition to protocols or pathways for bundles of complications were suggested. Patients were shown multiple variations of data display and the highest rated was chosen.

In the Trial Implementation, Adaptability emerged as both a facilitator and a barrier. Specifically, the constraints of the ACS NSQIP database-limited operation code choices, which frustrated providers at times. The flexibility in utilization of SURPAS allowed each clinic to implement the tool to fit their local needs, such as the option of having someone other than the surgeon input the data and then provide the results to the surgeon or another provider, such as an advanced practice provider (APP), to engage in the risk discussion.

Complexity

In the trial implementation, complexity was identified as an important aspect to the implementation of SURPAS. Specifically, its accessibility in the EHR system, ease of data input, direct EHR documentation, and the printable patient handout facilitated implementation in individual clinics. Overall, providers found it did not disrupt their clinic workflow.

“That’s one of the best things I like about it. I’ve been part of lots of studies and it’s real easy. It doesn’t mess up my clinic flow, it’s quick and easy and nice to work with, which is great.” (Provider #7).

Design quality and packaging

The design quality and packaging construct emerged as administrators, providers, and patients believed SURPAS provided a functional, quality, and efficient tool. All nine providers in the trial implementation found SURPAS to be easy to use. All nine found that SURPAS provided a concise mechanism to estimate and document the risk of surgery and that it was useful for sharing the anticipated surgical risk with patients.

Patients shared that they appreciated the data being presented both numerically and with a pictograph (Figure 2). Patients shared that their perception of the tool and their opinions of utilizing SURPAS in their pre-surgical appointment “.lowers my anxiety about the procedure”; is “reassuring”; “made me feel more comfortable”; and “clarified a lot of things” (Multiple patients).

Inner setting domain

Implementation climate

Throughout this study, the organizational commitment to implement SURPAS was identified as a critical component of the potential success of SURPAS in the UC Hospital setting. High-level administrators and surgical care providers described the contextual components that existed and provided suggestions to leverage these for the successful implementation and dissemination of SURPAS. This included the importance of integrating SURPAS into the EHR, broad communication to ensure clinic staff were aware that this system is supported by the administration, and collaboration between departments.

Tension did emerge between low-risk patients’ preference for SURPAS and providers’ concern about additional clinic time required for risk assessment in obviously low risk patients. Therefore, a few providers shared that they would most likely continue using SURPAS, but only with their high-risk patients, after the Trial Implementation ended.

Compatibility, a sub-construct, addresses the importance of integrating SURPAS into existing clinic workflows. Providers agreed that SURPAS improved clinic workflows in a couple of ways: SURPAS enabled a more in-depth discussion of risk than the current standard of care; and provided an opportunity to ensure patients were more educated about their operative risks. Providers said patients were very receptive to SURPAS, which helped patients see the surgical risks visually, improving their understanding. Providers of low-risk surgeries shared that many patients said it was beneficial to them and reassured them that the decision to undergo surgery was the correct one.

Providers identified barriers to the implementation of SURPAS. One shared that medicine is traditionally conservative and lags in information technology (IT) implementation. Unavailability of computers or the inability to print the patient document in the clinic may cause providers to be resistant to adoption of SURPAS. Providers shared that it is customary to load work on doctors who, thus, may be reticent about adding new tasks. One surgeon was concerned that providers could be held legally accountable if the risks were low but higher than national averages and nothing was done to mitigate the “elevated risk”.

Characteristics of individual domain

Knowledge and beliefs about the intervention

Administrators and providers believed SURPAS had the potential to support their clinic to attain various goals such as improving patient care and satisfaction, and reducing length-of-stay and re-admissions. They were interested in utilizing SURPAS to optimize patients’ outcomes by preventing infections, decreasing pain, improving functional abilities, and increasing awareness of social determinants of health. Administrators believed that a multi-faceted partnership with case management, social work, and other clinical disciplines, would improve pre-operative care to prepare for surgery. Additionally, individual providers’ motivation to improve patient selection, patient care, and patient engagement was a reason they supported using SURPAS. Nonetheless, one provider believed his complication rates were better than those reported by SURPAS and opted not to use it in his clinic.

“I found it helpful, primarily, with more of the more complicated patients presenting for more complicated surgery. It did provide a good reference as to their potential risks with the more complicated procedures or surgery with the less than healthy patients.” (Provider #9).

Process domain

Planning

Clinical providers and administrators consistently identified planning factors that they believed would be vital to the successful implementation of SURPAS. Senior administrators suggested planning ahead with identified teams responsible for the implementation of the tool and technical support. The next steps included “marketing” the value to the end user to “generate buzz” and the use of rapid feedback to improve the tool and its implementation. Administrators also suggested planning for successful implementation by tying SURPAS to strategic goals of the healthcare system, including innovation, safety, access, growth, and patient-centeredness.

Clinic administrators identified specific activities such as clinic staff being properly educated about SURPAS; defining specifically who in the clinic is responsible for SURPAS’ accessibility on the computers, availability of printers for the SURPAS patient handouts, and continued technical support; who will access the information and input the data into SURPAS; and who will have the risk assessment conversations with the patients. These participants identified that planning needed to be in alignment with the senior management group in both bundled care and care management continuums.

Engaging

The engaging construct identified factors that administrators and providers believed would encourage adoption of SURPAS. Attracting and involving key individuals in the implementation and use of SURPAS needs multifaceted strategies including marketing, education, role modeling, and training. Additional engagement strategies included: testimonials of patients and providers regarding their perceived value of SURPAS; data to support the accuracy of SURPAS; using multi-level communications, emails, one-on-one demonstrations, presentations at clinic meetings and Grand Rounds; and iterative trainings for the various levels of providers, such as surgical residents in training, as they rotate through the different surgical services.

Additional findings

Although penetration (the integration of a practice within a clinical setting), and sustainability (the maintaining or institutionalizing of an intervention within a clinical environment’s culture through policies and practices) are not CFIR constructs, they are key implementation outcomes (18). Surgical providers shared how they will use SURPAS outside of the study setting, indicating that ease of use was the key to facilitate its implementation. All participating groups thought that SURPAS supported risk assessment and shared decision making and facilitated the documentation of operational risks.

One unique environment that implemented SURPAS was the pre-procedure services clinic, where APPs conducted in-depth pre-operative patient evaluation. The APPs used SURPAS for their evaluation and provided the results back to the surgeon. However, since the patient had already seen their surgeon, and potentially discussed operative risk, the APPs in the pre-procedure services clinic voiced that they were not comfortable reviewing risk because of potential discrepancies with the surgeon’s assessment and discussion, and because they were not versed in further expansion of specific patient risks associated with each type of operation. This process prevented any discrepancies between the APP’s and the surgeon’s interpretation of SURPAS results and their effect on the decision to proceed with surgery.

Publicizing feedback about the progress and quality of the tool and its implementation emerged as the most frequent suggestion for the successful implementation and sustainability of SURPAS. Suggestions included: keep SURPAS updated as new data from the ACS NSQIP becomes available; provide recommendations to mitigate the risk; and provide feedback to the participating surgeons comparing them to their peers. Process suggestions included: touch base with clinic staff on a regular basis; ask “Do we need to tweak it? What’s working well? What’s not working well?” Public display of clinic goals and providing data on implementation progress was viewed as very important by providers.

Discussion

In this article, we describe the importance of using D&I science in surgery and the successful utilization of CFIR to identify and categorize barriers and facilitators during the design and implementation of a novel surgical clinical decision support tool, SURPAS. The identified constructs that strongly affected the broad implementation of SURPAS at specific UC Hospital surgical clinics are likely to affect scaling-up further spreading SURPAS to all six UC Health System hospitals.

We designed this study to guide the D&I of SURPAS with the objective of utilizing CFIR’s domains and constructs to organize the data collection and evaluation for the two phases in the formative evaluation: the contextual baseline assessment and the trial implementation (4,5,13). Using CFIR to systematically assess the qualitative data, we were able to integrate the data and identify strategies to enhance the D&I process. Strategies, defined as “methods or techniques used to enhance the adoption, implementation, and sustainability of a clinical program or practice” (19,20) are aimed at supporting the integration of SURPAS into various clinical environments across the UC Health System. We used intervention mapping (20-23) to map the CFIR constructs to: (I) identify barriers to and facilitators for scaling up; (II) describe how these manifest in context; (III) identify specific strategies to leverage the facilitators or mitigate the barriers; and (IV) identify the desired outcomes. Specific examples are shown in Table 3.

Table 3 Intervention mapping
Full table

Data from the contextual baseline assessment allowed us to improve the functionality of the SURPAS tool and enhanced our understanding of potential barriers and facilitators in the trial implementation. The findings from the trial implementation identified specific barriers and facilitators to implementing SURPAS across the six hospitals in the system, allowing the creation of specific strategies to address each one.

For example, although most participants were supportive of SURPAS, tension emerged between low-risk patients’ preference to use SURPAS and providers’ concern about additional clinic time required for formal risk discussion. Therefore, if this barrier is also identified at other hospitals, a strategy to address this issue is to use patient and family feedback and share the results with the providers and administrators through a rapid feedback loop (23). Additionally, providers highlighted barriers such as the medical profession being slow to adopt new technologies, which may indicate resistance to any new tool such as SURPAS. Strategies to address these issues involve incentivizing using SURPAS and de-incentivizing use of the traditional method of risk assessment with policies and procedures created through cooperation with administration.

A few issues arose due to the constraints of SURPAS’ background database having limited CPT choices. Therefore, strategies will need to be employed to work with surgical providers and technical experts to continually update SURPAS’ inclusion of CPT codes and sharing codes other providers used in place of missing codes. Dissemination of the latest information can be provided through educational meetings with stakeholders, individual meetings with clinical champions and implementation facilitators.

Most providers believed in the accuracy of the individual risk assessments provided by SURPAS; yet, one provider believed the complication rates provided by SURPAS were higher than his actual rates, and therefore did not use SURPAS. This barrier is categorized in the knowledge and beliefs construct. Audit and feedback strategies can address this barrier by collecting clinical performance data and providing it to clinicians and administrators to confirm or dispute such beliefs. The UC Health System presently collects these data, which could be reported back to each provider (23).

These strategies ensure interactive problem solving and support through individuals who are familiar with both the local clinical environment and SURPAS. Local IT professionals will be supported by an implementation expert and a technology team, which are available throughout the implementation process.

The literature supports the need to identify specific D&I strategies and to apply systematic methods for selecting and tailoring these strategies to specific contexts and innovations (19,20,24,25). The literature supports the variability in using CFIR coding yet it also supports the recommendation of Damschroder et al. to report the rationale for selection of each construct (3,13). Kirk et al. identified gaps in the use of CFIR in all of the phases of an individual implementation (5). This study contributes to the D&I literature through the successful utilization of CFIR in the translational process from the design of SURPAS to the proposal of the broad implementation. Additionally, this study provides data supporting the use of SURPAS, thus adding our findings to those from other studies, building the evidence for what, how, and when implementation factors and strategies improve implementation processes (26).

Strengths of this study include the initial integration of a broad range of opinions to improve SURPAS and identify barriers and facilitators of the implementation process. Our inclusion of all possible CFIR constructs will promote the integration of our findings with findings from other studies using CFIR. Potential limitations may include social desirability bias—i.e., participants responding in a certain way to please the interviewer—and that the qualitative data were elicited only from people directly involved in this study. Other surgeons, administrators, and patients who would be affected by a broad implementation were not included.

Conclusions

Using CFIR to guide and code the formative evaluation of introduction of new technology within surgery, such as SURPAS, has provided essential information by identifying constructs and processes to inform design of future larger-scale D&I of this technology (19,27). This prospective process using CFIR has allowed us to critically improve SURPAS and design the implementation process, improving the likelihood of a successful D&I of SURPAS in the surgical clinics of UC Hospital and further to the UC Health System.

Acknowledgements

Funding: This study was supported by a grant from the Agency for Healthcare Research and Quality (AHRQ) 1R21HS024124-01 grant.

Footnote

Conflicts of Interest: The authors have no conflicts of interest to declare.

Ethical Statement: The Colorado Multiple Institutional Review Board approved this study (#15-1044).

References

1. Bauer MS, Damschroder L, Hagedorn H, et al. An introduction to implementation science for the non-specialist. BMC Psychol 2015;3:32. [Crossref] [PubMed]
2. Albright K, Gechter K, Kempe A. Importance of Mixed Methods in Pragmatic Trials and Dissemination and Implementation Research Acad Pediatr 2013;13:400-7. [PubMed]
3. Damschroder LJ, Aron DC, Keith RE, et al. Fostering implementation of health services research findings into practice: a consolidated framework for advancing implementation science. Implement Sci 2009;4:50. [Crossref] [PubMed]
4. Keith RE, Crosson JC, O’Malley AS, et al. Using the Consolidated Framework for Implementation Research (CFIR) to produce actionable findings: a rapid-cycle evaluation approach to improving implementation. Implement Sci 2017;12:15. [Crossref] [PubMed]
5. Kirk MA, Kelley C, Yankey N, et al. A systematic review of the use of the Consolidated Framework for Implementation Research. Implement Sci 2016;11:72. [Crossref] [PubMed]
6. Goldfarb M, Perry Z, Hodin A, et al. Medical and surgical risks in thyroid surgery: lessons from the NSQIP. Ann Surg Oncol 2011;18:3551-8. [Crossref] [PubMed]
7. Gupta PK, Miller WJ, Sainath J, et al. Determinants of resource utilization and outcomes in laparoscopic Roux-en-Y gastric bypass: a multicenter analysis of 14,251 patients. Surg Endosc 2011;25:2613-25. [Crossref] [PubMed]
8. Meguid RA, Bronsert MR, Juarez-Colunga E, et al. SURPAS III: Accurate Preoperative Prediction of 8 Adverse Outcomes Using 8 Predictor Variables. Ann Surg 2016;264:23-31. [Crossref] [PubMed]
9. Hammermeister KE, Henderson WG, Bronsert MR, et al. Bringing Quantitative Risk Assessment Closer to the Patient and Surgeon. A Novel Approach to Improve Outcomes. Ann Surg 2016;263:1039-41. [Crossref] [PubMed]
10. Meguid RA, Bronsert MR, Juarez-Colunga E, et al. Surgical Risk Preoperative Assessment System (SURPAS): I. Parsimonious, Clinically Meaningful Groups of Postoperative Complications by Factor Analysis. Ann Surg 2016;263:1042-8. [Crossref] [PubMed]
11. Meguid RA, Bronsert MR, Juarez-Colunga E, et al. SURPAS II. Parsimonious Risk Models for Postoperative Adverse Outcomes Addressing Need for Laboratory Variables and Surgeon Specialty-specific Models. Ann Surg 2016;264:10-22. [Crossref] [PubMed]
12. Stetler CB, Legro MW, Wallace CM, et al. The Role of Formative Evaluation in Implementation Research and the QUERI Experience. J Gen Intern Med 2006;21 Suppl 2:S1-8. [Crossref] [PubMed]
13. Lambert-Kerzner A, Ford KL, Hammermeister KE, et al. Assessment of attitudes towards future implementation of the "Surgical Risk Preoperative Assessment System" (SURPAS) tool: a pilot survey among patients, surgeons, and hospital administrators. Patient Saf Surg 2018;12:12. [Crossref] [PubMed]
14. Miles MB, Huberman AM Saldaña J. Qualitative Data Analysis. A Methods Sourcebook. 3rd ed. Thousand Oaks, Ca: Sage Publications Ltd., 2014.
15. Averill JB. Matrix analysis as a complementary analytic strategy in qualitative inquiry. Qual Health Res 2002;12:855-66. [Crossref] [PubMed]
16. Patton M. Qualitative Research & Evaluation Methods. 3rd ed. Thousand Oaks, Ca: Sage Publications Ltd., 2002.
17. Tong A, Sainsbury P, Craig J. Consolidated criteria for reporting qualitative research (COREQ): a 32-item checklist for interviews and focus groups. Int J Qual Health Care 2007;19:349-57. [Crossref] [PubMed]
18. Proctor E, Silmere H, Raghavan R, et al. Outcomes for Implementation Research: Conceptual Distinctions, Measurement Challenges, and Research Agenda. Adm Policy Ment Health 2011;38:65-76. [Crossref] [PubMed]
19. Powell BJ, Beidas RS, Lewis CC, et al. Methods to Improve the Selection and Tailoring of Implementation Strategies. J Behav Health Serv Res 2017;44:177-94. [Crossref] [PubMed]
20. Powell BJ, Waltz TJ, Chinman MJ, et al. A refined compilation of implementation strategies: results from the Expert Recommendations for Implementing Change (ERIC) project. Implement Sci 2015;10:21. [Crossref] [PubMed]
21. Brownson RC, Colditz GA, Proctor EK. Dissemination and Implementation Research in Health Translating Science to Practice. Oxford University Press, 2012.
22. Bartholomew LK, Parcel GS, Kok G. Intervention mapping: A process for designing theory- and evidence-based health education programs. Health Educ Behav 1998;25:545-63. [Crossref] [PubMed]
23. Fernández ME, Gonzales A, Tortolero-Luna G, et al. Using Intervention Mapping to Develop a Breast and Cervical Cancer Screening Program for Hispanic Farmworkers: Cultivando La Salud. Health Promot Pract 2005;6:394-404. [Crossref] [PubMed]
24. Forman J, Harrod M, Robinson C, et al. First Things First: Foundational Requirements for a Medical Home in an Academic Medical Center. J Gen Intern Med 2014;29:S640-8. [Crossref] [PubMed]
25. Green CA, McCarty D, Mertens J, et al. The chief of the services is very enthusiastic about it”: A qualitative study of the adoption of buprenorphine for opioid addiction treatment. J Subst Abuse Treat 2014. [Crossref]
26. Leeman J, Birken SA, Powell BJ, et al. Beyond “implementation strategies”: classifying the full range of strategies used in implementation science and practice. Implement Sci 2017;12:125. [Crossref] [PubMed]
27. Birken SA, Bunger AC, Powell BJ, et al. Organizational theory for dissemination and implementation research. Implement Sci 2017;12:62. [Crossref] [PubMed]