Healthcare systems approach to patient reported outcomes—benefits and challenges in thoracic surgery

What is the problem we are trying to solve with patient reported outcomes (PROs)?

It has been more than two decades since the Institute of Medicine launched an international effort focused on improving the quality of health care across the United States. In 2000, the Committee on Quality of Health Care in America published a very comprehensive report, To Err Is Human: Building a Safer Health System, that not only addressed patient safety but was also able to set an ambitious international agenda so to reduce errors and ultimately improve patient safety across the care continuum (1). This set in place a series of interventions that were designed to improve patient outcomes and included championing for healthcare system-wide electronic medical records, 80 hour work weeks for residents and quality improvement training for all levels of healthcare organizations. Additionally, in 2005, a very innovative piece of legislature was established entitled “The Patient Safety and Quality Improvement Act” which was designed to increase transparency and voluntary confidential reporting of adverse events and was ultimately intended to improve communication between care providers (2). Not long before that, the Medicare Prescription Drug, Improvement, and Modernization Act introduced the Acute Care Episode demonstration, which was intended to shift the focus of patient care to address the value-based healthcare that patients were looking to receive (3). These were all very important initiatives borne out of a very real intention to improve outcomes. Interestingly, ultimately the initiative resulted in many millions of dollars saved with no negative impact on patient safety (4,5).

In thoracic surgery, there is a long history of using clinical outcomes through a robust data set, referred to as the society of thoracic surgeons (STS) National Database, which is a clinically validated instrument that has tracked patient outcomes for both complex and more routine operations (6). The STS database tracks traditional outcomes including mortality and post-operative complications, but of recent has also developed and implemented composite scoring for pulmonary resections that takes into account the complexity of our patients and the operative procedures that we perform (7). This was an important step for the STS as it truly validated what we have known for years as Thoracic surgeons; that our patients are complicated and even the very best clinically validated outcomes data is not always reflective of what we actually do as a subspecialty surgical profession. To that end, of late, the voice of the patient is now being incorporated into the data and our reporting of results and data points collected (8-11). Referred to as PROs, these data points which truly reflect the patient experience and perceived outcome, are pivotal to us in our quest to deliver value-based healthcare. Only through thoughtful collection and analysis of this patient generated data can we understand what matters to our patients and how to improve their outcomes.

What is at risk for healthcare systems if we don’t get PROs data right?

As our patient expectations have increased, so too have the payers expectations increased in almost an exponential fashion. As a result, ways in which to evaluate the overall quality and safety of hospitals has grown exponentially and often without any input from surgeons or healthcare staff. In some ways, quality assessment has become an explosive, publicly facing industry and has included an increasing number of patient-focused healthcare rating systems. Relatively little, however, is actually known about what these rating systems collect and analyze data and as there are so many of them, it does indeed support that there appears to be, facetiously, more than 700 Top 100 hospitals in America. So to better comprehend the differences as well as the patient users perception of the healthcare systems rating programs, a comparison of four national rating systems was completed (12). The rating systems that were included were U.S. News and World Report’s best hospitals, leapfrog hospital survey, CMS’s hospital compare, and consumer reports and healthgrades (12). Interestingly, in this study there was no US hospital that was rated as a high performer by all four national rating systems. Furthermore, there was only 10% of over 800 hospitals that were rated as a high performer by one rating system and were then rated as a high performer by any of the others. Additionally, the study revealed that there was a lack of consensus among the rating systems in that each system used its own rating methods, and different outcomes measures of performance were used. Furthermore, the research group that conducted the comparative study found that described variation across hospital ratings added tremendous complexity to figuring out a healthcare systems real quality and actually made it more difficult for stakeholders, including payers, to recognize and reward hospitals for good quality of care. Most concerning though, was the finding that the public-facing quality comparison programs complicated healthcare systems decision making processes relative to their improvement efforts, and even confused patients and their families (12). Needless to say, there were absolutely no PROs included in any of these quality reporting programs that were compared.

The challenge for healthcare systems and their thoracic surgeons is to provide surgical care to patients with high acuity and overall complex disease processes in an evidence-based fashion. This care must be delivered all the whilst maintaining quality and avoiding perceived penalties within reporting systems that use minimally risk-adjusted data and are not truly indicative of the level of care that is provided by thoracic surgeons. Several entities have joined these public reporting efforts by taking advantage of available data that is not clinically validated, referred to as administrative data, and never collecting the voice of the patient as PROs are intended to do. Much controversy within our profession and publicly, has resulted from the release of several of these types of databases, including the ProPublica surgeon scorecard, and Consumers’ Checkbook. For instance, the patient advocacy journalism company ProPublica analyzed the complication rates of over 16,000 surgeons performing operations across 3,575 hospitals. The resultant report used the administrative Medicare data from 2009 to 2013 for eight common procedures that were believed by the data analysists to be elective and relatively low risk. Interestingly, only two measures of harm were used: in-hospital mortality and readmissions within 30 days. A risk-adjustment model was said to have been applied but overall the model seemed unclear and not widely or uniformly used in the analysis. Furthermore, the true voice of the patient is not even closely approximated through this administrative data and yet the patients are meant to use it. The data was intended to be reflect of how a particular surgeon would perform at an average-performing hospital on an average patient population rather than the more complex patient population often treated at large academic institutions (13,14). Fortunately, Thoracic procedures were not included in this scorecard, but it is likely only a matter of time.

Another surgeon performance reporting tool and website is Consumers’ Checkbook which analyzed Medicare data from 2009 to 2012 for more than a dozen commonly performed surgical procedures. Each surgeon received a star ranking on the basis of his or her performance relative to 90-day mortality, prolonged hospital stay, and 90-day readmissions. The number of stars from three to five reflected the surgeons calculated quality; much like we would rank a hotel or toaster we bought on Amazon. Not surprisingly, the public presentation of these scorecards triggered discussion and concern relative to surgeon performance data and how it is actually meant to be used by both surgeons and patients. While intentions may have been positive so to help patients make good decisions, great caution must be given so to ensure that any reported data are actually clinically validated, reported accurately, and have the buy-in of the physicians and most importantly, the patient.

Reputation is clearly important, but financially the impact of ranking and quality outcomes is very significant as well. The value-based purchasing (VBP) program, established by the ACA, was implemented as a pay-for-performance program for healthcare systems that care for Medicare patients. Under the VBP Program, Medicare has adjusted a portion of reimbursement to healthcare systems on the basis of how well they clinically perform relative to specific measures compared with other hospitals and how much they improve their performance compared with a baseline period. The total performance score (TPS) is derived from four areas including clinical processes of care, patient experience scores, clinical outcomes and overall care efficiency of service. The VBP Program was initially designed to facilitate and ultimately report better clinical outcomes for healthcare system patients so to improve their care during hospital stays (15). Sadly this program is very dependent on invalidated administrative data and the AHRQ patient safety indicators that we have shown to be poorly correlated to outcomes (16). The only redeeming factor for the VBP program is that patient experience has held a high profile in the measurable metrics. Whilst a move in the right direction, patient experience scores are unlikely to be as impactful as PROs that are truly the voice of the patient.

Lastly, we as surgeons are now being asked to provide care not only during the acute episode but also for up to 90 days afterwards (17). The Bundle Payments for Care Improvement (BPCI) program connects reimbursement for care received during a single inpatient care episode with financial incentives for improved overall clinical performance. The expectation is that this model of episodic care will lead to higher health care quality and more coordinated care at a lower cost. In 2013, CMS announced a shift in healthcare system reimbursement from the traditional fee for service (FFS) to the BPCI model, with payment shifts from FFS to the bundle initially predicted to be 30% by 2016, 50% by 2018, and the remaining FFS payments linked to institutions’ quality data, and to 90% by 2018 (18). The BPCI program has not quite met its mark as of 2019, but there is clearly directional change from CMS coming and we as surgeons will be held responsible for longer.

Healthcare systems must embrace both reputational and financial transparency and active participation in quality outcomes, but the methodology must be of our own design rather than of outsiders. Patients and physicians must be active participants, further emphasizing the importance of engaging patients in the data collection and even analysis (19,20). This will take active participation of thoracic surgeons at the local and national level, Active engagement will ensure that we are proponents for the importance of PROs so to direct our resources in developing advanced data collection tools and analysis. Ultimately PROs-based databases can help to replace and even refute the current quality reporting programs that in no way approach validating the value we render to patients.

Capturing the patient voice: the healthcare system responsibility

Supporting the capture and ultimate use of the patient’s voice through PROs has every opportunity to not only enhance the patient’s experience it will also improve physician engagement. The ability to capture and scale PROs across a health system has to be supported not only by a very integrated health information technology infrastructure but by leadership that are committed to the most meaningful quality data (15). This can be a very difficult task for some healthcare systems and really stretch their resources. However, it is clear that practice transformation through patient engagement requires integration of the patient voice in the EMR. Proactive collection of PROs must permeate the multiple layers of healthcare organizations. To ensure that the resources are well used, the PROs that a healthcare system captures, in particular for thoracic oncology patients, must be able to meet several ideal criteria. Firstly, the PROs that are collected must be relevant; there must be patient as well as provider input so to ensure that the PROs are truly meaningful. Secondly, the PROs must be reliably collected, safely stored and accessible by the provider and the healthcare system. This requires aligning the electronic medical record with purposeful informatics support. Thirdly, there must be decision support so to reliably respond to the PROs and relevant symptoms reported. Fourthly, physicians and support staff must be trained so to be able to interpret and respond to PROs. Lastly, there must be a commitment from the healthcare system so to consistently review and develop on going quality improvement strategies relative to PROs collection and usage and to support ongoing research in this realm.

PROs are the quality data element of the future and only through supporting the accurate collection and analysis of them, can a healthcare system truly be deemed as providing value-based care.

Acknowledgments

Funding: None.

Footnote

Provenance and Peer Review: This article was commissioned by the Guest Editor (Peter J. Kneuertz) for the series “Patient reported Outcomes in Thoracic Surgery: A new Frontier” published in Journal of Thoracic Disease. The article has undergone external peer review.

Conflicts of Interest: The series “Patient reported Outcomes in Thoracic Surgery: A new Frontier” was commissioned by the editorial office without any funding or sponsorship. The author has no other conflicts of interest to declare.

Ethical Statement: The author is accountable for all aspects of the work in ensuring that questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved.

Open Access Statement: This is an Open Access article distributed in accordance with the Creative Commons Attribution-NonCommercial-NoDerivs 4.0 International License (CC BY-NC-ND 4.0), which permits the non-commercial replication and distribution of the article with the strict proviso that no changes or edits are made and the original work is properly cited (including links to both the formal publication through the relevant DOI and the license). See: https://creativecommons.org/licenses/by-nc-nd/4.0/.

References

1. Institute of Medicine. To Err Is Human: Building a Safer Health System. Washington: National Academies Press, 2000.
2. Patient Safety and Quality Improvement Act of 2005. Agency for Healthcare Research and Quality, U.S. Department of Health and Human Services 2008. Accessed December 19, 2019. Available online: https://archive.ahrq.gov/news/newsroom/press-releases/2008/psoact.html
3. Centers for Medicare and Medicaid Services. Acute Care Episode Demonstration. Baltimore. Accessed December 19, 2019. Available online: https://innovation.cms.gov/initiatives/ACE
4. Herman B. 2 major lessons from CMS’ bundled payment ACE demonstration. Becker’s Hospital Review. 2012;3.
5. Vesely R. An ACE in the deck? Bundled-payment demo shows returns. Mod Healthc 2011;41:32-3. [PubMed]
6. Broderick SR, Grau-Sepulveda M, Kosinski AS, et al. The Society of Thoracic Surgeons Composite Score Rating for Pulmonary Resection for Lung Cancer. Ann Thorac Surg 2020;109:848-55. [Crossref] [PubMed]
7. Fernandez FG, Shahian DM, Kormos R, et al. The Society of Thoracic Surgeons National Database 2019 Annual Report. Ann Thorac Surg 2019;108:1625-32. [Crossref] [PubMed]
8. Khullar OV, Fernandez FG. Patient-Reported Outcomes in Thoracic Surgery. Thorac Surg Clin 2017;27:279-90. [Crossref] [PubMed]
9. Hopkins KG, Ferson PF, Shende MR, et al. Prospective study of quality of life after lung cancer resection. Ann Transl Med 2017;5:204. [Crossref] [PubMed]
10. Fagundes CP, Shi Q, Vaporciyan AA, et al. Symptom recovery after thoracic surgery: Measuring patient-reported outcomes with the MD Anderson Symptom Inventory. J Thorac Cardiovasc Surg 2015;150:613-9.e2. [Crossref] [PubMed]
11. Khullar OV, Rajaei MH, Force SD, et al. Pilot Study to Integrate Patient Reported Outcomes After Lung Cancer Operations Into The Society of Thoracic Surgeons Database. Ann Thorac Surg 2017;104:245-53. [Crossref] [PubMed]
12. Austin JM, Jha AK, Romano PS, et al. National hospital ratings systems share few common scores and may generate confusion instead of clarity. Health Aff (Millwood) 2015;34:423-30. [Crossref] [PubMed]
13. Friedberg MW, Pronovost PJ, Shahian DM, et al. A Methodological Critique of the ProPublica Surgeon Scorecard. Santa Monica, CA: RAND Corporation; 2015. Accessed December 19 2019. Available online: https://www.rand.org/pubs/perspectives/PE170.html
14. Ban KA, Cohen ME, Ko CY, et al. Evaluation of the ProPublica surgeon scorecard “adjusted complication rate” measure specifications. Ann Surg 2016;264:566-74. [Crossref] [PubMed]
15. Ryan AM, Krinsky S, Maurer KA, et al. Changes in hospital quality associated with hospital value-based purchasing. N Engl J Med 2017;376:2358-66. [Crossref] [PubMed]
16. Gray DM, Hefner JL, Nguyen MC, et al. The link between clinically validated patient safety indicators and clinical outcomes. Am J Med Qual 2017;32:583-90. [Crossref] [PubMed]
17. Bundled Payments for Care Improvement Initiative (BPCI): General Information. Centers for Medicare and Medicaid Services. Available online: http://innovation.cms.gov/initiatives/bundled-payments
18. Kivlahan C, Orlowski JM, Pearce J, et al. Taking risk: early results from teaching hospitals’ participation in the Center for Medicare and Medicaid Innovation Bundled Payments for Care Improvement initiative. Acad Med 2016;91:936-42. [Crossref] [PubMed]
19. Austin E, LeRouge C, Hartzler AL, et al. Capturing the patient voice: implementing patient-reported outcomes across the health system Qual Life Res 2020;29:347-55. [Crossref] [PubMed]
20. Stover AM, Tompkins Stricker C, Hammelef K, et al. Using Stakeholder Engagement to Overcome Barriers to Implementing Patient-reported Outcomes (PROs) in Cancer Care Delivery: Approaches From 3 Prospective Studies. Med Care 2019;57:S92-9. [Crossref] [PubMed]