Teaching to the test: developing an assessment tool for novice echocardiographers

Focused critical care echocardiography (FCCE) is a powerful tool that allows for the rapid assessment of several important physiologic states quickly, inexpensively, and (seemingly) without risk to the patient. FCCE, performed by non-cardiologists, has gained popularity over the past several decades, and today many residencies and fellowship programs are incorporating some echocardiography into their curriculum. In addition, there are a large number of established physicians who are learning to incorporate FCCE into their practice. Finding ways, therefore, to ensure a minimum standard of operator competence is a crucial step to safeguard quality and patient safety. To date, however, no such minimum standard exists. Recent work by Gaudet and colleagues (1) has provided an important addition in this effort with their development of a score measuring efficiency and accuracy of image acquisition. We believe that such a tool should be incorporated into a broader assessment program—one whose goal is ensuring minimum clinical standards by emphasizing the real world application of FCCE in three important ways. It should weigh most heavily the images that are critical to diagnosis and management, emphasize assessment of images that novices typically struggle with and are most prone to misinterpret, and include evaluation of interpretation and application of FCCE.

Numerous trials have demonstrated that novice echocardiographers can attain a degree of competency in short order. Residents (2), critical care fellows (3), emergency physicians (4), and hospitalists (5) have all been shown to achieve facility with various aspects of FCCE with as little as 12 to 22 hours of training. Perhaps most strikingly, one study showed that medical students, after receiving only 18 hours of training in ultrasound, were able to diagnose cardiovascular pathology more accurately than attending cardiologists who used physical exam alone (6).

Such impressive results have fueled the rapid growth of the use of ultrasound in day-to-day clinical practice. However as FCCE has grown so too have questions about how it should be taught, assessed, and regulated (1,7). Over a decade ago the American Medical Association (AMA) published a resolution that ultrasound imaging is “within the scope of practice of appropriately trained physician” (8). Of course, the exact meaning of “appropriately trained”, now as then, remains in question. Ultrasound poses little if any direct harm to the patient. Its real hazard lies in a clinician’s misinterpretation and misapplication of images obtained making clear minimum standards all the more critical.

The recent publication by Guadet and colleagues takes an important step towards this goal (1). Previously, most assessment tools measured image quality or diagnostic accuracy alone. The addition of an efficiency score is a novel metric that reminds us that in the critical care setting, speed has value. Their tool scores images obtained in terms of completeness, quality, and time taken for acquisition. It looks at features such as the presence of key structures as well as appropriate depth, gain, and centering. Using their tool they compared the efficiency of trainees to expert critical care echocardiographers all of whom were American Society of Echocardiography (ASE) level III certified.

In considering the effectiveness of an assessment tool we must make clear its goals. Assessing clinicians in FCCE requires establishing a minimum standard for competency, and not a comprehensive valuation. That is we wish to define standards that ensure an operator can accurately and reliably use FCCE in day-to-day practice in a way that aids diagnosis without risk of harm to the patient. Comprehensive standards for board certification in echocardiography have been developed by the ASE and are in use currently. An analogy would be the use of chest imaging. While many clinicians use chest X-rays and computed tomography every day in their work few have the scope, depth, or skill of a radiologist. This does not (and should not) preclude clinicians from interpreting the images themselves especially at times when a formal read may be delayed by hours. Nevertheless, while an efficiency metric is a valuable contribution, it is not, by itself, a sufficient assessment of competency (a point Gaudet and colleagues themselves make).

In the fast-paced environment of the intensive care unit (ICU), efficiency is clearly an important metric. Undifferentiated shock is a medical emergency where every moment counts. Still, the goal of FCCE is to obtain images that can directly aid in diagnosis and treatment of critically ill patients. Increasing the efficiency of image acquisition is unimportant if useful images are not obtained, and an extra 5 or 10 minutes getting excellent images is likely time well spent. It is important to remember that a rapid limited echocardiogram should not preclude a formal echocardiogram performed by a technologist and interpreted by a certified physician at a later time. While we agree that efficiency is an important new addition, perhaps a better metric might assess diagnostic accuracy within a reasonable timeframe (for instance 10 or 15 minutes) rather than one that places equal weight on speed.

Additionally, most assessment tools place all images on equal footing. In practice however, not all images are of equal value. Certain features and anatomy are key to the diagnosis of commonly seen issues in the ICU. The recognition of a pericardial effusion with right ventricular (RV) diastolic collapse, for instance, is far more important diagnostically than mild or even moderate reduction in left ventricular (LV) systolic function. We should develop an assessment of competency that focuses heavily on echocardiography’s real world applications. Such a tool should emphasize the windows, views, and maneuvers which are most frequently utilized in clinical practice. For example, effective imaging and interpretation of RV size and function would be weighted more heavily than the left atrium. Severe aortic regurgitation (often a sign of an acute event such as aortic dissection or infective endocarditis) would take precedence over aortic stenosis because its clinical implications and consequences are more immediate. The differential weighting of images addresses key pathologies which are critical for treating acutely ill patients.

Furthermore, there are certain images which novice echocardiographers commonly image incorrectly, potentially leading to misdiagnosis and harm. For instance, when the probe is rotated off axis in the Apical Four Chamber (A4C), the right ventricle may look inappropriately small and cor pulmonale may be missed. Likewise when probe is placed too medial (i.e., not at the LV apex) in the A4C, the RV may appear artificially enlarged leading to an erroneous interpretation of cor pulmonale. Either can lead to misdiagnosis and inappropriate treatment, but were we to use an assessment tool that weighs all images equally this kind of critical mistake would only result in a nominally lower score while its results to the patient may be devastating.

We believe FCCE will continue to gain popularity and acceptance by a wide range of practitioners, and the assessment of provider skill will play a key role in maintaining safety and ensuring quality. Our current metrics fall short. Gaudet and colleagues have added an important element with the introduction of efficiency, and perhaps their efficiency measurement can be incorporated into a more comprehensive assessment for training practitioners—one that weighs more heavily the images necessary for FCCE’s real world uses, guards against its most common pitfalls, and emphasizes its interpretation and clinical application.

Acknowledgements

None.

Footnote

Provenance: This is an invited Commentary commissioned by the Section Editor Zhongheng Zhang (Department of Critical Care Medicine, Jinhua Municipal Central Hospital, Jinhua Hospital of Zhejiang University, Jinhua, China).

Conflicts of Interest: The authors have no conflicts of interest to declare.

Comment on: Gaudet J, Waechter J, McLaughlin K, et al. Focused Critical Care Echocardiography: Development and Evaluation of an Image Acquisition Assessment Tool. Crit Care Med 2016;44:e329-35.

References

1. Gaudet J, Waechter J, McLaughlin K, et al. Focused Critical Care Echocardiography: Development and Evaluation of an Image Acquisition Assessment Tool. Crit Care Med 2016;44:e329-35. [Crossref] [PubMed]
2. Vignon P, Mücke F, Bellec F, et al. Basic critical care echocardiography: validation of a curriculum dedicated to noncardiologist residents. Crit Care Med 2011;39:636-42. [Crossref] [PubMed]
3. Beraud AS, Rizk NW, Pearl RG, et al. Focused transthoracic echocardiography during critical care medicine training: curriculum implementation and evaluation of proficiency*. Crit Care Med 2013;41:e179-81. [Crossref] [PubMed]
4. Moore CL, Rose GA, Tayal VS, et al. Determination of left ventricular function by emergency physician echocardiography of hypotensive patients. Acad Emerg Med 2002;9:186-93. [Crossref] [PubMed]
5. Lucas BP, Candotti C, Margeta B, et al. Diagnostic accuracy of hospitalist-performed hand-carried ultrasound echocardiography after a brief training program. J Hosp Med 2009;4:340-9. [Crossref] [PubMed]
6. Kobal SL, Trento L, Baharami S, et al. Comparison of effectiveness of hand-carried ultrasound to bedside cardiovascular physical examination. Am J Cardiol 2005;96:1002-6. [Crossref] [PubMed]
7. Oren-Grinberg A, Talmor D, Brown SM. Focused critical care echocardiography. Crit Care Med 2013;41:2618-26. [Crossref] [PubMed]
8. H-230.960 Privileging for Ultrasound Imaging. American Medical Association 1999. Available online: https://www.ama-assn.org/ssl3/ecomm/PolicyFinderForm.pl?site=www.ama-assn.org&uri=/resources/html/PolicyFinder/policyfiles/HnE/H-230.960.HTM