Development of a precision multimodal surgical navigation system for lung robotic segmentectomy

Introduction

The use of sub lobar anatomical resection is likely to increase, as confirmed by ongoing clinical trials for the management of very early stage lung carcinoma (1). Anatomical variation has rarely been described unless for specific lobes, for example the middle and lower lobe (2). However, from a technical point of view segmentectomy is a difficult procedure to perform and is reserved for highly experienced teams especially when a minimally invasive approach is proposed (3,4).

We have identified four main technical difficulties in performing successful minimally invasive segmentectomy.

• Constant anatomical variation of the bronchovascular elements and irregular/unpredictable spatial conformation of a given pulmonary segment;
• Unclear delimitation of the intersegmental plan;
• Difficult assessment of a safe resection margin;
• Lymphadenectomy of the intrapulmonary nodes.

Our aim was to describe three-dimensional (3D) imaging throughout the surgical procedure from preoperative planning to intraoperative assistance and complementary invasive investigations such as radial endobronchial ultrasound (R-EBUS) and pleural dye marking. This work emphasizes the development of the concept of image-assisted surgery: a step by step evolution (Figure 1).

Figure 1 Flow chart of the implementation of our robotic technique of segmentectomy reinforced by our multimodal imaging system (5-7).

Experience

Our experience was based on 114 robotic segmentectomies performed between January 2012 and October 2017. All indications for the proposed segmentectomy were in accordance with preoperative multidisciplinary meeting decisions (<2 cm proven or suspected lung cancer or pulmonary metastasis).

The technique of our robotic segmentectomy (a simplified 3-arm approach) has been described previously (5). The clinical value of the 3D model was validated in 2014 in a pilot study (6). Progressively, we have reached the conclusion that the availability of such an anatomic model demonstrated a perfect operative anatomic accuracy with regard to the bronchial and vessels distribution, and improved the safety and reliability of procedures. Our point of view regarding the benefits of using 3D reconstruction in surgical practice is shared by other teams (8). We have used this complete protocol in routine practice for more than one year (Table 1),(Figure 2).

Table 1 Procedures performed using our multimodal system
Full table

Figure 2 Minimal invasive segmentectomies from 2008 to 2017 in our institution epithor registry. VATS, video-assisted thoracic surgery; RATS, robotic-assisted thoracic surgery.

Description of image acquisition and workflow

The rendering of the 3D models is outsourced to a private company, Visible Patient^TM (Strasbourg, France) (6). This requires no extra work for the referent surgeon. This point is relevant because the 3D model used for surgery may be complex to build and time consuming. To protect the identity of the patient, data are anonymized. A CT scan is performed by the radiology department. Images (DICOM format) stored in the hospital picture archiving and communication system (PACS) are uploaded to the Therapixel^TM server. Reconstructions are rendered by specialized radiological technicians (Visible Patient^TM employees) who contour all anatomical structures (arterial, venous and bronchial up to the fifth level of division). Lobar and sublobar segmentation are performed automatically by the software. The completed 3D models are approved for clinical use under the Council of Europe directive CEO459, FDA and in Canada (no 94537). The model itself is portable (6), and its different properties can be explored with dedicated software. The application used for accessing the models—“Anywhere imaging” provides easy access across multiple hardware platforms (mainstream computers, tablets, smartphones) and permits external image projection. This avoids the use of high performance workstations (Figure 3).

Figure 3 The workflow to get 3D reconstruction from outpatients’ clinic to the surgical field. 3D, three-dimensional.

The main functions of the completed 3D models that can be explored are:

• Individual exploration of the distinct anatomical bronchovascular elements;
• Virtual resections (up to level of the independent segment);
• Volume calculation of each segment—allowing assessment of postoperative volume to analyze the impact on the predictive postoperative respiratory function;
• Simulation of safety margin—important in assessing the feasibility of the proposed resection.

For advanced surgical navigation in the operating room we also use a customized hardware platform—Fluid^TM provided by Therapixel^TM (Valbone, France), (this is rated as a class II medical device) that uses hand gestures for touchless control, allowing the surgeon to remain sterile while exploring the imagery. We also use the same system for video-assisted thoracic surgery (VATS) procedures (Figures 4–7).

Figure 4 Operative planning using the 3D model provided by VP® using the software Anywhere by Therapixel®. Evolution with segmentation at the end of 2016 (right corner). 3D, three-dimensional.

Figure 5 S1 segmentectomy with pleural dye marking with methylene blue without 3D reconstruction. 3D, three-dimensional.

Figure 6 Blue pleural dye marking for lesion close to the intersegmental plan S1/S2.

Figure 7 Visualization of the 3D model and exploration on the fluid console (9). 3D, three-dimensional. Available online: http://www.asvide.com/article/view/24363

Continuing the integration of the 3D model to a higher level is done by displaying it in the direct field of view of the operating surgeon at the level of the robotic console, through a simple HDMI connection from a portable device (6). The surgeon maintains the option of exploring all the functions of the model. The control of the different visible layers is done manually, by increasing the transparency of the desired anatomical layers. This provides a virtual reality experience for the surgeon. A true fully automated superimposition of the model on the intraoperative view or “augmented reality” will soon be available. However, it is not yet possible because of the physical state of the deflated lung during selective ventilation.

Complementary procedures: pleural dye marking

Several cases (n=15) were done using pleural dye marking by R-EBUS and virtual bronchoscopy especially when the lesion was close to the intersegmental plan in order to localize small peripheral lung nodules. We added this latest procedure to our 3D reconstruction technique because previously we had missed one lesion in our series due to the proximity of the intersegmental plan. The efficacy of this procedure has been assessed in a clinical study (7).

We reported in this study that 25 nodules, including 6 ground glass opacities, were resected in 22 patients by video-assisted thoracoscopic wedge resection (n=11) or robotic-assisted thoracoscopic surgery (10 segmentectomies and 1 wedge resection). The median greatest diameter of nodules was 8 mm. No conversion to open thoracotomy was needed. The endoscopic procedure added an average 10 minutes to surgical resection. The dye was visible on the pleural surface in all cases for segmentectomy. Histological diagnosis and free margin resection were obtained in all cases. Median skin to skin operating time was 90 minutes for robotic segmentectomy. The same operative precision was considered impossible by the surgeon without dye marking in the 10 cases of segmentectomy.

The endoscopic procedure is performed without fluoroscopy, under general anesthesia in the operating room immediately before minimally invasive surgery. General endotracheal anesthesia is induced with a single-lumen endotracheal tube in the operating room. The endoscopy is performed using a BF-MP60F endoscope (Olympus^TM, Tokyo, Japan) with a 4-mm outer diameter and 2-mm working channel. The ultrasound probe used is a UM-S20-17S radial probe (Olympus^TM, Tokyo, Japan) with a 1.4-mm diameter and whose distal end is introduced into the 1.9-mm diameter guide sheath.

Before each procedure, the nodule location is mapped using virtual bronchoscopy software in order to identify the smallest bronchus leading to the lesion (Lungpoint Planning Bronchus^TM, USA). When the fiberscope reaches the most distal sub-segmental bronchus along the route previously defined in virtual bronchoscopy, the guide sheath with the R-EBUS probe is inserted into the working channel. If the lesion is (or not) visualized by ultrasound, the probe is pushed far away into the bronchus in order to be close to the pleural space without fluoroscopy guidance. The probe is removed and the guide sheath left in place within the working channel. Then, 1 mL of methylene blue is injected in the guide sheath with 20 mL of air (in order to push the methylene blue dye inside the guide sheath). The bronchoscope is then removed, and the patient is intubated with a double-lumen tube and appropriately positioned (lateral decubitus position) for minimally invasive resection of the lung lesion [VATS or Robotic-assisted thoracic surgery (RATS)]. On inspection of the lung surface, the blue methylene dye is visualized in order to perform a wedge resection or a segmentectomy with endoscopic stapler (Figures 5,8).

Figure 8 Pleural dye marking before right S1 segmentectomy for better margin resection (10). Available online: http://www.asvide.com/article/view/24364

Examples of procedures performed using our multimodal protocol

The first segmentectomy presented here was performed for the resection of a nodular lesion of the left segment 6, diagnosed on a surveillance CT scan after a renal carcinoma. The lesion was 9 mm in diameter, and was in the lower part of the segment. With the aid of the 3D images, we quickly identified an anatomical variation with two distinct arteries for the S6. There was no anatomical variation concerning the S6 vein. We used a standardized approach for dissection considering three target areas, in a clockwise direction (pulmonary ligament, posterior pulmonary hilum and fissure. The intervention was uneventful. Biopsy confirmed renal metastasis, all the lymph nodes were negative (Figure 9).

Figure 9 Left S6 with capture of the robotic console and 3D model (11). 3D, three-dimensional. Available online: http://www.asvide.com/article/view/24365

The second example presented was a more complex double segmentectomy right S2 and S6 for two ground glass opacities correlated with a positive pet scan. Here, the model allowed precise identification of the A2 artery and B2 bronchus. The vein for the S2 was stapled with the parenchyma. Dissection and stapling for the S6 was performed with complete dissection of all the broncho-vascular elements. Biopsy confirmed two distinct adenocarcinoma lesions without positive nodes (Figure 10).

Figure 10 Double segmentectomies right S2 + S6 with capture of the robotic console and 3D model (12). 3D, three-dimensional. Available online: http://www.asvide.com/article/view/24366

The third example was a highly selective left S2 segmentectomy for a 15-mm nodule. The lesion was difficult to localize on the CT scan, appearing to be located in the posterior part of the lingula. The etiology was unconfirmed, and was accompanied by a positive PET scan. Without the precise localization of the lesion on the preoperative planning this resection would not have been possible. The intervention was uneventful (Figures 11,12).

Figure 11 Left S2 segmentectomy.

Figure 12 Left S2 segmentectomy using the per-operative 3D reconstruction (13). 3D, three-dimensional. Available online: http://www.asvide.com/article/view/24367

The fourth example is a planning S6 segmentectomy with a rare anatomical variation with 2 arteries (A6 a,b) and 2 bronchus (B6 a,b) (Figure 13).

Figure 13 Right S6 segmentectomy with rare anatomical variation: 2 arteries and 2 segmental bronchus.

Comments

Lung segmentectomy is gaining increasing interest as both a diagnostic and curative procedure for various small and central lung lesions requiring surgical excision. It is considered oncologically safe and presents the advantage of having a very limited impact on pulmonary function. The technical aspects of the procedure require a thorough understanding of the complex and highly variable pulmonary anatomy which could be improve by 3D model (7).

This multimodal imaging system protocol integrates the preoperative assessment of anatomical conditions and puts them right where they are needed in the operative room. This level of integration is only possible during robotic surgery. This surgical cockpit allows the surgeon to be visually immersed in local conditions. It is a great way to alleviate informational overload. All the mental projections concerning intricate anatomy fit in naturally without supplemental effort from the operator, clearly impacting the decision-making process during critical dissection and no return decisions like stapling of the different broncho-vascular elements. We believe that a complete surgical approach using a multimodal imaging system and a robotic platform could prevent intraoperative accidents and postoperative complications, and could improve the general performance of the surgical act.

Such an approach has been described by others using Oisrix 3D reconstruction and the robotic platform (7,14,15). However, the quality of the Oisrix 3D model was less accurate than the 3D model provided by Visible Patient^TM and it was time consuming for the radiologist and the surgeon. The complex workflow involved in 3D reconstruction is the main drawback to its routine use in surgery. We have successfully simplified the workflow by associating Therapixel^TM (Valbone, France) and Visible Patient^TM (Strasbourg, France).

Future developments

This integrated system using a 3D model with pre-operative marking if needed could also progress with fluorescence. The next step will be the integration of the NIR-ICG Fluorescence, Fire Fly System (Intuitive Inc., Sunnyvale, USA) in 2018 (16,17). This integration will facilitate the localization of the intersegmental plan. After selective control of the segmental artery on the 3D model, the use of fluorescence will give us the perfect plan for the segmentectomy. This multimodal imaging system has been recently described to evaluate biliary anatomical variations and to secure the cholecystectomy. These visualization techniques could be a complement to reduce the likelihood of biliary injuries and in our study pulmonary injury (18). This system has been described as gall bladder navigation. By analogy we could describe our system as precision multimodal surgical navigation of the lung.

Another concept using a new fluorescent marker, folate receptor targeted near-infrared contrast agent (OTL38) can be used during robotic procedures and be added to the multi-modal system. This is known as intraoperative molecular imaging (IMI) (19). This new concept could replace pleural dye marking to assess the intersegmental plan for lesions close to two segments.

Our system is therefore adaptable and allows the addition of different technologies. This is the first step of image-assisted surgery. The near future will probably incorporate these technologies in one: robot and 3D reconstruction done in real time with augmented reality (20,21). The field of interaction between the operator and virtual reality is rapidly evolving. The next step in the image overlay process is the integration of the 3D reconstructed structure directly in the stereoscopic view of the operating field offered by the robot without any input from the operator. A true augmented reality experience will soon be real and has already been successfully applied in other surgical fields (adrenalectomy, prostatectomy, hepatic surgery, neurosurgery, gynecology) (22).

Using currently available technology and with the support of the Visible Patient^TM team, we managed to implement the use of computer-assisted surgery in our daily practice. The highly optimized model has a small size of only 15 to 25 MB, yet the representation of the anatomical accuracy of the bronchovascular elements, delimitation of each segment and lesion location are optimal.

Limitations

Ours is a composite system supported by three different companies: Therapixel^TM for anonymization, DICOM transfer and screen support to view the 3D reconstruction, and Visible Patient^TM which builds the 3D model and provides the software for use on a tablet or a phone. In addition, there is the robotic platform (Intuitive Inc^TM) which integrates the reconstruction in the operative field.

Associated with this relative complexity is the cost, which is a major obstacle to the development of this technology. The average cost for 3D segmentation is around 600 euros. It is likely that routine use of a 3D model will become the gold standard for minimally invasive pulmonary segmentectomy; hence, we need to work on lowering the cost and improving funding for this new surgical environment.

In our experience, the use of a multimodal imaging system combined with robotic surgery could improve the general performance of segmentectomy. However, after this descriptive study we have to prove the efficacy of this concept, which will be the next step in our work.

Acknowledgements

The authors are grateful to Nikki Sabourin-Gibbs, Rouen University Hospital, for her help in editing the manuscript. We also thank Visible Patient and Therapixel Compagnies for their great availability.

Footnote

Conflicts of Interest: Jean Marc Baste declares lecturing and proctoring for Intuitive Surgical. The other authors have no conflicts of interest to declare.

References

1. Landreneau RJ, D'Amico TA, Schuchert MJ, et al. Segmentectomy and lung cancer: why, when, how, and how good? Semin Thorac Cardiovasc Surg 2017;29:119-28. [Crossref] [PubMed]
2. Nagashima T, Shimizu K, Ohtaki Y, et al. Analysis of variation in bronchovascular pattern of the right middle and lower lobes of the lung using three-dimensional CT angiography and bronchography. Gen Thorac Cardiovasc Surg 2017;65:343-9. [Crossref] [PubMed]
3. Gossot D, Lutz J, Grigoroiu M, et al. Thoracoscopic anatomic segmentectomies for lung cancer: technical aspects. J Vis Surg 2016;2:171. [Crossref] [PubMed]
4. Pardolesi A, Park B, Petrella F, et al. Robotic anatomic segmentectomy of the lung: technical aspects and initial results. Ann Thorac Surg 2012;94:929-34. [Crossref] [PubMed]
5. Rinieri P, Peillon C, Salaün M, et al. Perioperativeoutcomes of video- and robot-assisted segmentectomies. Asian Cardiovasc Thorac Ann 2016;24:145-51. [Crossref] [PubMed]
6. Le Moal J, Peillon C, Dacher JN, et al. Three-dimensional computed tomography reconstruction for operative planning in robotic segmentectomy: a pilot study. J Thorac Dis 2018;10:196-201. [Crossref] [PubMed]
7. Lachkar S, Baste JM, Guisier F, et al. Pleural dye marking using radial endobronchial ultrasound and virtual bronchoscopy before sublobar pulmonary resection for small peripheral nodules. European Respiratory Journal 2017;50:PA3779.
8. Yang Q, Xie B, Hu M, et al. Thoracoscopic anatomic pulmonary segmentectomy: a 3-dimensional guided imaging system for lung operations. Interact Cardiovasc Thorac Surg 2016;23:183-9. [Crossref] [PubMed]
9. Baste JM, Soldea V, Lachkar S, et al. Visualization of the 3D model and exploration on the fluid console. Asvide 2018;5:406. Available online: http://www.asvide.com/article/view/24363
10. Baste JM, Soldea V, Lachkar S, et al. Pleural dye marking before right S1 segmentectomy for better margin resection. Asvide 2018;5:407. Available online: http://www.asvide.com/article/view/24364
11. Baste JM, Soldea V, Lachkar S, et al. Left S6 with capture of the robotic console and 3D model. Asvide 2018;5:408. Available online: http://www.asvide.com/article/view/24365
12. Baste JM, Soldea V, Lachkar S, et al. Double segmentectomies right S2 + S6 with capture of the robotic console and 3D model. Asvide 2018;5:409. Available online: http://www.asvide.com/article/view/24366
13. Baste JM, Soldea V, Lachkar S, et al. Left S2 segmentectomy using the per-operative 3D reconstruction. Asvide 2018;5:410. Available online: http://www.asvide.com/article/view/24367
14. Volonté F, Buchs NC, Pugin F, et al. Augmented reality to the rescue of the minimally invasive surgeon. The usefulness of the interposition of stereoscopic images in the Da Vinci™ robotic console. Int J Med Robot 2013;9:e34-8. [Crossref] [PubMed]
15. Volonté F, Pugin F, Bucher P, et al. Augmented reality and image overlay navigation with OsiriX in laparoscopic and robotic surgery: not only a matter of fashion. J Hepatobiliary Pancreat Sci 2011;18:506-9. [Crossref] [PubMed]
16. Firefly fluorescence imaging. Available online: http://www.roboticsurgery.com.au/product/firefly-fluorescence-imaging/
17. Zhou J, Yang F, Jiang G, et al. Applications of indocyanine green based near-infrared fluorescence imaging in thoracic surgery. J Thorac Dis 2016;8:S738-43. [Crossref] [PubMed]
18. Diana M, Soler L, Agnus V, et al. Prospective evaluation of precision multimodal gallbladder surgery navigation: virtual reality, near-infrared fluorescence, and X-ray-based intraoperative cholangiography. Ann Surg 2017;266:890-7. [Crossref] [PubMed]
19. Predina JD, Newton AD, Keating J, et al. Intraoperative molecular imaging combined with positron emission tomography improves surgical management of peripheral malignant pulmonary nodules. Ann Surg 2017;266:479-88. [Crossref] [PubMed]
20. Soler L, Nicolau S, Pessaux P, et al. Real-time 3D image reconstruction guidance in liver resection surgery. Hepatobiliary Surg Nutr 2014;3:73-81. [PubMed]
21. Bourdel N, Collins T, Pizarro D, et al. Use of augmented reality in laparoscopic gynecology to visualize myomas. Fertil Steril 2017;107:737-9. [Crossref] [PubMed]
22. Khor WS, Baker B, Amin K, et al. Augmented and virtual reality in surgery—the digital surgical environment: applications, limitations and legal pitfalls. Ann Transl Med 2016;4:454. [Crossref] [PubMed]