1Departement de Chirurgie Digestive, Centre Hospitalier Intercommunal de, Poissy/Saint-Germain-en-Laye, Poissy 78300, France.
2Department of Surgery, University of Magdeburg, Magdeburg 39106, Germany.
3Department of General, Visceral, and Transplant Surgery, Ludwig-Maximilians-University Munich, Munich 80539, Germany.
4Department of Surgery, Austin Health, The University of Melbourne, Heidelberg 3084, Victoria, Australia.
5Department of Surgery, Hôpital du Sacre Coeur, Montreal H4J 1C5, Canada.
6Department of Digestive, Oncologic and Metabolic Surgery, Institut Mutualiste Montsouris, Université de Paris, Paris 75014, France.
7Family Christian Health Center, Department of Information Systems, Science and Technology, Harvey, IL 60426, USA.
8INKA - Innovation Laboratory for Image Guided Therapy, Otto-von-Guericke University Magdeburg, Magdeburg 39120, Germany.
9Hepato-Biliary-Pancreatic Surgery Division, Department of Surgery, Graduate School of Medicine, The University of Tokyo, Tokyo 113-8654, Japan.
10Department of General and Robotic Surgery, The Brooklyn Hospital Center, New York, NY 11201, USA.
11Department of Surgery, Medical School, Istinye University, Istanbul 34010, Turkey.
12Department of General, Emergency Surgery and New Technologies, Baggiovara General Hospital, Modena 41126, Italy.
13Division of Translational Surgical Oncology, National Center for Tumor Diseases Dresden, Dresden 01307, Germany.
14Department of Surgical, Oncological and Gastroenterological Sciences, University of Padova, Padova 35122, Italy.
15Division of Colon and Rectal Surgery, Mount Sinai Hospital, New York, NY 10029, USA.
16Department of General Surgery, Hospital da Luz, Porto 4100-284, Portugal.
17Department of Surgery, Poliambulanza Foundation Hospital, Via Bissolati, Brescia 25100, Italy.
18Institute of Image-Guided Surgery, IHU-Strasbourg, Strasbourg 67000, France.
Correspondence to: Prof. Andrew A. Gumbs, Departement de Chirurgie Digestive, Centre Hospitalier Intercommunal de, Poissy/Saint-Germain-en-Laye, 10, Rue du Champ Gaillard, Poissy 78300, France. E-mail: firstname.lastname@example.org
© The Author(s) 2022. Open Access This article is licensed under a Creative Commons Attribution 4.0 International License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, sharing, adaptation, distribution and reproduction in any medium or format, for any purpose, even commercially, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.
This white paper documents the consensus opinion of the expert members of the Editorial Board of Artificial Intelligence Surgery regarding the definitions of artificial intelligence and autonomy in regards to surgery and how the digital evolution of surgery is interrelated with the various forms of robotic-assisted surgery. It was derived from a series of video conference discussions, and the survey and results were subsequently revised and approved by all authors.
Artificial intelligence surgery, artificial intelligence in surgery, autonomous actions, automatic, autonomy, consensus, white paper
There is increasing awareness and concerns in the surgical community regarding concepts of artificial intelligence (AI) in surgery. One problem is the proliferation of terminologies with which the clinician is faced, which can certainly be confusing and lead to misperceptions and anxiety. Surgeons today are rapidly adapting to the presence of robotics in the operating room (OR) but less so to AI-related medical devices for surgery. This consensus project, initiated by the journal Artificial Intelligence Surgery (AIS), sought to use the authoritative input of world experts on the topic to clarify the terminology around and current roles of the digital surgery revolution.
Due to the misconceptions and the exponential rise of AI in the field of surgery, we recently launched the first journal dedicated exclusively to AIS, entitled Artificial Intelligence Surgery (www.aisjournal.net). We hosted a video conference on the topic of autonomous actions in surgery among the Editorial Board (EB) members of our journal and sent out a questionnaire to find a consensus regarding the current issues and perceptions of surgical AI, the role and potential of machine learning (ML), and the future of autonomous actions with regards to surgical robotics. We also attempted to ascertain whether the future of robotic surgery will be operating from a remote console and with telemanipulation, the surgeon should remain at the OR table and use robotic assistance with handheld intelligent systems, or a mixture of the two.
A 1.5-h video conference was arranged with the 60 members of the EB of AIS at the time (there are currently 66 EB members). This was followed up with an electronic questionnaire designed to find consensus regarding the current definitions, issues, and perceptions of surgical AI, the role and potential of ML, and the future of autonomous actions with regards to surgical robotics and autonomous actions (AA) in surgery. The questions posed to the panel and on the questionnaire were conceived by a task force of the Editor-in-Chief (Andrew A. Gumbs), members of the Associate Editorial Board (Elie Chouillard, Konrad Karcz, and Roland Croner), and a member of the technology research team from the Ludwig-Maximilians-University Munich, Munich, Germany (Frank Alexander).
The 60 EB members represent the following demographics: 85% are surgeons; 33.3% are female; and there are 12 general surgeons, 11 hepatic-pancreatic and biliary surgeons, 7 surgical oncologists, 5 computer scientists, 4 pediatric surgeons, 3 bariatric surgeons, 3 thoracic surgeons, 3 transplant surgeons, 2 colorectal surgeons, 2 endocrine surgeons, 2 gynecologists/obstetricians, 2 radiologists, 2 radiation oncologists, 1 breast surgeon, and 1 cardiac surgeon. The median age of the EB is 51 (range, 26-73). All board members were selected for their recognized expertise in the field of AI in surgery. Almost 90% of the EB members of AIS have used complete robotic surgical systems (CRSS) with a console and telemanipulation, with 75% of them having a CRSS in their hospital. Currently, some form of robotics is used in the OR by more than 93% of EB members, including handheld robotics such as robotically controlled laparoscope holders, handheld surgical instruments, and powered staplers. Seven (12%) board members have a doctoral degree in computer science/AI and four (7%) have a master’s degree in AI.
Twenty-two (37%) EB members participated in the video conference, of whom 17 (73%) completed the online survey. The EB members agreed during the conference session that AI in surgery is any instance of ML in the care of surgical patients. Preoperative examples include AI in radiology, enabling enhanced surveillance and diagnostics such as in radiomics and the utilization of algorithms to help predict proper medical management of patients[2,4,5]. This is distinct from AIS, which involves ML during surgery. For surgery, being an interventional art, AIS has been more closely defined as intelligent or AA done with a computer interface and a robotic device. Unlike automatic actions that do not involve any sensors or interpretation of sensor information with algorithms, AA are modified based on incoming data that allow the device to adapt independently from human control.
The questionnaire revealed that 18.8% of EB members use robotics for logistics and supplies, while 12.5% use robotics for interventional procedures such as interventional radiology. Of those who use robotics, 87.5% use robotics during surgery, 6.3% during non-surgical procedures such as endoscopy and interventional radiologic procedures, and 6.3% during diagnostic procedures.
For a device or action to be considered Artificial Intellgence Surgery (AIS), movement visible to the human eye is not a requirement. AIS involves ML, which encompasses algorithms that enable computers or machines to make decisions that they were not necessarily explicitly programmed to do. AI consists of ML and deep learning (DL), which are algorithms structured to mimic the human brain to be able to interpret more complex situations and make better decisions [Figure 1]. Computer vision (CV) is a specific area of AI that enables computers to effectively perceive and understand visual things. Any use of algorithms, ML, DL, or CV in the OR that enables surgeons to have improved short- or long-term outcomes should be considered an example of AIS. On the contrary, AI used in the pre- or postoperative setting should be defined as AI in surgery or AI in the management of surgical patients, and it more closely approximates AI in the non-interventional medical fields.
Figure 1. AI in computerized visualization involves machine learning (ML), which encompasses deep learning (DL). Much of computer vision (CV) is made possible through the neural networks of DL. Computers and robots may be able to attain autonomous surgical actions through a combination of traditional CV, but also through instrument priors, motion analysis, and other non-visual data points.
The concept of autonomy, or independent decision making, is central to AI in surgery and the future of surgical robotics. True independent decision making would need to be driven by AI and optimally learn from its mistakes (ML). Currently, six levels of autonomy have been defined: Level 0 indicates no autonomy; Level 1 is telemanipulation of a robot; Level 2 is autonomy limited to one action or co-manipulation; Level 3 is conditional autonomy where the robot has more than one autonomous action that the surgeon can choose from; Level 4 is when the device makes some autonomous decisions but is still under a doctor’s control; and Level 5 is considered full autonomy. The problem is that many surgeons think that autonomy is an “all or nothing concept” (i.e., either full autonomy is present, or the action should not be considered autonomous). This is made clearer when we consider the concepts or “weak” and “strong” AI, with Level 5 usually being considered “strong” AI.
The Food and Drug Administration (FDA) in the USA and the Medical Device Regulation (MDR) system of Europe currently consider all AI applications in surgery to be medical devices and subject to medical device regulatory standards. Medical devices are divided into four risk categories used to adjudicate the introduction and adoption of new products [Figure 2]. The system defining these four classes is the following: Class 1 includes devices with limited contact with the patient and includes items such as surgical instruments; Class 2 includes contact lenses and magnetic resonance imaging devices; Class 3 comprises mainly implantable items such as dialysis catheters, implantable meshes, or prostheses for joint replacements; and Class 4 includes pacemakers, automatic implantable cardioverter-defibrillators, artificial heart-assist devices, etc.. Although surgical instruments ranging from complete robotic surgical systems to energy devices and intelligent handheld powered staplers are currently classified as Class 1, the panel felt that a fifth class should probably be developed to account for surgical instruments with Level 1 autonomy (telemanipulation) and certainly for surgical instruments with limited (Level 2) or conditional autonomy (Level 3) and beyond.
Robotic-assisted surgery is a form of surgery that today utilizes telemanipulation and can be used via an open approach or minimally invasively. Its definition is currently poorly understood. Different forms of robotic-assisted surgery include complete robotic surgical systems with four arms, either integrated or modular. One arm usually holds the laparoscope, with the others used for retraction and working arms for the primary surgeon.
The dominant current systems on the market are robotic-assisted console surgery. They involve remote control with the surgeon operating at a console. Prototypes exist with one, three, four, and even six robotic arms, but all have the primary surgeon at a console and require an assistant who stands at the bedside to exchange instruments or use needed devices that are not available for the robot such as suction/irrigation. For future systems using robotic assistance and no console, with the surgeon standing by the patient throughout surgery, the term “non-console robotic-assisted surgeon” might be appropriate.
A complete robotic surgical system (da Vinci Surgical System, Intuitive Surgical, Sunnyvale, CA, USA and Versius Surgical Robotic System, CMR, Cambridge, UK, and HUGO, Medtronic, Dublin, Ireland) is simply telemanipulation and only has Level 1 autonomy. Thus, the term “robotic-assisted surgery” is probably more appropriate than AIS or AI in surgery given the current level of technology.
Multiple examples of handheld robotic-assisted surgical instruments exist. In general, these devices currently do not fall under the usual definition of robotic surgery. Examples of handheld robotic-assisted surgical devices include robotically controlled laparoscope holders that can be autoclavable and connected directly to the OR table (ViKY, Videoendoskopy, Endocontrol, Grenoble, France), handheld powered articulating instruments (Jaimy, Endocontrol, Grenoble, France and HandX, Human Extensions, Assia Medical, Tel Aviv-Yafo, Israel), and some powered staplers (Signia, Medtronic, Dublin, Ireland).
The most famous early example of robotic assistance in surgery was the Automated Endoscopic System for Optimal Positioning (AESOP) robotically controlled laparoscope holder (Automated Endoscopic System for Optimal Positioning, Computer Motion Inc., Goleta, CA, USA). Popular and broadly utilized, AESOP was purchased by Intuitive Surgical and promptly removed from the market. Unlike the ViKY laparoscope holder, AESOP was extremely heavy, could not be autoclaved, and was either attached to the operating table or positioned on a cart. Consequently, the panel did not consider it a true example of handheld robotic-assisted surgery.
AI is permeating most of the common devices used in the consumer world and the same is true for many of the instruments used today in the OR. While the panel recognized the importance of these “IoT” devices, they were not considered independent AI medical devices. Three examples of importance to surgeons are given.
Multiple energy devices use algorithms to calculate hundreds of thousands of data points per second to determine the optimal coagulation velocity. Although no movement or gesture is involved, these devices should be considered an example of AIS because a machine is using algorithms to seal vessels and independently determine how long this action should take; this is an example of limited or Level 2 autonomy. Another example of this would be the Ligasure vessel sealing device (Medtronic, Dublin, Ireland) because it has only one option/setting. Energy devices that have more than one option for a surgeon to choose from would constitute Level 3 autonomy. Some devices have two options, sealing/cutting (Thunderbeat, Olympus, Japan), while the Ultrasonic shears have three vessel sealing levels (Harmonic scalpel, Ethicon Endosurgery Cincinnati, OH, USA). Finally, the panel did not consider these as an example of a handheld robotic-assisted instrument because their movements are completely controlled by the operating surgeon.
Powered staplers currently on the market consist of automatic and autonomous versions. The Echelon Stapler (Echelon, Ethicon Endosurgery, Cincinnati, OH, USA) does not have any sensors and simply fires once activated. Conversely, the Signia stapler (Signia Stapler, Medtronic, Dublin, Ireland) has sensors that block the stapling action if the tissue is too thick or not thick enough. Consequently, it can be considered an example of AIS. As there is only one setting, it is an example of Level 2 autonomy. Furthermore, because it is handheld, it is defined as a handheld robotic-assisted surgical instrument, as discussed in the previous section.
An OR table (TruSystem™ 7500, Trumpf Medical, Puchheim, Germany) has been integrated with the latest complete robotic surgical system made by Intuitive Surgical (da Vinci Surgical System, Intuitive Surgical, Sunnyvale, CA, USA). When the OR table angles are altered, the integrated table and four robotic arms move automatically. This is an example of Level 2 autonomy. This technology could also be useful for non-console robotic-assisted surgery.
Another and even more advanced version is that created by Auris Robotics (Johnson & Johnson, Redwood City, CA, USA). Their console-based robotic platform is fully integrated into a proprietary operating table and has six arms able to control both laparoscopic and flexible endoscopic instruments. This is currently in Beta testing and is expected to be on the market by 2023.
In addition to the categorization of the field and definitions agreed on above, several other statements regarding AIS were agreed on by the majority of the panelists (statements not obtaining majority agreement are not listed):
1. AI-driven medical devices such as radiomics or surgical outcomes predictors that can operate autonomously should be allowed and encouraged but always verified by a human medical professional.
2. Level 5 autonomy could eventually be possible and perhaps even is the logical conclusion to advancements in the surgical arts. However, according to the human paradigm in the early third millennium, surgery should not be done without the control of a human surgeon who is fully capable of performing and completing the procedure via robotic assistance, standard laparoscopy, and/or open surgical techniques.
3. Initially, it may be preferable for only parts of surgical procedures to become fully automated with an emphasis on creating autonomous dexemes (parts of surgical gestures) and then surgemes (parts of entire operations) that are safe for patients and can be proven to provide patient benefit.
4. Autonomous dissection may be the riskiest part of surgery to be done autonomously and should be evaluated last. Initial research on autonomous actions should focus on the development of sensors so that more intelligent devices can be developed that increase safety and precision during operations.
5. Enhanced computer vision with augmented reality, virtual reality, and mixed reality are fields of AI that have and should continue to be developed initially because these technologies have the greatest potential to positively improve surgical outcomes and reduce short-term complications.
6. Handheld robotic assistance and non-console systems may be able to provide haptics better than robotic-assisted console solutions. Increased funding should go into these approaches as they may be the safest and most cost-effective way towards more autonomous actions in surgery.
7. Whether or not a console is used should be up to the operating surgeon who uses robotic assistance. Nonetheless, we believe that more funding needs to be funneled towards technologies that advance handheld robotic-assisted instrumentation and non-console robotic-assisted systems.
8. It is hoped that advances in automatic and autonomous actions in conjunction with enhanced monitoring and AI in the OR and via simulations before surgery that surgeons will be able to operate with less stress and more valuable information, and that these innovations will ultimately result in short and long-term benefits for patients.
Some surgeons and patients think that we are far away from autonomy in surgery, while others feel we are much closer. Since most surgeons are not well versed in the academic definitions of autonomy, AI, ML, or even surgical robotics, this discrepancy is particularly hard to resolve, which makes it difficult to properly inform patients of their treatment.
We present the results of an informal consensus of thought leaders in this emerging field and attempt to clarify some common misunderstandings by better defining the terminology surrounding both AI and robotics for surgeons as well as some pathways towards the future of this disruptive surgical innovation.
Made substantial contributions to conception and design of the study and performed data analysis and interpretation: Gumbs AA, Alexander F, Karcz K, Grasso V, Illanes A, Swanström LL
Performed data acquisition, as well as provided administrative, technical, and material support: Chouillard E, Croner R, Coles-Black J, de Simone B, Gagner M, Gayet B, Ishizawa T, Milone L, Özmen MM, Piccoli M, Spiedel S, Spolverato G, Sylla P, Vilaça J
Contributed to conceptualization, validation, review & editing: Abu Hilal MAvailability of data and materials
Not applicable.Financial support and sponsorship
None.Conflicts of interest
All authors declared that there are no conflicts of interest.Ethical approval and consent to participate
Not applicable.Consent for publication
© The Author(s) 2022.
1. Gumbs AA, Perretta S, d’Allemagne B, Chouillard E. What is artificial intelligence surgery? Art Int Surg 2021;1:1-10.DOI
2. Gumbs AA, Abu-hilal M, Tsai T, Starker L, Chouillard E, Croner R. Keeping surgeons in the loop: are handheld robotics the best path towards more autonomous actions? Art Int Surg 2021;1:38-51.DOI
3. Gumbs AA, Frigerio I, Spolverato G, et al. Artificial intelligence surgery: how do we get to autonomous actions in surgery? Sensors (Basel) 2021;21:5526.DOIPubMed PMC
4. Lao J, Chen Y, Li ZC, et al. A deep learning-based radiomics model for prediction of survival in glioblastoma multiforme. Sci Rep 2017;7:10353.DOIPubMed PMC
5. Attanasio A, Scaglioni B, De Momi E, Fiorini P, Valdastri P. Autonomy in surgical robotics. Annu Rev Control Robot Auton Syst 2021;4:651-79.DOI
6. Wesdorp NJ, Hellingman T, Jansma EP, et al. Advanced analytics and artificial intelligence in gastrointestinal cancer: a systematic review of radiomics predicting response to treatment. Eur J Nucl Med Mol Imaging 2021;48:1785-94.DOIPubMed PMC
7. Hashimoto DA, Rosman G, Rus D, Meireles OR. Artificial intelligence in surgery: promises and perils. Ann Surg 2018;268:70-6.DOIPubMed PMC
8. Gumbs AA, De Simone B, Chouillard E. Searching for a better definition of robotic surgery: is it really different from laparoscopy? Mini-invasive Surg 2020;4:90.DOI
9. Guiot J, Vaidyanathan A, Deprez L, et al. Development and validation of an automated radiomic CT signature for detecting COVID-19. Diagnostics (Basel) 2020;11:41.DOIPubMed PMC
Gumbs AA, Alexander F, Karcz K, Chouillard E, Croner R, Coles-Black J, de Simone B, Gagner M, Gayet B, Grasso V, Illanes A, Ishizawa T, Milone L, Özmen MM, Piccoli M, Spiedel S, Spolverato G, Sylla P, Vilaça J, Abu Hilal M, Swanström LL. White paper: definitions of artificial intelligence and autonomous actions in clinical surgery. Art Int Surg 2022;2:93-100. http://dx.doi.org/10.20517/ais.2022.10