White paper: definitions of artificial intelligence and autonomous actions in clinical surgery
, ... Abstract
This white paper documents the consensus opinion of the expert members of the Editorial Board of Artificial Intelligence Surgery regarding the definitions of artificial intelligence and autonomy in regards to surgery and how the digital evolution of surgery is interrelated with the various forms of robotic-assisted surgery. It was derived from a series of video conference discussions, and the survey and results were subsequently revised and approved by all authors.
Keywords
An Erratum is published on 13 Jul 2022. You can check it here.
INTRODUCTION
There is increasing awareness and concerns in the surgical community regarding concepts of artificial intelligence (AI) in surgery. One problem is the proliferation of terminologies with which the clinician is faced, which can certainly be confusing and lead to misperceptions and anxiety. Surgeons today are rapidly adapting to the presence of robotics in the operating room (OR) but less so to AI-related medical devices for surgery. This consensus project, initiated by the journal Artificial Intelligence Surgery (AIS), sought to use the authoritative input of world experts on the topic to clarify the terminology around and current roles of the digital surgery revolution.
METHODS
Due to the misconceptions and the exponential rise of AI in the field of surgery, we recently launched the first journal dedicated exclusively to AIS, entitled Artificial Intelligence Surgery (www.oaepublish.com/ais)[1]. We hosted a video conference on the topic of autonomous actions in surgery among the Editorial Board (EB) members of our journal and sent out a questionnaire to find a consensus regarding the current issues and perceptions of surgical AI, the role and potential of machine learning (ML), and the future of autonomous actions with regards to surgical robotics. We also attempted to ascertain whether the future of robotic surgery will be operating from a remote console and with telemanipulation, the surgeon should remain at the OR table and use robotic assistance with handheld intelligent systems, or a mixture of the two[2].
Consensus meetings
A 1.5-h video conference was arranged with the 60 members of the EB of AIS at the time (there are currently 66 EB members). This was followed up with an electronic questionnaire designed to find consensus regarding the current definitions, issues, and perceptions of surgical AI, the role and potential of ML, and the future of autonomous actions with regards to surgical robotics and autonomous actions (AA) in surgery[3]. The questions posed to the panel and on the questionnaire were conceived by a task force of the Editor-in-Chief (Andrew A. Gumbs), members of the Associate Editorial Board (Elie Chouillard, Konrad Karcz, and Roland Croner), and a member of the technology research team from the Ludwig-Maximilians-University Munich, Munich, Germany (Frank Alexander).
Demographics
The 60 EB members represent the following demographics: 85% are surgeons; 33.3% are female; and there are 12 general surgeons, 11 hepatic-pancreatic and biliary surgeons, 7 surgical oncologists, 5 computer scientists, 4 pediatric surgeons, 3 bariatric surgeons, 3 thoracic surgeons, 3 transplant surgeons, 2 colorectal surgeons, 2 endocrine surgeons, 2 gynecologists/obstetricians, 2 radiologists, 2 radiation oncologists, 1 breast surgeon, and 1 cardiac surgeon. The median age of the EB is 51 (range, 26-73). All board members were selected for their recognized expertise in the field of AI in surgery. Almost 90% of the EB members of AIS have used complete robotic surgical systems (CRSS) with a console and telemanipulation, with 75% of them having a CRSS in their hospital. Currently, some form of robotics is used in the OR by more than 93% of EB members, including handheld robotics such as robotically controlled laparoscope holders, handheld surgical instruments, and powered staplers. Seven (12%) board members have a doctoral degree in computer science/AI and four (7%) have a master’s degree in AI.
RESULTS
Panel demographics
Twenty-two (37%) EB members participated in the video conference, of whom 17 (73%) completed the online survey. The EB members agreed during the conference session that AI in surgery is any instance of ML in the care of surgical patients. Preoperative examples include AI in radiology, enabling enhanced surveillance and diagnostics such as in radiomics and the utilization of algorithms to help predict proper medical management of patients[2,4,5]. This is distinct from AIS, which involves ML during surgery. For surgery, being an interventional art, AIS has been more closely defined as intelligent or AA done with a computer interface and a robotic device. Unlike automatic actions that do not involve any sensors or interpretation of sensor information with algorithms, AA are modified based on incoming data that allow the device to adapt independently from human control.
The questionnaire revealed that 18.8% of EB members use robotics for logistics and supplies, while 12.5% use robotics for interventional procedures such as interventional radiology. Of those who use robotics, 87.5% use robotics during surgery, 6.3% during non-surgical procedures such as endoscopy and interventional radiologic procedures, and 6.3% during diagnostic procedures.
WHAT ARTIFICIAL INTELLIGENCE IN SURGERY IS
For a device or action to be considered Artificial Intellgence Surgery (AIS), movement visible to the human eye is not a requirement. AIS involves ML, which encompasses algorithms that enable computers or machines to make decisions that they were not necessarily explicitly programmed to do. AI consists of ML and deep learning (DL), which are algorithms structured to mimic the human brain to be able to interpret more complex situations and make better decisions [Figure 1]. Computer vision (CV) is a specific area of AI that enables computers to effectively perceive and understand visual things. Any use of algorithms, ML, DL, or CV in the OR that enables surgeons to have improved short- or long-term outcomes should be considered an example of AIS. On the contrary, AI used in the pre- or postoperative setting should be defined as AI in surgery or AI in the management of surgical patients, and it more closely approximates AI in the non-interventional medical fields.
Figure 1. AI in computerized visualization involves machine learning (ML), which encompasses deep learning (DL). Much of computer vision (CV) is made possible through the neural networks of DL. Computers and robots may be able to attain autonomous surgical actions through a combination of traditional CV, but also through instrument priors, motion analysis, and other non-visual data points.
Autonomy in AI and robotics
The concept of autonomy, or independent decision making, is central to AI in surgery and the future of surgical robotics. True independent decision making would need to be driven by AI and optimally learn from its mistakes (ML). Currently, six levels of autonomy have been defined: Level 0 indicates no autonomy; Level 1 is telemanipulation of a robot; Level 2 is autonomy limited to one action or co-manipulation; Level 3 is conditional autonomy where the robot has more than one autonomous action that the surgeon can choose from; Level 4 is when the device makes some autonomous decisions but is still under a doctor’s control; and Level 5 is considered full autonomy[6]. The problem is that many surgeons think that autonomy is an “all or nothing concept” (i.e., either full autonomy is present, or the action should not be considered autonomous). This is made clearer when we consider the concepts or “weak” and “strong” AI, with Level 5 usually being considered “strong” AI[3].
Classes of medical device risk
The Food and Drug Administration (FDA) in the USA and the Medical Device Regulation (MDR) system of Europe currently consider all AI applications in surgery to be medical devices and subject to medical device regulatory standards. Medical devices are divided into four risk categories used to adjudicate the introduction and adoption of new products [Figure 2]. The system defining these four classes is the following: Class 1 includes devices with limited contact with the patient and includes items such as surgical instruments; Class 2 includes contact lenses and magnetic resonance imaging devices; Class 3 comprises mainly implantable items such as dialysis catheters, implantable meshes, or prostheses for joint replacements; and Class 4 includes pacemakers, automatic implantable cardioverter-defibrillators, artificial heart-assist devices, etc.[3]. Although surgical instruments ranging from complete robotic surgical systems to energy devices and intelligent handheld powered staplers are currently classified as Class 1, the panel felt that a fifth class should probably be developed to account for surgical instruments with Level 1 autonomy (telemanipulation) and certainly for surgical instruments with limited (Level 2) or conditional autonomy (Level 3) and beyond.
Figure 2. Levels of risk for new technologies in healthcare depending on regions.
Definitions of robotic surgery
Complete robotic surgical systems
Robotic-assisted surgery is a form of surgery that today utilizes telemanipulation and can be used via an open approach or minimally invasively. Its definition is currently poorly understood[7]. Different forms of robotic-assisted surgery include complete robotic surgical systems with four arms, either integrated or modular. One arm usually holds the laparoscope, with the others used for retraction and working arms for the primary surgeon.
Console or non-console surgery
The dominant current systems on the market are robotic-assisted console surgery. They involve remote control with the surgeon operating at a console. Prototypes exist with one, three, four, and even six robotic arms, but all have the primary surgeon at a console and require an assistant who stands at the bedside to exchange instruments or use needed devices that are not available for the robot such as suction/irrigation. For future systems using robotic assistance and no console, with the surgeon standing by the patient throughout surgery, the term “non-console robotic-assisted surgeon” might be appropriate.
Telemanipulation
A complete robotic surgical system (da Vinci Surgical System, Intuitive Surgical, Sunnyvale, CA, USA and Versius Surgical Robotic System, CMR, Cambridge, UK, and HUGO, Medtronic, Dublin, Ireland) is simply telemanipulation and only has Level 1 autonomy. Thus, the term “robotic-assisted surgery” is probably more appropriate than AIS or AI in surgery given the current level of technology.
Handheld robotic surgery
Multiple examples of handheld robotic-assisted surgical instruments exist. In general, these devices currently do not fall under the usual definition of robotic surgery[8]. Examples of handheld robotic-assisted surgical devices include robotically controlled laparoscope holders that can be autoclavable and connected directly to the OR table (ViKY, Videoendoskopy, Endocontrol, Grenoble, France), handheld powered articulating instruments (Jaimy, Endocontrol, Grenoble, France and HandX, Human Extensions, Assia Medical, Tel Aviv-Yafo, Israel), and some powered staplers (Signia, Medtronic, Dublin, Ireland).
The most famous early example of robotic assistance in surgery was the Automated Endoscopic System for Optimal Positioning (AESOP) robotically controlled laparoscope holder (Automated Endoscopic System for Optimal Positioning, Computer Motion Inc., Goleta, CA, USA). Popular and broadly utilized, AESOP was purchased by Intuitive Surgical and promptly removed from the market. Unlike the ViKY laparoscope holder, AESOP was extremely heavy, could not be autoclaved, and was either attached to the operating table or positioned on a cart. Consequently, the panel did not consider it a true example of handheld robotic-assisted surgery.
Examples of weak AIS
AI is permeating most of the common devices used in the consumer world and the same is true for many of the instruments used today in the OR. While the panel recognized the importance of these “IoT” devices, they were not considered independent AI medical devices. Three examples of importance to surgeons are given.
Energy devices
Multiple energy devices use algorithms to calculate hundreds of thousands of data points per second to determine the optimal coagulation velocity. Although no movement or gesture is involved, these devices should be considered an example of AIS because a machine is using algorithms to seal vessels and independently determine how long this action should take; this is an example of limited or Level 2 autonomy. Another example of this would be the Ligasure vessel sealing device (Medtronic, Dublin, Ireland) because it has only one option/setting. Energy devices that have more than one option for a surgeon to choose from would constitute Level 3 autonomy. Some devices have two options, sealing/cutting (Thunderbeat, Olympus, Japan), while the Ultrasonic shears have three vessel sealing levels (Harmonic scalpel, Ethicon Endosurgery Cincinnati, OH, USA). Finally, the panel did not consider these as an example of a handheld robotic-assisted instrument because their movements are completely controlled by the operating surgeon.
Powered staplers
Powered staplers currently on the market consist of automatic and autonomous versions. The Echelon Stapler (Echelon, Ethicon Endosurgery, Cincinnati, OH, USA) does not have any sensors and simply fires once activated. Conversely, the Signia stapler (Signia Stapler, Medtronic, Dublin, Ireland) has sensors that block the stapling action if the tissue is too thick or not thick enough. Consequently, it can be considered an example of AIS. As there is only one setting, it is an example of Level 2 autonomy. Furthermore, because it is handheld, it is defined as a handheld robotic-assisted surgical instrument, as discussed in the previous section.
Operating table integrated with a complete surgical system
An OR table (TruSystem™ 7500, Trumpf Medical, Puchheim, Germany) has been integrated with the latest complete robotic surgical system made by Intuitive Surgical (da Vinci Surgical System, Intuitive Surgical, Sunnyvale, CA, USA). When the OR table angles are altered, the integrated table and four robotic arms move automatically. This is an example of Level 2 autonomy. This technology could also be useful for non-console robotic-assisted surgery.
Another and even more advanced version is that created by Auris Robotics (Johnson & Johnson, Redwood City, CA, USA). Their console-based robotic platform is fully integrated into a proprietary operating table and has six arms able to control both laparoscopic and flexible endoscopic instruments. This is currently in Beta testing and is expected to be on the market by 2023.
Additional consensus opinions
In addition to the categorization of the field and definitions agreed on above, several other statements regarding AIS were agreed on by the majority of the panelists (statements not obtaining majority agreement are not listed):
1. AI-driven medical devices such as radiomics or surgical outcomes predictors that can operate autonomously should be allowed and encouraged but always verified by a human medical professional.
2. Level 5 autonomy could eventually be possible and perhaps even is the logical conclusion to advancements in the surgical arts. However, according to the human paradigm in the early third millennium, surgery should not be done without the control of a human surgeon who is fully capable of performing and completing the procedure via robotic assistance, standard laparoscopy, and/or open surgical techniques.
3. Initially, it may be preferable for only parts of surgical procedures to become fully automated with an emphasis on creating autonomous dexemes (parts of surgical gestures) and then surgemes (parts of entire operations) that are safe for patients and can be proven to provide patient benefit.
4. Autonomous dissection may be the riskiest part of surgery to be done autonomously and should be evaluated last. Initial research on autonomous actions should focus on the development of sensors so that more intelligent devices can be developed that increase safety and precision during operations.
5. Enhanced computer vision with augmented reality, virtual reality, and mixed reality are fields of AI that have and should continue to be developed initially because these technologies have the greatest potential to positively improve surgical outcomes and reduce short-term complications.
6. Handheld robotic assistance and non-console systems may be able to provide haptics better than robotic-assisted console solutions. Increased funding should go into these approaches as they may be the safest and most cost-effective way towards more autonomous actions in surgery.
7. Whether or not a console is used should be up to the operating surgeon who uses robotic assistance. Nonetheless, we believe that more funding needs to be funneled towards technologies that advance handheld robotic-assisted instrumentation and non-console robotic-assisted systems.
8. It is hoped that advances in automatic and autonomous actions in conjunction with enhanced monitoring and AI in the OR and via simulations before surgery that surgeons will be able to operate with less stress and more valuable information, and that these innovations will ultimately result in short and long-term benefits for patients.
CONCLUSION
Some surgeons and patients think that we are far away from autonomy in surgery, while others feel we are much closer[9]. Since most surgeons are not well versed in the academic definitions of autonomy, AI, ML, or even surgical robotics, this discrepancy is particularly hard to resolve, which makes it difficult to properly inform patients of their treatment.
We present the results of an informal consensus of thought leaders in this emerging field and attempt to clarify some common misunderstandings by better defining the terminology surrounding both AI and robotics for surgeons as well as some pathways towards the future of this disruptive surgical innovation.
DECLARATIONS
Authors’ contributionsMade substantial contributions to conception and design of the study and performed data analysis and interpretation: Gumbs AA, Alexander F, Karcz K, Grasso V, Illanes A, Swanström LL
Performed data acquisition, as well as provided administrative, technical, and material support: Chouillard E, Croner R, Coles-Black J, de Simone B, Gagner M, Gayet B, Ishizawa T, Milone L, Özmen MM, Piccoli M, Spiedel S, Spolverato G, Sylla P, Vilaça J
Contributed to conceptualization, validation, review & editing: Abu Hilal M
Availability of data and materialsNot applicable.
Financial support and sponsorshipNone.
Conflicts of interestAll authors declared that there are no conflicts of interest.
Ethical approval and consent to participateNot applicable.
Consent for publicationNot applicable.
Copyright© The Author(s) 2022.
REFERENCES
1. Gumbs AA, Perretta S, d’Allemagne B, Chouillard E. What is artificial intelligence surgery? Art Int Surg 2021;1:1-10.
2. Gumbs AA, Abu-hilal M, Tsai T, Starker L, Chouillard E, Croner R. Keeping surgeons in the loop: are handheld robotics the best path towards more autonomous actions? Art Int Surg 2021;1:38-51.
3. Gumbs AA, Frigerio I, Spolverato G, et al. Artificial intelligence surgery: how do we get to autonomous actions in surgery? Sensors (Basel) 2021;21:5526.
4. Lao J, Chen Y, Li ZC, et al. A deep learning-based radiomics model for prediction of survival in glioblastoma multiforme. Sci Rep 2017;7:10353.
5. Attanasio A, Scaglioni B, De Momi E, Fiorini P, Valdastri P. Autonomy in surgical robotics. Annu Rev Control Robot Auton Syst 2021;4:651-79.
6. Wesdorp NJ, Hellingman T, Jansma EP, et al. Advanced analytics and artificial intelligence in gastrointestinal cancer: a systematic review of radiomics predicting response to treatment. Eur J Nucl Med Mol Imaging 2021;48:1785-94.
7. Hashimoto DA, Rosman G, Rus D, Meireles OR. Artificial intelligence in surgery: promises and perils. Ann Surg 2018;268:70-6.
8. Gumbs AA, De Simone B, Chouillard E. Searching for a better definition of robotic surgery: is it really different from laparoscopy? Mini-invasive Surg 2020;4:90.
Cite This Article
Export citation file: BibTeX | RIS
OAE Style
Gumbs AA, Alexander F, Karcz K, Chouillard E, Croner R, Coles-Black J, de Simone B, Gagner M, Gayet B, Grasso V, Illanes A, Ishizawa T, Milone L, Özmen MM, Piccoli M, Spiedel S, Spolverato G, Sylla P, Vilaça J, Abu Hilal M, Swanström LL. White paper: definitions of artificial intelligence and autonomous actions in clinical surgery. Art Int Surg 2022;2:93-100. http://dx.doi.org/10.20517/ais.2022.10
AMA Style
Gumbs AA, Alexander F, Karcz K, Chouillard E, Croner R, Coles-Black J, de Simone B, Gagner M, Gayet B, Grasso V, Illanes A, Ishizawa T, Milone L, Özmen MM, Piccoli M, Spiedel S, Spolverato G, Sylla P, Vilaça J, Abu Hilal M, Swanström LL. White paper: definitions of artificial intelligence and autonomous actions in clinical surgery. Artificial Intelligence Surgery. 2022; 2(2): 93-100. http://dx.doi.org/10.20517/ais.2022.10
Chicago/Turabian Style
Gumbs, Andrew A., Frank Alexander, Konrad Karcz, Elie Chouillard, Roland Croner, Jasamine Coles-Black, Belinda de Simone, Michel Gagner, Brice Gayet, Vincent Grasso, Alfredo Illanes, Takeaki Ishizawa, Luca Milone, Mehmet Mahir Özmen, Micaela Piccoli, Stefanie Spiedel, Gaya Spolverato, Patricia Sylla, Jaime Vilaça, Mohammad Abu Hilal, Lee L. Swanström. 2022. "White paper: definitions of artificial intelligence and autonomous actions in clinical surgery" Artificial Intelligence Surgery. 2, no.2: 93-100. http://dx.doi.org/10.20517/ais.2022.10
ACS Style
Gumbs, AA.; Alexander F.; Karcz K.; Chouillard E.; Croner R.; Coles-Black J.; de Simone B.; Gagner M.; Gayet B.; Grasso V.; Illanes A.; Ishizawa T.; Milone L.; Özmen MM.; Piccoli M.; Spiedel S.; Spolverato G.; Sylla P.; Vilaça J.; Abu Hilal M.; Swanström LL. White paper: definitions of artificial intelligence and autonomous actions in clinical surgery. Art. Int. Surg. 2022, 2, 93-100. http://dx.doi.org/10.20517/ais.2022.10
About This Article
Copyright
Data & Comments
Data
Cite This Article 72 clicks
Like This Article 29
likes
Comments
Comments must be written in English. Spam, offensive content, impersonation, and private information will not be permitted. If any comment is reported and identified as inappropriate content by OAE staff, the comment will be removed without notice. If you have any queries or need any help, please contact us at support@oaepublish.com.