Applications of machine learning in surgery: ethical considerations

Data bias, fairness, and equity

The principle of justice in medical ethics deals with the equal distribution of resources within a society and avoiding discrimination of individuals^[16]. Considering the data-driven nature of ML, the machine’s learning process can be compromised by the specific characteristics of training data. Systematic biases in societies are reflected in clinical data collections and have limited the inclusion of underrepresented minorities in databases^[18,19]. Developing ML algorithms based on biased datasets not only potentially perpetuates systematic inequities in societies but also can limit the performance of ML as a diagnostic and treatment tool due to the lack of generalizability^[20]. As with other surgical innovations, it is crucial that innovation in surgical ML is complemented by efforts to reduce the risk of bias. We must ensure that the benefits of these new technologies are broadly shared across different courtiers and among people with diverse gender, race, ethnicity, culture, and socioeconomic status. A possible approach to prevent these already recognized biases is to expand surgeons’ involvement and supervision on clinical data collection. Before providing data scientists with clinical data to develop ML algorithms, it is our responsibility to critically assess the quality of the data registries and ensure the inclusion of the underrepresented minorities into the dataset.

With the rapid development of data sciences and the expansion of data-driven clinical research, the publication of freely available datasets has become invaluable. Many public databases collect heterogeneous and multidimensional medical and surgical data on open platforms. The success of clinical research based on open access data is highly dependent on the quality of the data. As researchers, it is important to remember the value of making data publicly available in a standardized machine-readable format. Moreover, it is recommended to publish both negative and positive result data to reduce the data bias^[21].

To establish ground truth in surgical data sciences that involve supervised ML and computer vision, the need for correctly labeled data is a key challenge. The process of annotating surgical data such as surgical videos is a time-consuming task and requires skilled personnel, as wrong annotations can dramatically affect the performance of the trained model. In this regard, the Society of Gastrointestinal and Endoscopic Surgeons has provided a set of recommendations on a general framework to standardize the annotation of surgical video data. Standardizing the video annotation process may allow the combination or concatenation of heterogeneous datasets from different sources^[22].

Data privacy and security

Given that the ML algorithms must be trained with a sizeable amount of personal health information, data privacy and security are prominent challenges^[20]. Anonymization (the removal of private data) and pseudonymization (replacement of sensitive data by one or more artificial identifiers or pseudonyms allowing re-attribution using a look-up table) are currently the most common privacy preservation techniques for medical datasets^[23]. As a high-dimensional data domain in surgery, we handle different types of personal information, including patients’ electronic files, medical images, and surgical videos. Anonymization of medical images and surgical videos requires removing all pertinent metadata entries (e.g., patient name, gender, date, camera type, etc.). Pseudonymization of such high-dimensional data poses additional difficulties, as it requires data deletion and data manipulation as well as the safekeeping of the look-up tables for reversing the process^[23]. While all the de-identification methods aim to reduce the risk of re-identification and minimize the loss of data utility, even the most complex and sophisticated of solutions never offer informative data with zero risk of re-identification^[24]. As a consequence, complex surgical datasets are prone to technical errors, resulting in re-identification or inaccuracy of data, and, therefore, must be processed more rigorously. Since appropriate data collection is a crucial step toward developing surgical ML algorithms, the trade-offs among data accuracy, interpretability, and privacy need to be further researched.

Technical robustness and safety

The principles of beneficence and non-maleficence imply that the application of ML technologies in surgery must be beneficial and non-maleficent for the individual patient^[16]. It is important to remember that some ML use cases have the potential to violate these principles. For example, a falsely optimistic prognosis tool, trained based on an unsuitable dataset for training an ML model, could trigger potentially inappropriate surgical intervention. When applied wisely, there is a tremendous opportunity for surgeons to use ML technologies pre-, intra-, and post-operatively to improve patient care^[25]. The optimal outcome for patients can only be expected with surgeons who can make informed decisions on when to apply an ML algorithm and how to interpret its results. Therefore, using ML for high-stakes decision-making in surgery must be assessed rigorously and implemented in an evidence-based fashion similar to the introduction of other surgical innovations^[10].

While medical device manufacturers and healthcare institutions increasingly apply ML to innovate their products, specific regulatory requirements need to be met before these can be used in clinical practice. To this end, the European Medical Device Regulation and the Food and Drug Administration in the United States have imposed stringent requirements in filling regulatory gaps to develop ML-based medical devices and software tools^[26-28].

Autonomy, transparency, and explainability

The principles of patient autonomy refer to the rights of patients to make decisions about their medical care based on their own values and beliefs^[16]. Patients should be offered options along with sufficient relevant medical information and allowed to make voluntary choices about potentially life-changing healthcare interventions. While applying ML as a decision-aid tool, lack of explainability or so-called “black-box” design is a controversial topic with implications extending beyond AI’s technical properties. Explainability can be defined as a characteristic of an ML-driven tool allowing human intelligence to reconstruct why a certain algorithm came up with the presented prediction or decision^[29]. Concerning autonomy, the lack of explainable AI has implications for both patients and surgeons. Before signing informed consent for surgical intervention, patients must receive comprehensive and understandable information regarding the nature and risks of the procedure and alternative options. To date, disclosing the use of medical ML algorithms is not a mandatory requirement of a medical informed consent. The often-unexplainable nature of algorithms makes it potentially difficult to discuss with patients and explain the pros and cons. However, failure to disclose the use of an ML algorithm in the process of patient management may undermine patients’ autonomy, be considered a betrayal of trust, and jeopardize the doctor-patient relationship^[29]. Surgeons’ involvement early in the designing process of ML algorithms may help improve the interpretability of data-driven analyses. Despite advances in ML, one should realize that machines still cannot provide clinically meaningful context unless being interpreted by human physicians^[25]. Perhaps shared decision-making with patients or their surrogates is the best way to finally decide upon the consequences of using ML-based technologies.