Application of Machine Learning in the Field of Intraoperative Neurophysiological Monitoring: A Narrative Review

Abstract

Intraoperative neurophysiological monitoring (IONM) is being applied to a wide range of surgical fields as a diagnostic tool to protect patients from neural injuries that may occur during surgery. However, several contributing factors complicate the interpretation of IONM, and it is labor- and training-intensive. Meanwhile, machine learning (ML)-based medical research has been growing rapidly, and many studies on the clinical application of ML algorithms have been published in recent years. Despite this, the application of ML to IONM remains limited. Major challenges in applying ML to IONM include the presence of non-surgical contributing factors, ambiguity in the definition of false-positive cases, and their inter-rater variability. Nevertheless, we believe that the application of ML enables objective and reliable IONM, while overcoming the aforementioned problems that experts may encounter. Large-scale, standardized studies and technical considerations are required to overcome certain obstacles to the use of ML in IONM in the future.

1. Introduction

Intraoperative neurophysiological monitoring (IONM) is an essential diagnostic tool for the improvement of patient safety via detection of neurological changes during surgery. IONM is currently being applied in various types of surgery, such as open cranial surgery, spinal decompression, head and neck surgery, and peripheral nerve surgery [1,2,3]. The most prominent advantage of IONM is its use to confirm functional integrity in real time during surgery. When a warning sign occurs, an immediate rescue intervention can be performed in the operating room, minimizing neural injuries and enabling rapid postoperative treatment [4].

However, despite its advantages, some limitations exist in interpreting IONM. In particular, several factors complicate the real-time interpretation of multimodal signals during IONM. In interpreting neurophysiological changes related to surgical factors, a multidisciplinary approach between surgeons and physiatrists is essential, and substantial information sharing between them is vital for accurate interpretation [5]. Further, non-surgical factors, such as anesthesia, the general condition of the patient, and mechanical defects, have to be considered simultaneously with surgical factors [6]. Another hurdle in interpreting IONM is that experts may interpret the same results differently [7]. Therefore, the performance and interpretation of IONM require a substantial level of training to minimize inter-rater variability. Lastly, to respond sensitively to neural deterioration that occurs during surgery, the expert must maintain high concentration even during long operations. Consequently, IONM is a complicated, labor-intensive, and expensive process (Figure 1) [8].

The use of machine learning (ML) in medical research and clinical practice is rapidly expanding [9]. In particular, ML is increasingly being used for diagnosis and prognosis [10,11], as well as for disease classification, replacing existing methods [10]. Moreover, ML can execute proxy decision-making at the level of medical experts [12] and can readily and efficiently handle a large number of samples and variables [13]. Artificial intelligence (AI) models have the additional benefit of continuously improving themselves by learning from additional data and by applying more advanced techniques [14]. Although their performance depends on data quality and learning algorithms, in general, ML models have yielded promising results in clinical medicine [10,15].

In this narrative review, we focus on ML and its application in the field of IONM. We first summarize studies in which ML algorithms have been applied to IONM and then present a comprehensive review of ML algorithms. Furthermore, we discuss the limitations that should be considered in the application of ML to IONM. Finally, we conclude by pointing out the scope for future research to enable ML-based technologies to support human experts and cover the shortcomings of IONM.

2. Literature Review

2.1. Search Protocol

We searched articles from 2016 to 2022 with the following search terms: (“machine learning” OR “deep learning” OR “artificial intelligence”) AND (“intraoperative neuromonitoring” OR “intraoperative neurophysiological monitoring”). We searched the literature from the Cochrane Library, MEDLINE, and EMBASE and conducted a hand search of suitable manuscripts. We selected only original articles with human participants. In addition, we only included articles written in English. Finally, we excluded imaging or morphological research and anesthetic research (Figure 2).

2.2. Related Studies

Table 1 summarizes studies on the application of ML algorithms to IONM.

Jamaludin et al. [16] presented k-nearest neighbors (KNN)- and bagged trees-based ML models to predict positive outcomes after lumbar surgeries in 55 patients. The positive outcome was defined as a motor improvement after the surgery. They compared ML-based prediction methods with pre-existing criteria (50% of transcranial motor evoked potential improvement). In their work, the ML-based method showed a relatively higher sensitivity (87.5%) but lower specificity (33.3%), which was inferior to the pre-existing criteria for predicting postoperative motor improvement. Consequently, they suggested that their proposed method had more room to advance with a large-scale study.

Agaronnik et al. [17] developed a deep learning-based automated detection system for neuromonitoring documentation. They first identified operative reports containing neuromonitoring documentation by rule-based natural language processing. Subsequently, they applied a deep learning-based natural language processing model to identify events indicating a change in status, difficulty in establishing baseline signals, and a stable course, from the reports of 993 patients who underwent spinal surgery. For detection of a change in status, they achieved an area under the receiver operating characteristic curve (AUROC) of 1.0 and an F1 score of 0.80 (discussed further below). Their results suggest that deep learning-based natural language processing models can identify medical documentation of IONM from a large volume of reports in a validated and timely manner.

Kortus et al. [18] predicted electromyography (EMG) signal characteristics during thyroid surgery in 34 patients. They utilized a Bayesian convolutional neural network (CNN) approach to simultaneously classify action potentials and assess signal characteristics. The extended model with two hidden layers with sigmoid activation yielded the best predictive value, with an accuracy of 97.6%. By applying a Bayesian deep learning model, they estimated the uncertainty of the model output, which improved the interpretability of the prediction. They demonstrated that the deep learning model was suited for robust interpretation of electrophysiological signals.

Zha et al. [19] applied a deep learning model to free-running EMG for recurrent laryngeal nerve monitoring during thyroid surgery. They classified the EMG according to morphology and presented a hybrid model that combined a CNN approach with a long short-term memory (LSTM) network. Their proposed CNN-LSTM hybrid model yielded an accuracy of 89.54% and a sensitivity of 94.23%. Their results demonstrated the possibility of reducing inter-rater variability in the reading of free-running EMGs by using deep learning models, reducing the interpretive burden on the expert.

Verdonck et al. [20] presented a model for the interpretation of outliers via train-of-four (TOF) measurements during intraoperative acceleromyographic neuromuscular monitoring. They used a cost-sensitive logistic regression model to analyze 533 TOF measurements from 35 patients. In terms of the predictive power of this model, the AUROC was 0.91 (95% confidence interval: 0.72–0.97) and the F1 score was 0.86 (0.48–0.97). Their model proved outstanding for binary classification. Their study is important since it showed that the model could analyze TOF measurements to automatically identify outliers during intraoperative accelero-myographic neuromuscular monitoring.

Qiao et al. [21] conducted visual evoked potential (VEP) monitoring in 76 patients who underwent surgical decompression for sellar region tumors. They presented a model that could classify amplitude changes in VEPs during surgery, by combining CNN and recurrent neural network (RNN) algorithms. The target class was divided into three groups: increased VEP amplitude (>25% increase), decreased VEP amplitude (>25% decrease), and no change in VEP amplitude. In this study, the overall accuracy of multiclass classification was 87.4% (84.2–90.1%). The sensitivities for classification of no change in VEP, increasing VEP, and decreasing VEP were 92.6%, 78.9%, and 83.7%, respectively, and their specificities were 80.5%, 93.3%, and 100.0%, respectively.

Somatosensory evoked potential (SEP) is a modality that acts as the framework of intraoperative spinal surgery monitoring [4]. Fan et al. [22] utilized least squares and multi-support vector regression models on 15 patients undergoing spinal surgery to intraoperatively interpret the SEP results. They defined the warning criteria as an amplitude reduction of ≥50% or a latency delay of ≥10%. Target outcomes were classified as successful, false-positive, or trauma cases. Their intelligent decision system lowered the false warning rate compared with their conventional method and enabled more accurate detection of spinal cord trauma. The multi-support vector regression model performed better than the least squared support vector regression model.

Table 1. The application of machine learning in the field of intraoperative neurophysiological monitoring.

Author (Year)	Samples	IONM Modality	Models	Target Outcome	Summary of Results
Jamaludin et al. [16] (2022)	55 patients who underwent lumbar surgeries	MEP	KNN and Bagged trees	Positive outcome (motor improvement)	The proposed method was inferior to the existing criteria. - Sensitivity: 87.5% - Specificity: 33.3%
Agaronnik et al. [17] (2022)	993 patients who underwent spinal surgery	MEP and SEP	Deep learning-based natural language processing	Change in status	- AUROC: 1.00 - F1 score: 0.80
				Difficulty establishing baseline	- AUROC: 0.97 - F1 score: 0.80
				Stable course	- AUROC: 0.91 - F1 score: 0.93
Kortus et al. [18] (2021)	34 patients who underwent thyroid surgery	EMG	Bayesian CNN	Classification of action potentials	- Accuracy: 97.6% - Precision: 97.7% - Recall: 97.6%
Zha et al. [19] (2021)	5 patients who underwent thyroid surgery	Free-running EMG	Hybrid CNN-LSTM model	EMG signal waveforms (quiet, evoked, irritation, burst, injury, and artifact)	The hybrid model could automatically classify the free-running EMG. - Accuracy: 89.54% - Sensitivity: 94.23%
Verdonck et al. [20] (2021)	533 TOF samples from 35 patients	AMG	Cost-sensitive logistic regression	Outlier TOF measurement	AMG-based intraoperative measurements of TOF outliers displayed an increased monitoring consistency. - F1 score: 0.86 - AUROC: 0.91
Qiao et al. [21] (2019)	76 cases with sellar region tumor	VEP	CNN and RNN combination	Increasing, decreasing, or no change of VEP amplitude	- Overall accuracy of CNN/RNN combined vs. traditional method using single VEP images: 87.4% and 83.1%, respectively
Fan et al. [22] (2016)	10 successful surgeries (158 samples)	SEP	LS-SVR and M-SVR	Successful case: no interruption False positive case: surgery interrupted by an expert without spinal cord injury Trauma case: surgery interrupted by an expert, with spinal cord injury	- False positive rates NBM vs. LS-SVR vs. M-SVR: 0.304, 0.080, and 0.068, respectively. - True warning rate NBM vs. LS-SVR vs. M-SVR: 0.500, 0.714, and 0.714, respectively.
	4 false positives (72 samples)
	1 trauma case (14 samples)

IONM, intraoperative neurophysiological monitoring; MEP, motor evoked potential; KNN, k-nearest neighbors; SEP, somatosensory evoked potential; AUROC, area under the receiver operating characteristic curve; EMG, electromyography; CNN, convolutional neural network; LSTM, long short-term memory; TOF, train-of-four; AMG, acceleromyography; VEP, visual evoked potential; RNN, recurrent neural network; LS, least squares; M, multi; SVR, support vector regression; NBM, nominal baseline method.

Table 1. The application of machine learning in the field of intraoperative neurophysiological monitoring.

Author (Year)	Samples	IONM Modality	Models	Target Outcome	Summary of Results
Jamaludin et al. [16] (2022)	55 patients who underwent lumbar surgeries	MEP	KNN and Bagged trees	Positive outcome (motor improvement)	The proposed method was inferior to the existing criteria. - Sensitivity: 87.5% - Specificity: 33.3%
Agaronnik et al. [17] (2022)	993 patients who underwent spinal surgery	MEP and SEP	Deep learning-based natural language processing	Change in status	- AUROC: 1.00 - F1 score: 0.80
				Difficulty establishing baseline	- AUROC: 0.97 - F1 score: 0.80
				Stable course	- AUROC: 0.91 - F1 score: 0.93
Kortus et al. [18] (2021)	34 patients who underwent thyroid surgery	EMG	Bayesian CNN	Classification of action potentials	- Accuracy: 97.6% - Precision: 97.7% - Recall: 97.6%
Zha et al. [19] (2021)	5 patients who underwent thyroid surgery	Free-running EMG	Hybrid CNN-LSTM model	EMG signal waveforms (quiet, evoked, irritation, burst, injury, and artifact)	The hybrid model could automatically classify the free-running EMG. - Accuracy: 89.54% - Sensitivity: 94.23%
Verdonck et al. [20] (2021)	533 TOF samples from 35 patients	AMG	Cost-sensitive logistic regression	Outlier TOF measurement	AMG-based intraoperative measurements of TOF outliers displayed an increased monitoring consistency. - F1 score: 0.86 - AUROC: 0.91
Qiao et al. [21] (2019)	76 cases with sellar region tumor	VEP	CNN and RNN combination	Increasing, decreasing, or no change of VEP amplitude	- Overall accuracy of CNN/RNN combined vs. traditional method using single VEP images: 87.4% and 83.1%, respectively
Fan et al. [22] (2016)	10 successful surgeries (158 samples)	SEP	LS-SVR and M-SVR	Successful case: no interruption False positive case: surgery interrupted by an expert without spinal cord injury Trauma case: surgery interrupted by an expert, with spinal cord injury	- False positive rates NBM vs. LS-SVR vs. M-SVR: 0.304, 0.080, and 0.068, respectively. - True warning rate NBM vs. LS-SVR vs. M-SVR: 0.500, 0.714, and 0.714, respectively.
	4 false positives (72 samples)
	1 trauma case (14 samples)

IONM, intraoperative neurophysiological monitoring; MEP, motor evoked potential; KNN, k-nearest neighbors; SEP, somatosensory evoked potential; AUROC, area under the receiver operating characteristic curve; EMG, electromyography; CNN, convolutional neural network; LSTM, long short-term memory; TOF, train-of-four; AMG, acceleromyography; VEP, visual evoked potential; RNN, recurrent neural network; LS, least squares; M, multi; SVR, support vector regression; NBM, nominal baseline method.

3. Overview of Machine Learning

ML is a subfield of AI in which the knowledge to perform a target task is learnt from data. ML can be applied to various tasks, such as regression and classification [23]. An ML system consists of multiple steps as illustrated in Figure 3 (top), some of which can be omitted depending the type of the target task, the nature of the data, and the property of the ML model. In IONM, the ML system often takes (serial) evoked potentials and EMG signals as input. The preprocessing step transforms and normalizes the data to make it easier to process in subsequent steps. The feature extraction step converts the data into vector representation, from which the ML model produces the prediction result. In IONM, the ML model predicts discriminating altered signals, postoperative neurologic deficit, or postoperative functional gain. Optionally, the postprocessing step refines or reformats the prediction results.

There are various ML models. Linear models are widely used for numerical data analysis owing to their simplicity and interpretability. However, more sophisticated methods, such as neural networks, support vector machines (SVMs), and decision trees (DTs), are also frequently used in medical data analysis [24]. In particular, tree-based ensemble techniques such as random forests and extreme gradient boosting (XG-Boost) have exhibited excellent performances in numerical data analysis. In recent years, deep learning has attracted considerable attention owing to its outstanding performance [25]. Deep learning is based on neural networks and is particularly effective in analyzing complex data, such as images, text, and time-series data [26,27].

The training algorithms adjust the parameters or structure of the model to optimize a learning objective as shown in Figure 3 (bottom). Widely used learning criteria include loss minimization and likelihood maximization. In supervised learning, the model learns from the ground truth labels specified by human experts. The training algorithm minimizes loss, such as cross-entropy or the mean squared error, that reflects the difference between the model output and the ground truth [28]. Further, unsupervised learning is used to learn data distribution or specific tasks, such as clustering and reproduction, without labels [29]. In recent years, self-supervised learning has been widely used for learning feature representations from a large volume of unlabeled data [30]. In self-supervised learning, the model achieves knowledge for multiple tasks through artificially defined tasks whose ground truth can be derived from the data itself, e.g., predicting the next data from the past data and filling in the masked part of the data. Reinforcement learning trains a model that interacts with the environment. The model then performs actions that affect the environment, and the environment rewards the model’s actions. The training algorithm optimizes the model to select the actions that maximize the expected reward.

Figure 3. A general framework for training and applying ML models. The bottom figure illustrates supervised learning using ground truth labels.

Figure 3. A general framework for training and applying ML models. The bottom figure illustrates supervised learning using ground truth labels.

The validation and performance evaluation of ML models is crucial in their application to the field of medicine [28]. For evaluation, the primary metric for classification tasks is accuracy, which is the fraction of correctly classified samples among all test samples. Precision and recall are widely used for detection and identification tasks. Precision is the fraction of retrieved instances that were relevant, while recall is the fraction of relevant instances that were retrieved. The F1-score, defined as the harmonic mean of precision and recall, is used to measure the balanced performance of the model [31].

In general, researchers divide data into three non-overlapping subsets with different purposes: training, validation, and testing. The training set is used only to train the model, and the validation set is used to determine hyperparameters and select a model. The performance of the selected ML model is assessed on the testing set. One potential problem in evaluation is that bias in the selection of the testing set can harm the reliability of the evaluation results [32]. This can be minimized by applying techniques such as k-fold cross-validation. The k-fold cross-validation procedure is as follows [33]:

• Randomly split the dataset into $k$ groups (e.g., for $k=5$, the groups are ${\mathrm{S}}_1, {\mathrm{S}}_2, {\mathrm{S}}_3, {\mathrm{S}}_4,$ and ${\mathrm{S}}_5$).
• Repeat the training and testing $k$ times. At the $k$-th iteration, use ${\mathrm{S}}_{\mathrm{k}}$ for testing and all other groups for training and hyperparameter determination.
• Average the $k$ evaluation results.

However, despite the rapid development of ML, its adoption in medicine has been slower than that in other areas [34]. In clinical practice, the decision-making process of a human expert is very sophisticated and complex. A model can only learn such a process by being trained on a large number of high-quality samples [9]. The collection and labeling of large-scale medical datasets are labor-intensive and expensive. In addition, medical research involves the use of human data, which is accompanied by ethical considerations. Another hurdle in the application of ML to the medical field is the hesitation of many human experts to accept the results of black-box models [35]. Moreover, deep learning-based image segmentation techniques are essential for the analysis of advanced medical images, such as those generated with computed tomography or magnetic resonance imaging [36]. Therefore, deep learning models can only be applied to medical research if the infrastructure is in place for the accurate and efficient processing of large quantities of image segmentation results [37,38].

4. Representative Machine Learning Models for IONM-Related Research

Previous works on IONM have applied various types of ML models. In this section, we briefly introduce the representative ML models that can be useful to predict or analyze IONM data, including those listed in Table 1.

4.1. Neural Networks

Neural networks have been the major ML models applied to the field of IONM. For example, these analyze neuromonitoring documents by a natural language processing model implemented with a Transformer. CNN and RNN have been used to classify action potentials and EMG signal waveforms, respectively. Here, we described the overview of neural networks.

4.1.1. Artificial Neural Networks

An artificial neural network (ANN) is an ML algorithm imitating the human brain [39]. Neurons in the human brain combine signals (stimuli) from multiple upstream neurons; when the combined stimulus exceeds the threshold, the neuron relays the resulting signals to the downstream neurons. ANNs were designed according to this principle. A neural network consists of an input layer that receives data from multiple inputs, an output layer that produces the output results, and one or more hidden layers between them [24]. Each layer comprises multiple neurons and contains learning parameters, such as connection weights ($w_{ij}$) and biases ($b_j$), where i and j are the indices for the input and output neurons, respectively. Given an input vector,

$$x=\left(x_1,x_2,\dots ,x_n\right)$$

each layer computes the output,

$$y_j=\left(y_1,y_2,\dots ,y_m\right)$$

via a linear combination followed by a nonlinearity; for instance, the output may be calculated as follows:

$$y_j=f({\displaystyle \sum }_i{w}_{ij}{x}_i+b_j)$$

where $f\left(·\right)$ denotes a nonlinear activation function. The behavior of a neural network depends on the values of the connection weights and biases. The learning algorithm determines the optimal values of these parameters for the target task by minimizing the loss function on the training data.

The neural network learns to map an input into the desired output. An ANN composed of multiple layers with nonlinear activation functions can approximate complex maps [27]. However, the training of a large-scale neural network requires a large amount of training data and computational power [40]. In particular, when trained with a limited number of samples, ANNs often suffer from overfitting and are not generalizable to different input data.

4.1.2. Convolutional Neural Networks

Composed of large numbers of layers, deep neural networks can perform high levels of abstraction and are, therefore, excellent in processing complex data such as images, text, and sounds. CNNs are the most popular deep learning architectures for image processing [41]. CNNs have heterogeneous structures that combine various types of layers, such as those for convolution, pooling, normalization, and self-attention, as well as dense layers, skip connections, and dropouts [36]. Most CNN layers are composed of multi-channel feature maps. The convoluted layer learns the position-invariant local features to capture the 2D local patterns of the input image.

4.1.3. Recurrent Neural Networks

RNNs are a class of neural networks that are specialized in the processing of iterative and sequential data. Unlike feedforward networks, RNNs have feedback connections to deliver information from the past to the future [42]. RNNs have been widely used in natural language and speech processing. In medicine, RNNs are frequently used for the analysis of medical data that require continuous signal reading, such as EMG or fluoroscopic images [19,43]. An encoder RNN and a decoder RNN can be combined to learn mappings between sequential data of different lengths [9].

Simple RNNs have the limitation that they cannot learn long-term dependencies from time-series data. LSTM networks [44] and gated recurrent units (GRUs) [45] are advanced RNN architectures that overcome this limitation by applying the gating mechanism. However, all RNNs, including LSTM networks and GRUs, have the drawback that they are hard to parallelize and, therefore, slower than other architectures.

4.1.4. Transformers

Recently, transformers have exhibited higher performance than RNNs in natural language and speech processing [46]. A transformer network consists of a stack of transformer blocks, each of which combines a self-attention sub-layer and a multi-layer perceptron sub-layer. Self-attention is easy to parallelize and refers to a wider context than recurrent connections in RNNs do. RNNs and transformers can be used to process medical textual data (e.g., extracting a specific word from a text-based medical record) [47].

4.1.5. Bayesian Neural Networks

Most neural networks provide prediction results as fixed values without an indication of uncertainty. Without uncertainty modeling, it is challenging to determine the appropriate level of confidence in the output. This obstacle causes reluctance among medical experts to accept the results of ML models.

Bayesian neural networks provide the posterior distribution of the output value rather than a single deterministic value [48]. In general, the prediction and its uncertainty are provided as the mean and variance of the output distribution, respectively.

Bayesian neural networks model one or both of two types of uncertainties: aleatoric uncertainty, which captures variation inherent in the data, and epistemic uncertainty, which accounts for uncertainty in the model [49]. Epistemic uncertainty is modeled by estimating the posterior distribution of the model parameters, whereas aleatoric uncertainty is modeled by a neural network that estimates the density of the output quantity. Bayesian neural networks can be implemented with various types of neural networks including CNN, RNN, and transformers.

4.2. Support Vector Machines

SVMs have higher generalizability than other traditional ML models when the training data are limited. When the training samples are represented in vector space, SVMs find the boundary with the maximum margin from the nearest samples in each class [50]. Linear SVMs find linear boundaries, whereas nonlinear SVMs combine a linear SVM and a nonlinear transformation using the “kernel trick” to classify complex nonlinear patterns [51]. SVMs are binary classifiers. However, it is possible to discriminate between multiple classes by combining multiple SVMs in one-to-one or one-to-the-rest approaches.

4.3. Regularized Logistic Regression

Logistic regression is a classification algorithm that extends linear regression by combining it with a logistic function. This method is widely used in medical statistics. However, if the model deals with many input variables with an insufficient number of training samples, logistic regression poses the risk of overfitting. When overfitting, the model yields a low degree of training loss but a substantial degree of validation loss. Overfitting is a common issue with most ML models that generally increases with the capacity of the model [52]. Overfitting can be reduced with the application of regularization techniques. The loss function of regularized logistic regression was designed to reduce not only the prediction error but also the norm of the weight vector. L1 and L2 regularization reduce the L1 and L2 norms of the weight vector, respectively [10,53]. L2 regularization is generally more effective in reducing overfitting, whereas L1 regularization is used more frequently for feature selection. L1 regularization is also known as lasso regularization, and L2 regularization as ridge regularization. Elastic net is a hybrid model that applies both L1 and L2 regularizations [54].

4.4. Random Forests

Ensemble learning is a promising technique to improve the accuracy of ML by combining the results of multiple classification or regression models [10,55]. There are two primary strategies to combine ML models: bagging and boosting. In bagging, the training data are randomly split into overlapping subsets, and each model is trained on one of them. The results of the models are combined by averaging or voting. Random forests are ensemble models that combine decision trees by using the bagging strategy. Decision trees are vulnerable to overfitting [28], which is effectively reduced with bagging [56]. In addition, random forests retain the advantage of decision trees in that they do not require data normalization and are relatively easy to interpret.

4.5. Extreme Gradient Boosting

Boosting, or bootstrapping, is another ensemble strategy that builds multiple sequential models, one at a time. In each step, the model learns by focusing on the samples that were misclassified by the previous models. XG-Boost is one of the most advanced boosting algorithms, yielding excellent performance with tabular data [55]. It is an extension of the gradient boosting machine (GBM) with improved scalability [57]. GBMs build decision trees via function approximation. In each step, the GBM finds a new decision tree that reduces the bias of the previous trees via a gradient-based method derived from second-order Taylor approximations. GBMs excel in many tasks but are slow and not scalable [58]. XG-Boost uses multiple techniques to improve the efficiency of the GBM in terms of computational and memory requirements. Consequently, it is 10 times faster than conventional GBMs and scalable to billions of examples in distributed or memory-limited environments. In addition, its accuracy can be improved and overfitting reduced by adjustment of the hyperparameters according to the data and environment, such as computing infra and memory size [59].

4.6. Hyperparameters for Each ML Model

Each ML model has a set of hyperparameters to specify the model size, the types of the components, the strength of regularization, and other properties. The performance of the ML model is significantly affected by the hyperparameters [60,61].

Table 2. Important hyperparameters of machine learning models.

ML Models	Major Hyperparameters
ANN	The number of layers The number of units in each layer The type of activation functions (e.g., ReLU, sigmoid, tanh, softmax, ELU, swish, mish) Training hyperparameters (e.g., batch size, learning rate, maximum number of iterations, early stopping criteria, and the algorithms for initialization and optimization)
CNN	All of the hyperparameters of artificial neural networks The type of each layer (e.g., conv., max-pooling, batch-norm, dropout, group conv.) The width, height, and channel of each layer Kernel size, stride, padding Existence of skip connection
RNN (including LTSM and GRU)	All of the hyperparameters of artificial neural networks Unidirectional or bidirectional The size of cell blocks (LSTM)
Transformers	The number of layers The total dimension of hidden features The number of heads in the multi-head attention The dimension of key and value The dimension of MLP layers All of the training hyperparameters of artificial neural networks
SVM	The weight of soft margin The type of kernel (e.g., polynomial, Gaussian, RBF) and its parameters (e.g., γ of RBF kernel) Training hyperparameters (e.g., learning rate, number of iterations)
Regularized logistic regression	Regularization factors Training hyperparameters (e.g., learning rate, number of iterations)
Random forests	The number of trees Maximum depth of the tree Maximum number of leaf nodes Quality measure of a split (e.g., Gini, entropy, log_loss) Regularization factor
XG-Boost	The number of trees Maximum depth of the tree Type of booster and its parameters (e.g., learning rate, gamma, max delta step) Regularization factors

ANN, artificial neural network; ReLU, rectified linear unit; ELU, exponential linear unit; CNN, convolutional neural network; RNN, recurrent neural networks; LSTM, long short-term memory; GRU, gated recurrent unit; MLP, multi-layer perceptron; SVM, support vector machines; RBF, radial basis function; XG, extreme gradient.

lists the major hyperparameters of ML models. Some hyperparameters in Table 2 specify training options, such as the batch size, learning rate, maximum number of training iterations, early stopping criteria, and algorithms for initialization and optimization, which are important for training ML models.

5. Limitations

Nonsurgical factors are important confounders in the interpretation of IONM. In particular, IONM modalities are very sensitive to changes brought about by anesthesia-related factors [62]. The anesthetic methodology used, the use of neuromuscular blockade, the patient’s blood pressure and body temperature, and prolonged operation time, among other factors, can substantially affect IONM signals, even in the absence of a neural insult [63]. Cross-disciplinary collaboration is essential in the construction of a model that can consider these various factors simultaneously. In addition, since many variables need to be processed, it is essential that models are trained and validated on high-quality datasets with large numbers of samples that share the same features.

Another point to consider is that false positives results in ambiguity in the interpretation of IONM signals [64]. When a warning signal occurs during surgery, regardless of its reliability (true or false positive), surgeons and anesthesiologists respond by initiating a rescue intervention process [6]. This may be an important confounding factor in the predictive value of ML algorithms. If postoperative neurological deficit is defined as a dependent variable during the construction of an ML model, there may be disagreements in the input to provide to an ML algorithm when a warning signal is issued. Therefore, as demonstrated by Zha et al. [19], morphological classification may be a more realistic alternative to the direct interpretation of evoked potential. In other words, although the ML algorithm reads the signals, the human expert’s intervention remains essential in determining the reliability and cause of a warning sign.

However, inter-rater variability in the interpretation of IONM signals is inevitable when human experts are involved [65]. For example, results will be interpreted differently depending on the definition of the baseline. The presence or absence of a warning sign depends on whether the baseline is static or changes in response to previous waveforms during the surgery [66]. Studies also vary in their definitions of postoperative neurological deficits [67]. This difference in the interpretation of the ground truth can cause high variability between ML algorithms. The interpretation of IONM results may also vary depending on the degree of training of the expert [8].

6. Future Perspectives

To build an ML-based IONM model that can be applied to the clinical field in the future, the following issues should be considered.

IONM is a diagnostic tool that uses multiple modalities, and the interpretational method differs slightly among modalities. To date, we are aware of only limited studies on the use of ML in the interpretation of MEP and SEP; such a model may play a key role in central nervous system (CNS) surgeries. Therefore, additional validation studies of the ML models for each modality should be conducted. Furthermore, several studies have demonstrated that, in the interpretation of IONM signals, predictive power can be increased by considering multiple modalities simultaneously rather than single modalities [68,69]. Therefore, there is a need for complex models that can utilize various modalities simultaneously.

Moreover, models should be tailored for the prediction of optimal outcomes according to each surgical method. For example, evoked-potential warning signs in open cranial surgery differ slightly from those that occur during spinal cord decompression surgery [70]. In addition, warning signs for peripheral nerve surgery are completely different from those that occur in CNS surgery [71].

We believe that ML-based IONM interpretational models will be useful in the prediction of patients’ outcomes by assessing their intraoperative evoked potential. This topic has been actively studied in the field of cervical decompression surgery [72]. Furthermore, such studies are being conducted for open cranial and peripheral nerve surgeries [71]. If ML models can perform precise sequential interpretation, they will be able to interpret a patient’s neural integrity during surgery to predict subsequent clinical recovery.

7. Conclusions

IONM is a valuable tool to improve patient safety and minimize neurological damage during surgery. However, its interpretation is relatively complicated, expensive, and labor-intensive. Therefore, it requires extensive training for human experts. The efficiency and reliability of IONM may be enhanced with the use of ML models. However, standardized, large-scale data collection and technical considerations are required to overcome the limitations to such use of ML models and to provide practical support for experts. Further, since IONM comprises many modalities and is used in different types of surgery, ML models should be tailored accordingly for optimal performance. Similarly, ML models that use multiple modalities should be established. Much research is required to achieve these goals. Through these efforts, ML-based IONM interpretation can become a valuable and applicable technology and can ultimately further guarantee patient safety.