Machine Learning-Based Pain Severity Classification of Lumbosacral Radiculopathy Using Infrared Thermal Imaging

Abstract

Pain is subjective and varies among individuals. Doctors determine pain severity based on a patient’s self-reported symptoms. In such situations, a language barrier may prevent patients from expressing their pain accurately, which may cause doctors to underestimate their pain degree. Moreover, patients’ subjective descriptions of pain can determine their eligibility for secondary benefits, as in the case of compensation for traffic or industrial accidents. Therefore, to perform a multiclass prediction of the severity of lumbar radiculopathy, the authors applied digital infrared thermographic imaging (DITI) to a machine-learning (ML) algorithm. The DITI dataset included data from a healthy population and patients with radiculopathy with herniated lumbar discs at the L3/4, L4/5, and L5/S1 levels. The dataset of 1000 patients was split into training and test datasets in a 7:3 ratio to evaluate the model’s performance. For the training dataset, the average accuracy, precision, recall, and F1 score were 0.82, 0.76, 0.72, and 0.74, respectively. For the test dataset, these values were 0.77, 0.71, 0.75, and 0.73, respectively. Applying the ML algorithm to a pain-severity classification using thermographic images will aid in the treatment of lumbosacral radiculopathy and allow providers to monitor the therapeutic effect of interventions through an assessment of physiological evidence.

1. Introduction

Diagnostic tests for lumbar myopathy include physical examination, magnetic resonance imaging (MRI), and electromyography. However, objectively measuring the degree of radiating pain is difficult in routine physical examinations [1], and MRI, computed tomography (CT), or electromyography alone may be insufficient diagnostic tools [2].

According to previous studies, MRI is a good diagnostic tool for lumbar stenosis but is insufficient to objectively predict the degree of radiating pain [3]. Electromyography is a valuable method for diagnosing the presence of radiating pain [4]. However, it is an invasive procedure, as needles are inserted into the patient’s body, causing pain during the examination.

Digital infrared thermographic imaging (DITI) can noninvasively demonstrate functional changes in body temperature at a low cost and has a short examination time. Hypothermia or hyperthermia is induced at the lesion site and can be objectively observed [5]. DITI can visualize changes in the skin temperature, especially in patients with radiating pain [6], thus helping clinicians determine a treatment plan for herniated discs [7]. Additionally, if DITI characteristics change according to the severity of radiating pain, an objective judgment index for subjective pain can be provided.

Research on artificial intelligence has been conducted to improve accuracy or support diagnosis and treatment in various medical fields (radiology [8], ultrasound [9], and pathology [10]). Therefore, the machine-learning (ML) method used in previous studies was applied to DITI data derived from patients with lumbosacral radiculopathy. As the most frequently affected lumbar discs are L3/4, L4/5, and L5/S1 [11,12], patients with these radiculopathies were included in our study. We aimed to classify the two classes of patient-reported pain severity for modulating severe and intractable pain first, followed by moderate pain.

To the best of our knowledge, this study is a significant contribution to the field of pain management and diagnosis. The application of ML to classify pain severity using DITI data is a novel approach that has the potential to improve pain assessment accuracy and optimize treatment plans. This study’s focus on lumbosacral radiculopathy is also noteworthy, as this is a common and often debilitating condition that can significantly impact a patient’s quality of life. One of the main advantages of using ML in pain assessment and diagnosis is that it can enable local medical personnel, who may not be spine specialists, to develop treatment plans with conservative interventions. This can be particularly beneficial in areas where access to specialized medical care is limited, and patients may have to rely on general practitioners or non-specialists for pain management. ML can provide a reliable and objective tool for these medical professionals to accurately assess pain severity and develop effective treatment plans. In addition to its potential clinical applications, we also applied an ML model calibration for general applications. Model calibration is a critical step in the development of ML models, as it ensures that the model’s predictions are accurate and reliable. This is particularly important in the medical field, where inaccurate predictions can have serious consequences for patient health and well-being. This study provides valuable insights into the potential applications of thermography and ML in pain assessment and management.

2. Materials and Methods

2.1. Dataset of Patients with Lumbosacral Radiculopathy and Ethical Approval

A trained spinal neurosurgeon reviewed the data of 1728 patients who underwent a one-level discectomy at the Yonsei University Hospital, including Gangnam Severance Hospital and National Health Insurance Corporation Ilsan Hospital, from 2006 to 2021. Altogether, 1000 patients were included in this retrospective study. The research ethics committee of the Gangnam Severance Hospital and National Health Insurance Corporation Ilsan Hospital reviewed and approved the study protocol (Gangnam Severance Hospital Institutional Review Board approval number: 2021-0551-002). The dataset (National Reference Standard Korean Thermal Data Center at the National Health Insurance Service Ilsan Hospital and Gangnam Severance Hospital) of the patients who underwent a one-level discectomy and the National Reference Standard Korean Thermal Data Center Research Platform were used to develop and externally validate an ML-based pain severity classification algorithm.

2.2. Inclusion and Exclusion Criteria

The inclusion criteria were as follows: (1) patients who were diagnosed with a lumbar disease, including lumbar disc herniation, at the National Health Insurance Service Ilsan Hospital from 2 March 2000 to 30 June 2021 and from whom thermographic images of the lower extremity were acquired; (2) those with at least one of the following preoperative radiologic criteria: diagnosed with herniated lumbar disc (HLD) using CT or MRI; and (3) those whose diagnoses were verified by two radiologists. Patients with developmental stages of lumbar radiculopathy ranging from an acute to chronic status were included. The patients were divided into two groups according to their degree of pain on the visual analog scale (VAS): 1–7 points were classified as mild-to-moderate pain, and 8–10 points indicated severe-to-intractable pain, with 10 points indicating the highest severity.

Meanwhile, the exclusion criteria were as follows: (1) patients with diseases that can affect the skin temperature of the lower extremities, such as diabetes, peripheral arterial occlusive disease, and other endocrine diseases; (2) those who had undergone previous cervical or lumbosacral surgery; and (3) those who did not report pain in the past.

2.3. Clinical Data Collection, Labeling, Preprocessing, ML-Based Classification Modeling, and Calibration Algorithm

Our study center retrospectively collected clinical data, including information on diagnosis, using DITI. Clinical characteristics, such as age, sex, height, weight, duration of pain, location of pain, maximum angle of pain on the straight leg elevation test, and degree of disc herniation, were used as variables. Because variants of convolutional neural networks have been applied to DITI data in previous studies [13], we instead extracted the data using the region of interest on the National Reference Standard Korean Thermal Data Center Research Platform. DITI included lower extremity sequences, such as the anterior and posterior leg and feet. In the DITI images, 10 region-of-interest values were extracted and used for the left and right areas of the anterior thigh, shin, posterior thigh, calf, and pelma (undersurface of the foot). The authors split the DITI dataset into the training algorithm and test dataset in a 7:3 ratio to evaluate the model’s performance. The algorithm for classifying pain severity was evaluated, and the isotonic calibration algorithm was applied using Python 3.6.

Among the models that can be implemented with python, we compared logistic regression (LR), which is widely used conventionally; decision tree (DT), which is a representative classical tree algorithm; random forest (RF), which is a representative bagging method; and support vector machine (SVM), which has recently been used in general applications [14]. In the RF method proposed by Breiman, each tree is a standard classification or regression tree that uses the so-called Gini impurity decrement as the splitting criterion and selects the splitting predictor from a randomly selected subset of predictors. Each tree consists of a bootstrap sample extracted with a replacement from the original dataset, and predictions from all trees are finally aggregated by majority vote [15].

The definition of the model evaluation parameters is shown in

Table 1. The definition of the machine-learning model evaluation parameters. PPV, positive predictive value; NPV, negative predictive value.

		Actual Values
		Positive	Negative
Predicted Values	Positive	a (True Positive)	b (False Positive)	PPV = a/(a + b)
	Negative	c (False Negative)	d (True Negative	NPV = d/(c + d)
		Sensitivity = a/(a + c)	Specificity = d/(b + d)

. Sensitivity is true positive (TP)/(TP+ false negative [FN]), specificity is true negative (TN)/(TN + false positive [FP]), positive predicted value (precision) is TP/(TP + FP), recall is TP/(TP + FN), and negative predictive value is TN/(TN + FN). The balanced accuracy (BA), the arithmetic mean of sensitivity and specificity, is 1/2 × (sensitivity + specificity). The F1 score, a weighted average of precision and recall, is 2 × precision × recall/(precision + recall). These parameters have advantages when dealing with imbalanced data and performance evaluation [16].

In terms of calibration, we also assessed the Brier score (BS). The BS measures the overall correction and is defined as the mean squared difference between the predicted probability and actual outcome. Values were between 0 and 1, with lower values indicating a better calibration. The calculation is described below.

$$BS=\frac{1}{N}{\displaystyle \sum }_{t=1}^N\left(f_t-o_t\right){ }^2$$

(1)

N = the number of items for which a Brier score is being calculated;

(2)

f_t is the forecast probability (i.e., 25% chance);

(3)

o_t is the outcome (1 if it happened, 0 if it did not);

(4)

∑ is the summation symbol, indicating that all of the values must be added together.

Regarding the computer environment for ML, the central processing unit was AMD Ryzen 7 PRO 4750G, the graphics processing unit was NVIDIA’s GeForce RTX 3090 24 GB, and 64 GB of RAM were used for this study.

3. Results

3.1. Demographic Characteristics

The patient data were collected according to age, degree of pain, and duration of pain (

Table 2. Distribution of patients according to severity of pain, age, and duration of symptoms.

	Mild-to-Moderate Pain		Severe-to-Intractable Pain
Age Group	<6 Months	>6 Months	<6 Months	>6 Months
0–19	1	1	1	2
20–29	10	17	20	13
30–39	26	43	30	46
40–49	20	36	43	65
50–59	45	74	40	61
60–69	30	76	60	70
70	40	49	35	46

). Of the 1000 patients, 468 patients had mild-to-moderate pain, and 532 had severe-to-intractable pain. The number of women and men were 601 and 499, respectively.

Patients aged ≥50 years formed the majority of the participants, and the predominant age range in the population was between 60 and 69 years. In all groups, more patients with persistent pain for >6 months were observed.

Several patients aged >70 years had undergone surgery, and the mild-to-moderate pain group had more patients than the severe-to-intractable pain group. Thus, not only the severity of the pain, but also the neurological examination findings, such as muscle weakness, should be considered for spinal surgery (Table 2).

3.2. ML Model Comparison

The classification accuracy of pain severity was calculated using body temperature image data obtained using the ML analysis method, as follows: BA, F1 score, sensitivity, particularity, positive predictive value, and negative predictive value, sequentially. The respective obtained values were as follows: LR: 0.636, 0.682, 0.748, 0.524, 0.627, and 0.661; DT: 0.599, 0.620, 0.632, 0.566, 0.609, and 0.590; RF: 0.664, 0.699, 0.748, 0.579, 0.655, and 0.683; and SVM: 0.650, 0.616, 0.548, 0.752, 0.702, and 0.609 (Figure 1 and

Table 3. Model evaluation parameters according to the machine-learning algorithm.

Parameter	LR	DT	RF	SVM
BA	0.636	0.599	0.664	0.650
F1 score	0.682	0.620	0.699	0.616
Sensitivity	0.748	0.632	0.748	0.548
Specificity	0.524	0.566	0.579	0.752
PPV	0.627	0.609	0.655	0.702
NPV	0.661	0.590	0.683	0.609

LR, logistic regression; DT, decision tree; RF, random forest; SVM, support vector machine; BA, balanced accuracy; PPV, positive predictive value; NPV, negative predictive value.

).

Figure 2 presents the receiver operating characteristic (ROC) and precision-recall curves of each ML model. Each ML model had area-under-the-ROC-curve (AUC) values of: LR, 0.67; DT, 0.58; RF, 0.70; and SVM, 0.62. The respective AUC values for the precision-recall curves were: LR, 0.68; DT, 0.70; RF, 0.71; and SVM, 0.74.

3.3. Calibration of the RF Model

The calibration curves of the RF model are plotted in Figure 3. After isotonic calibration, the performance of the RF classifier showed a better performance in each predicted probability bin. The BS indicated no calibration (0.225) and isotonic calibration (0.211).

Calibration curves compare how the probabilistic predictions of the RF binary classifier are calibrated. The true frequency of the positive label was plotted against its predicted probability for binned predictions. The x-axis represents the average predicted probability in each bin. The y-axis is the fraction of positives.

3.4. Case Discussion

Two women who visited our hospital were presented as case studies (Figure 4). Both patients were diagnosed with HLD L4/5 on opposite sides. Both patients had lower-extremity pain, which was rated on the VAS. Infrared thermal images were taken, objectively showing that the temperature change appeared along the dermatome of the diagnosed HLD-related level. The temperature difference was larger in the case of severe pain, showing that thermal imaging can visualize both the location of a lesion and severity of pain.

Case 1. A 60-year-old woman complained of decreased sensation in the right calf, and her pain at the time of the visit was a 5 on the VAS. The symptom duration was approximately 2 months. She had taken medications and physical therapy, but there was no change in the pain. MRI showed a ruptured lumbar disc at the L4/5 level on the right side with upward migration (Figure 4A,B). On infrared thermal images, color difference was checked in the right L4/5 dermatome region. The maximum temperature difference between the left and right lower limbs was −1.03 °C. A microscopic discectomy was performed for L4/5 intervertebral disc herniation, which was observed on examination, for the treatment of L5 nerve radiating pain.

Case 2. A 51-year-old woman complained of muscle weakness in her left ankle, and her pain at the time of visit was a 9 on the VAS. The symptom duration was approximately 3 months. She had taken medications and physical therapy, but the pain increased in severity. MRI showed a ruptured lumbar disc at the L4/5 level on the left side (Figure 4D,E). On infrared thermal images, color difference was checked in the left L4/5 dermatome region. The maximum temperature difference between the left and right lower extremities was −2.93 °F. A microscopic discectomy was performed for lumbar 4/5 intervertebral disc herniation, observed on examination, for the treatment of lumbar five nerve radiating pain.

After the discectomy surgery, both patients’ pain improved, and both were discharged without any complications. Unlike the rapid improvement of pain symptoms, numbness decreased over 3 and 6 months after discharge, and the thermography showed a decrease in the temperature difference between the lower extremities of both sides.

4. Discussion

Little academic progress has been made in the analysis of acute and chronic pain using infrared thermography. In particular, identifying the exact location of a neuromuscular lesion through thermographic imaging without radiological information from MRI or CT may have different results depending on the skill of the physician; hence, an ML method can be helpful. Therefore, previous studies by the authors have reported a method for differentially diagnosing lumbosacral radiculopathy of HLD L4/5 and L5/S1 as one of the most frequently herniated discs, using feature extraction techniques with a bag-of-words feature extraction algorithm [17].

The location of a patient’s pain can be easily diagnosed using an infrared thermal image to compare the temperature difference between the left and right sides and by identifying the area with the most significant temperature change [18]. A previous study reported that the temperature difference in the lower limbs decreased when pain improved postoperatively, and several studies have reported that the temperature changes along with pain at the location of the lesion [19]. According to these prior studies, the greater the degree of pain, the greater the temperature difference in the thermographic image.

Nevertheless, this study was the first to classify the severity of pain by integrating the patient’s sex, age, duration of pain, and nonlinear features using ML. The temperature difference occurs because the greater the pressure on the nerve root following the herniated disc, the higher the intensity of pain in the patient. This low-temperature phenomenon intensifies. Moreover, the sympathetic nerve activity of the autonomic reflex arc may change according to the change in the amount of information transmitted to the retrodural nerve. The case comparison reported here demonstrates an evident thermal difference depending on the severity of pain in acute lumbosacral radiculopathy (Figure 4).

Zhang et al. reported a correlation in which the temperature difference between the left and right sides increased as disc herniation became more severe [11]. As the degree of pain decreased, the temperature difference also decreased. In some cases, pain persists, although the temperature difference decreases; cases where there are pain and temperature increases in the case of trauma have also been reported [12]. The degree of pain has a robust subjective tendency that varies depending on the individual’s experience, environment, secondary gain, and education level. Because the pain and local low-temperature phenomenon may fluctuate, classifying and evaluating the severity of pain using only infrared body temperature images taken at a specific time may be difficult for nonskilled medical staff. Therefore, if the ML method performed in this study is applied by integrating and applying other clinical characteristics as variables through a detailed history, including the period of pain, the temperature distribution characteristics observed in the infrared body temperature image and nonlinear analysis can be applied, which can be clinically helpful in determining the severity of pain. The RF model demonstrated the best performance for the F1 score, whereas the SVM model demonstrated the best performance for specificity (Table 2 and Figure 2). To improve accuracy, the developed ML models need to be calibrated, with, for example, an isotonic calibration for generality (Figure 3).

The authors evaluated why RF’s results outperformed several other algorithms. Perhaps, since it has a bootstrap ensemble method, RF is advantageous for small perturbations in the data. Moreover, due to the implicit feature selection mechanism, the representation becomes resilient to noninformative or noisy features. Additionally, the RF algorithm derives task-specific similarity concepts for target information in contrast to classical hierarchical clustering approaches.

The current study has several advantages. First, it is the first to classify the severity of pain using infrared thermography and ML, providing a non-invasive and objective method for pain evaluation. Second, the study used nonlinear feature extraction techniques to capture the complex relationship between the patient’s characteristics and pain severity. Third, the study integrated several clinical characteristics, including sex, age, duration of pain, and nonlinear features, to increase the accuracy of pain severity classification.

However, there are also several limitations to the current study. First, the sample size was relatively small, which may limit the generalizability of the findings. The small number of patients was insufficient to apply ML. Future studies involving more patients are warranted. To ensure generalization performance, data from two other hospitals were used, and verification was performed by applying the calibration method.

Second, the study did not consider clinical outcomes with a model net benefit. Although the model’s accuracy was evaluated using several indicators, such as an F1 score, balanced accuracy, sensitivity, and specificity, and we also performed model calibration to overcome the overfitting issue, it was insufficient to verify the analysis of the net benefit of the clinical outcomes of the proposed model. The modeling of variables confined to linear regression analysis may not be suitable for examining the benefits of nonlinear analysis and models by observing the net benefit with or without the specific variable. However, in recent years, mathematical methods and prior studies have been presented to overcome these shortcomings [20,21,22], and the authors will have to conduct additional analyses in future studies considering the net benefit of the model to evaluate its clinical usefulness.

Third, the study used patient-reported pain as a label, which is subjective and may vary between individuals. The variation between individuals’ subjective reports was minimized through the use of two classifications: severe-to-intractable pain and mild-to-moderate pain. Future studies should consider objective measures of pain severity, such as functional MRI, to improve the accuracy of pain classification.

The proposed method may have potential for application to other types of pain, such as cancer pain or chronic headaches, but future studies are needed to investigate its generalizability across different pain types. Additionally, future research should investigate the predictive power of the proposed method to improve pain management and treatment outcomes. Another potential use of the proposed method is in evaluating the efficacy of pain interventions, such as medication, physical therapy, or surgery. Future studies should investigate the use of the proposed method to monitor the effectiveness of pain treatments and optimize treatment plans. Furthermore, the proposed method may be integrated with other pain-assessment tools, such as self-report questionnaires, to improve pain-assessment accuracy. Future studies should investigate the use of the proposed method in combination with other pain-assessment tools to improve the accuracy and reliability of pain assessment. Finally, the proposed method may have potential for use in real-time pain-monitoring systems, such as hand-held thermography devices with mobile apps. Future studies should investigate the feasibility and effectiveness of implementing the proposed method in real-time pain-monitoring systems to provide patients with continuous pain-monitoring and self-management tools. Overall, the proposed method shows promise for improving pain assessment and management, but further research is needed to fully explore its potential applications and limitations.

5. Conclusions

We explored the possibility of classifying the degree of pain using thermographic images and ML algorithms. The RF algorithm showed good results in BA, sensitivity, F1 score, and negative predicted value. Through model calibration, we attempted to overcome the overfitting in limited data. This study will be clinically helpful in objectively determining the severe pain caused by lumbosacral radiculopathy.