Next Article in Journal
Possible Association of Cholesterol as a Biomarker in Suicide Behavior
Next Article in Special Issue
Sex Differences in Spironolactone and the Active Metabolite Canrenone Concentrations and Adherence
Previous Article in Journal
Endothelial Heme Dynamics Drive Cancer Cell Metabolism by Shaping the Tumor Microenvironment
Previous Article in Special Issue
Capillary Electrophoresis–Mass Spectrometry with Multisegment Injection and In-Capillary Preconcentration for High-Throughput and Sensitive Determination of Therapeutic Decapeptide Triptorelin in Pharmaceutical and Biological Matrices
 
 
Search for Articles:
Title / Keyword
Author / Affiliation / Email
Journal
Article Type
 
 
Advanced Search
 
Section
Special Issue
Volume
Issue
Number
Page
 
Journals
Biomedicines
Volume 9
Issue 11
10.3390/biomedicines9111558
biomedicines-logo
Submit to this Journal Review for this Journal Propose a Special Issue
► Article Menu

Article Menu

share Share announcement Help format_quote Cite question_answer Discuss in SciProfiles
first_page
settings
Order Article Reprints
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

Predicting Serum Levels of Lithium-Treated Patients: A Supervised Machine Learning Approach

by
Chih-Wei Hsu
1,2,*,
Shang-Ying Tsai
3,4,
Liang-Jen Wang
5,
Chih-Sung Liang
6,7,
Andre F. Carvalho
8,
Marco Solmi
9,10,11,
Eduard Vieta
12,
Pao-Yen Lin
1,13,
Chien-An Hu
1 and
Hung-Yu Kao
2,*
1
Department of Psychiatry, Kaohsiung Chang Gung Memorial Hospital and Chang Gung University College of Medicine, Kaohsiung 83301, Taiwan
2
Department of Computer Science and Information Engineering, National Cheng Kung University, Tainan 70101, Taiwan
3
Department of Psychiatry, School of Medicine, College of Medicine, Taipei Medical University, Taipei 110301, Taiwan
4
Department of Psychiatry and Psychiatric Research Center, Taipei Medical University Hospital, Taipei Medical University, Taipei 110301, Taiwan
5
Department of Child and Adolescent Psychiatry, Kaohsiung Chang Gung Memorial Hospital, Chang Gung University College of Medicine, Kaohsiung 83301, Taiwan
6
National Defense Medical Center, Department of Psychiatry, Beitou Branch, Tri-Service General Hospital, Taipei 112003, Taiwan
7
National Defense Medical Center, Department of Psychiatry, Taipei 114201, Taiwan
8
IMPACT (Innovation in Mental and Physical Health and Clinical Treatment) Strategic Research Centre, School of Medicine, Barwon Health, Deakin University, Geelong, VIC 3216, Australia
9
Psychiatry Department, University of Ottawa, Ottawa, ON K1N 6N5, Canada
10
The Ottawa Hospital, University of Ottawa, Ottawa, ON K1H 8L6, Canada
11
Clinical Epidemiology Program, Ottawa Hospital Research Institute (OHRI), University of Ottawa, Ottawa, ON K1N 6N5, Canada
12
Bipolar and Depressive Disorders Unit, Hospital Clinic, IDIBAPS, CIBERSAM, University of Barcelona, 08036 Barcelona, Catalonia, Spain
13
Institute for Translational Research in Biomedical Sciences, Kaohsiung Chang Gung Memorial Hospital, Kaohsiung 83301, Taiwan
 
add Show full affiliation list
 
remove Hide full affiliation list
*
Authors to whom correspondence should be addressed.
Biomedicines 2021, 9(11), 1558; https://doi.org/10.3390/biomedicines9111558
Submission received: 27 September 2021 / Revised: 23 October 2021 / Accepted: 25 October 2021 / Published: 28 October 2021
(This article belongs to the Special Issue Pharmacokinetics and Therapeutic Drug Monitoring)
Browse Figures
Versions Notes

Abstract

:
Routine monitoring of lithium levels is common clinical practice. This is because the lithium prediction strategies available developed by previous studies are still limited due to insufficient prediction performance. Thus, we used machine learning approaches to predict lithium concentration in a large real-world dataset. Real-world data from multicenter electronic medical records were used in different machine learning algorithms to predict: (1) whether the serum level was 0.6–1.2 mmol/L or 0.0–0.6 mmol/L (binary prediction), and (2) its concentration value (continuous prediction). We developed models from 1505 samples through 5-fold cross-validation and used 204 independent samples to test their performance by evaluating their accuracy. Moreover, we ranked the most important clinical features in different models and reconstructed three reduced models with fewer clinical features. For binary and continuous predictions, the average accuracy of these models was 0.70–0.73 and 0.68–0.75, respectively. Seven features were listed as important features related to serum lithium levels of 0.6–1.2 mmol/L or higher lithium concentration, namely older age, lower systolic blood pressure, higher daily and last doses of lithium prescription, concomitant psychotropic drugs with valproic acid and -pine drugs, and comorbid substance-related disorders. After reducing the features in the three new predictive models, the binary or continuous models still had an average accuracy of 0.67–0.74. Machine learning processes complex clinical data and provides a potential tool for predicting lithium concentration. This may help in clinical decision-making and reduce the frequency of serum level monitoring.

1. Introduction

Lithium is an important therapeutic drug and is considered an archetypal mood stabilizer for the management of mood disorders or schizoaffective disorder [1,2,3]. Many international guidelines recommend lithium as a first-line drug, especially for bipolar disorder [4,5]. Even though lithium has clear beneficial effects, the therapeutic range of its serum levels is narrow (<1.2 mmol/L) [6]. Due to the narrow therapeutic index of lithium, a routine monitoring of its serum levels is suggested. However, the need for frequent monitoring may limit the clinical prescription of lithium. For example, in the acute phase of treatment, clinicians first prescribed an initial low-dose lithium dosage regimen that was titrated upwards based on the serum level and clinical response. This approach usually takes days to weeks to reach the optimal serum concentration, which may delay the therapeutic response [7]. Moreover, in the maintenance phase of lithium therapy, the need for frequent blood draws to obtain serum levels may reduce patients’ willingness to use lithium [3,8,9].
In the past few decades, some studies have attempted to solve this issue and provided formulas to calculate the expected steady-state lithium levels for a specific dose [10]. However, these studies have great limitations that hinder their clinical application in real-world settings. For example, the sample size of these previous studies is often small (usually <200 participants) [11,12,13]; while others have included patients in a conditional experimental environment, rather than patients from a real-world clinical environment [10]; prediction errors of these models were not enough to meet clinical needs (i.e., root-mean-square error (RMSE) ≥0.37 mmol/L) [14]. Recently, algorithm-driven machine learning models have been developed as important tools in mental health field [15,16,17]. These algorithms may provide programs that optimize performance under the guidance of training experience, for example, using gender-specific gene expression biomarkers to predict lithium treatment response [18]. Furthermore, with the increasing popularity of big data, such as data from electronic medical records (EMRs), large-scale datasets provide machine learning for training and have the potential to make better prediction models of serum lithium levels more likely [19]. After independent replication, these models may become clinically useful in routine psychiatric care.
This study used a large-scale blood sample derived from EMRs and different algorithmic machine learning methods to develop predictive models of patients’ serum lithium levels. This study aimed to predict serum lithium levels at a therapeutic level of 0.6–1.2 mmol/L (binary prediction) and blood concentration value (continuous prediction). For further clinical applications, the importance of the features in these models was ranked, and new reduced models with fewer features were reconstructed.

2. Materials and Methods

2.1. Data Source and Study Subjects

The research protocol was approved by the institutional review board of Chang Gung Memorial Hospital on 2 February 2021 (No.202100131B0). Figure 1 depicts a flowchart of the selection process and the study design. We used data from the Chang Gung Research Database (CGRD) medical claims between 1 January 2002, and 31 December 2019 to predict serum lithium levels of inpatients with mental disorders. The CGRD is a multicentric EMR, which includes deidentified personal data on demographics (age, sex), medical visits (outpatient and inpatient), pharmacy records (medication type, dosage, frequency, and duration of supply), disease diagnosis by the International Classification of Disease, Ninth Revision (ICD-9) or ICD-10, and laboratory data (hematology tests, biochemical tests, and blood draw time) from seven medical institutes throughout Taiwan [20]. The CGRD covered 14% of inpatients with mental disorders in Taiwan’s total medical population from 1997 to 2010 [21]. Since inpatients routinely take medications under the supervision of nurses (good medication compliance), we only included inpatient pharmacy and laboratory data to develop machine learning algorithms. Patients’ lithium concentration records had to meet the following inclusion criteria: (1) Patients should take the same daily dose for at least 5-day consecutively before the blood test was performed to achieve the steady-state concentration of lithium therapy based on its half-life [22]; (2) Had no chronic kidney disease (i.e., glomerular filtration rate ≥60 mL/min/1.73 m2) [23]; (3) Data on serum lithium level ranging from 0.0–1.2 mmol/L [4]; (4) Serum lithium samples were drawn 8–16 h after the last dose [11,24,25]. Additionally, the same set of eligibility criteria to extract additional data on serum lithium samples from outpatients was adopted.

2.2. Definition of Outcome Targets, Predictive Features, and Analysis Subject

In this study, two outcome targets for binary and continuous variables were defined. Since serum levels of lithium of 0.6–1.2 mmol/L have been considered the target range for mood disorders [26] and covered most of the recommended treatment guidelines for the manic and maintenance phases of bipolar disorder [4,5,27], binary variable results of the prediction model are divided into 0.6–1.2 mmol/L group (proper treatment) and 0.0–0.6 mmol/L group (undertreatment). Additionally, the serum lithium level is directly regarded as a target for continuous outcomes.
Based on the results of previous studies on lithium interaction and the available data in the database [10,28], we included 114 features for analysis, including basic information (age, sex, height, weight, blood pressure, characteristics of lithium prescription), concomitant medications, comorbidities, and laboratory data. The patient’s comorbidity was defined as whether the disease was recorded in the CGRD within 2 years before admission. Concomitant medication was considered if the patient had used other medications within 5-day before the lithium blood test. For different concomitant psychotropic medications, we calculated the ratio of the average daily dose to the defined daily dose (DDD) used for the modeling. The DDD determined by the World Health Organization Collaborating Centers for Drug Statistics Methodology was used to assume the average maintenance dose per day in adults [29]. Laboratory data were extracted at the same time as the lithium blood sample was collected or at other times within a week. Due to the lack of laboratory data for some patients, we kept patients with at least the other characteristics mentioned above. We then adopted k-nearest neighbor (k-NN) imputation to fill in missing values (laboratory data) [30]. We computed the median of the given variable in the five nearest neighbors of a given subject to fill these values. Detailed information on all features and availability rates of laboratory data are provided in Supplementary Tables S1 and S2.
To obtain prediction models with high generalizability, the inpatient data were divided into an internal development (training/validation) set and an external test set (inpatient test set). First, we randomly selected 10% of the entire data as the test set for external validation [15] and then implemented 1:9 propensity score matching (PSM) on the remaining data to obtain the development set [31]. Development set was used to develop the machine learning model and internal validation [15]. Additionally, we were interested in the applicability of machine learning algorithms to outpatients, so the outpatient data were used for the other external test set (outpatient test set). Finally, the development, inpatient test, and outpatient test sets included 1505, 204, and 7 samples, respectively.

2.3. Machine Learning Models and Model Evaluation

Five common machine learning algorithms were used in this study (logistic regression (LogR), linear regression (LinR), support vector machine (SVM) with radial basis function kernel, random forests (RF), and extreme gradient boosting (XGBoost) [18,32,33,34,35]. These algorithms were used to develop predictive models of binary outcomes (LogR, SVM, RF, and XGBoost) and continuous outcomes (LinR, SVM, RF, and XGBoost). The LogR model uses probabilities for classification problems with two classes of dichotomous criteria, and the LinR model predicts the continuous value as a weighted sum of the feature inputs. SVM constructs a set of hyperplanes in a higher-dimensional space to achieve the maximum separation distance of the nearest training data points of any class [36]. RF operates by constructing multiple decision trees during training and outputting a comprehensive prediction based on the mean prediction of individual trees (110 trees with a depth of 7 layers were used in the analysis) [37]. XGBoost builds an ensemble of decision trees by iteratively focusing on harder to predict subsets of the training data (200 trees with a depth of five layers were used in the analysis) [38]. We first apply the regularization method to all features to maintain a common scale range (0–1), so that we can avoid distortion of the value range and overfitting and improve the prediction accuracy of the model. Then, we used the 5-fold cross-validation method to develop the above machine learning models and evaluate their performance through two test sets.
Different performance parameters were calculated for binary and continuous outcomes. We mainly used sensitivity, specificity, the area under the curve of receiver operator characteristic (AUC-ROC), and accuracy for binary prediction and mean absolute error (MAE), mean square error (MSE), RMSE, and accuracy for continuous prediction. For better clinical practice, we defined the accuracy of continuous results (i.e., if the difference between the predicted value and true value is within 0.2 mmol/L, the predicted result will be regarded as a correct prediction). See Supplementary Tables S4 and S5 for all performance parameters. In addition, we conducted Y randomization (Y scrambling) test on the continuous result prediction model to ensure its robustness [39,40]. The lithium concentration value was randomly shuffled once, and a new prediction model was developed using the original features. If the primary predictive model is acceptable, the new predictive model is expected to have a lower R2 value than the primary model.

2.4. Model Interpretation and Statistical Analysis

We used three methods to rank the importance of features and interpret the model predictions. First, LogR and LinR adopt the least absolute shrinkage and selection operator (LASSO) algorithm. LASSO is a regression analysis that uses the L1 constraint to perform variable selection [41]. Second, we used a backward stepwise method to determine the importance order among the features. This process uses a series of steps to allow features to leave the SVM model one at a time, which allows for interactions between residual features [42]. Third, RF and XGBoost chose Shapley additive explanation (SHAP) to interpret the model predictions. SHAP comes from game theory, and its interpretation is based on the SHAP value of each feature, which represents the contribution of the feature to predicting the event risk. A positive or negative SHAP value indicates that the corresponding feature has an increase or decrease in the lithium concentration, respectively. The SHAP value of each variable is additive, allowing the contribution of each variable to be converted into a part of the output classification probability [43].
We selected three different feature combinations to reconstruct the prediction model to evaluate the clinical applicability of the model with fewer variables. First, we only applied basic information in prediction Model 1. Second, we combined basic information and concomitant psychotropic medications as predictive features in Model 2. Finally, we used the top 10 ensemble features (close to 10% of 114 features) in Model 3, selected from the feature importance ranking results of the different machine learning algorithms mentioned above (consensus features obtained by combining the results of different models).
The independent t-test and χ2 test were used to compare the baseline characteristics of the development and test data and the difference between the primary model and the secondary analysis in different feature combinations. We also used a one-way analysis of variance to check the accuracy differences between the four prediction models. All statistical analyses were performed using SAS software (v. 9.4; SAS Institute Inc., Cary, NC, USA). Statistical significance was set at p-value < 0.05. All machine learning models were established with Windows Python 3.8 (scikit-learn package v. 0.24.2), and the codes were provided on the GitHub website (github.com/harwic/LithML01) accessed on 18 October 2021.

3. Results

3.1. Characteristics of Study Participants

A total of 1709 inpatient data were included in this study (mean age, 43.1 years, 45.8% female). Table 1 presents further clinical characteristics after PSM. There was no significant difference between development and inpatient test data. The demographic data of the seven outpatient data are listed in Supplementary Table S3.

3.2. Predictive Model Performance

Table 2 first shows the binary outcomes (0.6–1.2 mmol/L vs. 0.0–0.6 mmol/L) of the four algorithms. For inpatient test data, these models had high sensitivity (LogR, 0.89; SVM, 0.94; RF, 0.96; XGBoost, 0.90), low specificity (LogR, 0.43; SVM, 0.32; RF, 0.22; XGBoost, 0.38), with an average AUC-ROC exceeding 0.75 (LogR, 0.75; SVM, 0.76; RF, 0.78; XGBoost, 0.78). The average accuracy of the four algorithms was 0.70–0.73 with no significant differences between them (F = 2.36, p = 0.110, data not shown). When the four models were applied to outpatient test data, their model performances were similar to the inpatient test data, such as average accuracy (inpatient: 0.70–0.73; outpatient: 0.77–1.00). The detailed model performance of the binary results is listed in Supplementary Table S4. Table 2 shows the continuous results of the four algorithms. For the inpatient test data, the average MAE, MSE, and RMSE of the four algorithms was 0.14–0.16 mmol/L, 0.03–0.04 mmol/L, and 0.17–0.20 mmol/L, respectively. The average accuracy was 0.68–0.75, and the SVM had the highest accuracy (F = 15.52, p < 0.001, data not shown). When the four models were used to predict outpatient test data, the average accuracy was 0.67–0.78, and no differences were observed between the four models (F = 1.33, p = 0.299, data not shown). Supplementary Table S5 shows the detailed model performance of continuous results. Additionally, the R2 of the new models had lower values than the primary models after Y randomization test (new models: 0.117–0.148, primary models: 0.209–0.370, data not shown).

3.3. Feature Importance and Model Performance under Different Feature Combinations

Table 3 summarizes the top 10 features of the different algorithms. Among the top 10 ensemble features of the binary and continuous algorithm models, seven features appeared together and were associated with higher lithium concentrations: older age, lower systolic blood pressure, higher daily and last doses of lithium prescription, concomitant psychotropic medications with valproic acid and -pines drugs, and comorbid substance-related disorders. Of these seven features, the daily dose of lithium prescription and age were the first two important features in predicting serum lithium levels. For detailed information on feature selection, such as LASSO, backward stepwise, and SHAP, as well as the importance ranking and positive-negative relationship of all features of different algorithm models, see Supplementary Materials.
Supplementary Figure S9A shows the accuracy of binary outcomes using four algorithms with different feature combinations. The accuracy could be above 0.70 (range from 0.70 to 0.73) regardless of the algorithm or feature combination used. The predictive performance of the three new models was similar to that of the primary model, and no statistical differences were observed. Moreover, Supplementary Figure S9B shows the accuracy of continuous outcomes (0.67–0.75). For the LinR and SVM algorithms, the accuracy of the primary model using all features was significantly higher than that of Model 1/Model 2 and Model 1/Model 3, respectively. However, the models with fewer features performed nearly as well as the primary model in the RF and XGBoost algorithms. Additionally, detailed performance data for various feature combinations are shown in Supplementary Materials. In addition, Figure 2 summarizes the step-by-step decisions recommended for clinicians using the binary or continuous predictive model of this study.

4. Discussion

This study used machine-learning algorithm-driven models to predict serum lithium levels. We collected a large number of blood samples from hospitalized patients to develop machine learning models and used independent inpatient and outpatient data to make predictions. Whether it is binary prediction or continuous prediction, the accuracy was 67–100% (binary, 70–100%; continuous: 67–78%). Moreover, we analyzed the feature rankings of these models and found seven important features. Furthermore, we reduced the number of features required to reconstruct the prediction model. The accuracy of most of the new models is close to that of the primary model, with no significant difference.
Several studies have developed predictive lithium dose equations [11,12,13,25,44,45]. Compared to previous studies, this study with large-scale blood sample data in real-world settings is more complicated. For example, participants in this study took various concomitant drugs, such as mood stabilizers, antidepressants, and antipsychotics, which may interact with the pharmacokinetics of lithium (Table 1) [28]. Under such an arduous task, this study has better sensitivity (previous studies, 0.80–0.90; inpatient data result, 0.89–0.96) [14] or RMSE (previous studies, 0.21–0.59 mmol/L; inpatient data result, 0.17–0.20 mmol/L) [14,46], but lower specificity (previous studies, 0.67–0.76; inpatient data result, 0.22–0.43) [14], or higher MAE (previous studies, 0.13 mmol/L; inpatient data result, 0.14–0.16 mmol/L) [25]. Furthermore, these primary models of the inpatient data performed equal or better in most of the outpatient data (Supplementary Tables S4 and S5), indicating that our model has a certain generalization. Notably, the SVM algorithm of the continuous model exhibited the best performance in the inpatient test data among the four algorithms. Clinicians may consider using the SVM algorithm first to predict an inpatient’s lithium concentration.
Compared with previous studies, the predictive performance of the current machine learning algorithm-driven model should be acceptable and useful for real-world clinical practice (Figure 2). Here, we assume that two clinical scenarios are recommended for these predictive tools. First, the binary prediction model can help clinicians track whether lithium treatment reaches the therapeutic concentration in the outpatient setting and reduces the frequency of blood draw. Many studies have indicated that patients receiving lithium treatment should monitor their plasma concentration regularly throughout their life [47,48], but this is a burden on patients. The proportion of patients who receive regular blood lithium monitoring is often lower than the recommended standard [8,9,49]. Therefore, we can use patient information to predict whether the level of lithium treatment is sufficient. If the prediction is sufficient, we may not draw blood (higher sensitivity); otherwise, we draw blood to recheck the lithium concentration (lower specificity). Second, the continuous predictive model can help clinicians adjust the medications of their patients. The current model of this study can estimate the concentration value and increase or decrease the daily dose of lithium to reach the appropriate therapeutic range [6].
The interpretable machine learning model is an important issue; hence, this study summarizes the top 10 features of different algorithms with binary and continuous outcomes. First, it is easy to understand that increasing daily and last doses of lithium elevate its serum level. Second, our study found that the older the age, the higher the lithium concentration. There is evidence that older people have lower lithium clearance, which may explain the results of this study [50,51]. Third, the simultaneous use of valproic acid or -pines drugs is also an important feature in predicting serum lithium levels. Previous clinical trials have shown that valproic acid or quetiapine may interact with lithium and slightly increase serum concentration [28,52,53]. Some studies also indicated that lithium plus valproic acid or quetiapine has a better therapeutic response than lithium monotherapy [54,55]. Compared to lithium monotherapy, lithium plus valproic acid or -pine drugs can enhance therapeutic effects by increasing lithium concentration, partially supporting our findings. In addition, some case reports have found that the combined use of lithium and antipsychotics can cause neurotoxicity, but serum lithium levels may not increase at the same time [56,57,58]. The feature importance ranking of our model can also partially explain it, that is, different types of antipsychotics have different effects on lithium concentration. For example, the most influential drugs in our models are -pine drugs (Supplementary Tables S6 and S7). Fourth, a higher systolic blood pressure reduces the lithium concentration. Increased blood pressure is related to the excessive activity of sodium-lithium countertransport in red blood cells, leading to a decrease in serum lithium levels [59,60]. Finally, a previous study reported that drinking alcohol increases lithium concentration [61], which may reflect the importance of substance use disorders in predictive models. In summary, factors affecting lithium concentration proved by previous studies may play an important predictive role in the machine learning model of real-world data. Furthermore, it is worth noting that true indicators of kidney function, such as blood urea nitrogen or serum creatinine, are not a priority feature to predict lithium concentration in this study [10]. This may be due to the inclusion criteria (glomerular filtration rate ≥60 mL/min/1.73 m2), indicating that the renal function of the study participants was relatively normal.
For further clinical applications, we attempted to reduce the features of the algorithm. Even though the new model and the primary model had differences in the accuracy performance of the different algorithms, the overall accuracy of all new models was approximately 70%. This finding inspired us to consider using basic information with or without concomitant psychotropic drugs to predict serum lithium levels of the patient. Furthermore, using only the top 10 features in the predictive model may also be another option. Our findings show that this flexible feature combination of algorithm-driven machine learning models is clinically more practical. For example, clinicians can only use 10 predictors, such as basic information or the top 10 features mentioned in this study, to build a simple model with predictive performance similar to the original model that used 114 predictors.
This study had several limitations. First, we used PSM to select independent inpatient data to reduce the bias of background characteristics between development and test data [16], but collecting development and test data from the same dataset may reduce the generalizability of our models. Second, we extracted outpatient data as another test set to verify reproducibility [16]. Compared to the inpatient test set, the model performance was still acceptable; however, the outpatient test set only included seven blood samples, which may reduce validity. Third, we extracted the patient’s laboratory data with a 1-week buffer period and used the k-NN method to fill in the missing values. What we do may obscure the true value of individual laboratory data and reduce its usefulness in algorithms. For example, renal function is not a priority feature in the current model compared to a previous study [10]. Finally, this study excluded some extreme or outlier data from the database, such as patients with chronic kidney disease, serum lithium levels exceeding 1.2 mmol/L, or blood samples collected less than 8 h or more than 16 h after the last dose; the domain of applicability of our models cannot be extended to those with these conditions [39,62].

5. Conclusions

We used real-world EMR data to develop machine learning models to predict serum lithium levels. The average accuracy of binary results or continuous results was 68–75%. Older age, lower systolic blood pressure, higher daily and last doses of lithium prescription, concomitant psychotropic medications with valproic acid and -pines drugs, and comorbid substance-related disorders were important features associated with higher lithium concentrations. We altered the prediction models with fewer features, and the average accuracy was still close to 70%. Our model processed more complex clinical data and provided useful clinical tools for predicting serum lithium levels.

Supplementary Materials

The following are available online at https://www.mdpi.com/article/10.3390/biomedicines9111558/s1. Figure S1: LASSO method for selecting the top 10 features in logistic regression, binary outcomes. Figure S2: Backward stepwise method for selecting the top 10 features in support vector machine, binary outcomes. Figure S3: Shapley additive explanations method for selecting the top 10 features in random forests, binary outcomes. Figure S4: Shapley additive explanations method for selecting the top 10 features in extreme gradient boost, binary outcomes. Figure S5: LASSO method for selecting the top 10 features in linear regression, continuous outcomes. Figure S6: Backward stepwise method for selecting the top 10 features in support vector machine, continuous outcomes. Figure S7: Shapley additive explanations method for selecting the top 10 features in random forests, continuous outcomes. Figure S8: Shapley additive explanations method for selecting the top 10 features in extreme gradient boost, continuous outcomes. Table S1: Detailed drug name with codes and diagnostic codes for mental disorders or medical diseases. Table S2: Proportion of laboratory data with no missing test values in inpatient data. Table S3: Characteristics of lithium-treated patients, comparing inpatient test data and outpatient test data. Table S4: Detail information of binary outcomes in inpatient test data and outpatient test data. Table S5: Detail information of continuous outcomes in inpatient test data and outpatient test data. Table S6: 114 feature importance ranking results of the 4 different machine learning algorithms in binary outcomes. Table S7: 114 feature importance ranking results of the 4 different machine learning algorithms in continuous outcomes. Table S8: Detail information of binary outcomes of logistic regression algorithm under different feature combinations. Table S9: Detail information of binary outcomes of support vector machine algorithm under different feature combinations. Table S10: Detail information of binary outcomes of random forests algorithm under different feature combinations. Table S11: Detail information of binary outcomes of extreme gradient boosting algorithm under different feature combinations. Table S12: Detail information of continuous outcomes of linear regression algorithm under different feature combinations. Table S13: Detail information of continuous outcomes of support vector machine algorithm under different feature combinations. Table S14: Detail information of continuous outcomes of random forests algorithm under different feature combinations. Table S15: Detail information of continuous outcomes of extreme gradient boosting algorithm under different feature combinations.

Author Contributions

Research idea and study design, C.-W.H.; data acquisition, C.-W.H. and C.-A.H.; data interpretation, C.-W.H., A.F.C., L.-J.W., M.S., and H.-Y.K.; statistical analysis, C.-W.H.; manuscript drafting, C.-W.H.; manuscript revision, C.-W.H., S.-Y.T., L.-J.W., C.-S.L., A.F.C., M.S., E.V., P.-Y.L., and H.-Y.K. Each author contributed important intellectual content during manuscript drafting or revision and accepts accountability for the overall work by ensuring that questions pertaining to the accuracy or integrity of any portion of the work are appropriately investigated and resolved. All authors have read and agreed to the published version of the manuscript.

Funding

This research was funded by grants from the Ministry of Science and Technology, Taiwan (MOST 109-2314-B-182A-009-MY2), and the funding sources had no role in the design of the study.

Institutional Review Board Statement

The study was conducted according to the guidelines of the Declaration of Helsinki and approved by the Institutional Review Board of Chang Gung Memorial Hospital on 2 February 2021 (No.202100131B0).

Data Availability Statement

The data that support the findings of this study are not publicly available but can be accessed with permission from the Chang Gung Memorial Hospital in Taiwan.

Acknowledgments

The authors would like to thank Pei-Ying Yang for the technical support.

Conflicts of Interest

Eduard Vieta has received grants and served as a consultant, advisor or CME speaker for the following entities: AB-Biotics, Abbott, Allergan, Angelini, AstraZeneca, Bristol-Myers Squibb, Dainippon Sumitomo Pharma, Farmindustria, Ferrer, Forest Research Institute, Gedeon Richter, Glaxo-Smith-Kline, Janssen, Lundbeck, Otsuka, Pfizer, Roche, SAGE, Sanofi-Aventis, Servier, Shire, Sunovion, Takeda, the Brain and Behaviour Foundation, the Spanish Ministry of Science and Innovation (CIBERSAM), the EU Horizon 2020 and the Stanley Medical Research Institute. Other authors declare no financial interests or potential conflicts of interest regarding the authorship and publication of this article.

References

  1. Carvalho, A.F.; Firth, J.; Vieta, E. Bipolar Disorder. N. Engl. J. Med. 2020, 383, 58–66. [Google Scholar] [CrossRef]
  2. Leucht, S.; Helfer, B.; Dold, M.; Kissling, W.; McGrath, J.J. Lithium for schizophrenia. Cochrane Database Syst. Rev. 2015, 2015, Cd003834. [Google Scholar] [CrossRef] [PubMed]
  3. Carvalho, A.F.; Solmi, M.; Husain, M.I.; Berk, M.; Vieta, E. The rebirth of lithium as the archetypal mood stabilizer. Bipolar Disord. 2021, 23, 211–212. [Google Scholar] [CrossRef]
  4. Yatham, L.N.; Kennedy, S.H.; Parikh, S.V.; Schaffer, A.; Bond, D.J.; Frey, B.N.; Sharma, V.; Goldstein, B.I.; Rej, S.; Beaulieu, S.; et al. Canadian Network for Mood and Anxiety Treatments (CANMAT) and International Society for Bipolar Disorders (ISBD) 2018 guidelines for the management of patients with bipolar disorder. Bipolar Disord. 2018, 20, 97–170. [Google Scholar] [CrossRef] [PubMed]
  5. Fountoulakis, K.N.; Grunze, H.; Vieta, E.; Young, A.; Yatham, L.; Blier, P.; Kasper, S.; Moeller, H.J. The International College of Neuro-Psychopharmacology (CINP) Treatment Guidelines for Bipolar Disorder in Adults (CINP-BD-2017), Part 3: The Clinical Guidelines. Int. J. Neuropsychopharmacol. 2017, 20, 180–195. [Google Scholar] [CrossRef] [PubMed]
  6. Hsu, C.W.; Carvalho, A.F.; Tsai, S.Y.; Wang, L.J.; Tseng, P.T.; Lin, P.Y.; Tu, Y.K.; Vieta, E.; Solmi, M.; Hung, C.F.; et al. Lithium concentration and recurrence risk during maintenance treatment of bipolar disorder: Multicenter cohort and meta-analysis. Acta Psychiatr. Scand. 2021, 144, 368–378. [Google Scholar] [CrossRef]
  7. Dugas, J.E. Slowly titrated lithium doses delay therapeutic response. Clin. Pharm. 1984, 3, 236–238. [Google Scholar]
  8. Collins, N.; Barnes, T.R.; Shingleton-Smith, A.; Gerrett, D.; Paton, C. Standards of lithium monitoring in mental health Ttrusts in the UK. BMC Psychiatry 2010, 10, 80. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  9. Nederlof, M.; Egberts, T.C.; van Londen, L.; de Rotte, M.C.; Souverein, P.C.; Herings, R.M.; Heerdink, E.R. Compliance with the guidelines for laboratory monitoring of patients treated with lithium: A retrospective follow-up study among ambulatory patients in the Netherlands. Bipolar Disord. 2019, 21, 419–427. [Google Scholar] [CrossRef]
  10. Sienaert, P.; Geeraerts, I.; Wyckaert, S. How to initiate lithium therapy: A systematic review of dose estimation and level prediction methods. J. Affect. Disord. 2013, 146, 15–33. [Google Scholar] [CrossRef]
  11. Abou-Auda, H.S.; Al-Yamani, M.J.; Abou-Shaaban, R.R.; Khoshhal, S.I. A new accurate method for predicting lithium clearance and daily dosage requirements in adult psychiatric patients. Bipolar Disord. 2008, 10, 369–376. [Google Scholar] [CrossRef] [PubMed]
  12. Terao, T.; Okuno, K.; Okuno, T.; Nakano, H.; Shinkai, T.; Ohmori, O.; Yoshimura, R.; Suzuki, T.; Abe, K. A simpler and more accurate equation to predict daily lithium dose. J. Clin. Psychopharmacol. 1999, 19, 336–340. [Google Scholar] [CrossRef]
  13. Jermain, D.M.; Crismon, M.L.; Martin, E.S., 3rd. Population pharmacokinetics of lithium. Clin. Pharm. 1991, 10, 376–381. [Google Scholar]
  14. Radhakrishnan, R.; Kanigere, M.; Menon, J.; Calvin, S.; Srinivasan, K. Comparison of three a-priori models in the prediction of serum lithium concentration. Indian J. Pharmacol. 2012, 44, 234–237. [Google Scholar] [CrossRef] [PubMed]
  15. Chekroud, A.M.; Bondar, J.; Delgadillo, J.; Doherty, G.; Wasil, A.; Fokkema, M.; Cohen, Z.; Belgrave, D.; DeRubeis, R.; Iniesta, R.; et al. The promise of machine learning in predicting treatment outcomes in psychiatry. World Psychiatry 2021, 20, 154–170. [Google Scholar] [CrossRef] [PubMed]
  16. Radua, J.; Carvalho, A.F. Route map for machine learning in psychiatry: Absence of bias, reproducibility, and utility. Eur. Neuropsychopharmacol. 2021, 50, 115–117. [Google Scholar] [CrossRef]
  17. Lin, E.; Lin, C.-H.; Lane, H.-Y. Precision Psychiatry Applications with Pharmacogenomics: Artificial Intelligence and Machine Learning Approaches. Int. J. Mol. Sci. 2020, 21, 969. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  18. Eugene, A.R.; Masiak, J.; Eugene, B. Predicting lithium treatment response in bipolar patients using gender-specific gene expression biomarkers and machine learning. F1000Research 2018, 7, 474. [Google Scholar] [CrossRef] [PubMed]
  19. Manchia, M.; Vieta, E.; Smeland, O.B.; Altimus, C.; Bechdolf, A.; Bellivier, F.; Bergink, V.; Fagiolini, A.; Geddes, J.R.; Hajek, T.; et al. Translating big data to better treatment in bipolar disorder—A manifesto for coordinated action. Eur. Neuropsychopharmacol. 2020, 36, 121–136. [Google Scholar] [CrossRef]
  20. Shao, S.C.; Chan, Y.Y.; Kao Yang, Y.H.; Lin, S.J.; Hung, M.J.; Chien, R.N.; Lai, C.C.; Lai, E.C. The Chang Gung Research Database-A multi-institutional electronic medical records database for real-world epidemiological studies in Taiwan. Pharmacoepidemiol. Drug Saf. 2019, 28, 593–600. [Google Scholar] [CrossRef]
  21. Tsai, M.S.; Lin, M.H.; Lee, C.P.; Yang, Y.H.; Chen, W.C.; Chang, G.H.; Tsai, Y.T.; Chen, P.C.; Tsai, Y.H. Chang Gung Research Database: A multi-institutional database consisting of original medical records. Biomed. J. 2017, 40, 263–269. [Google Scholar] [CrossRef] [PubMed]
  22. Ward, M.E.; Musa, M.N.; Bailey, L. Clinical pharmacokinetics of lithium. J. Clin. Pharmacol. 1994, 34, 280–285. [Google Scholar] [CrossRef] [PubMed]
  23. Peralta, C.A.; Shlipak, M.G.; Judd, S.; Cushman, M.; McClellan, W.; Zakai, N.A.; Safford, M.M.; Zhang, X.; Muntner, P.; Warnock, D. Detection of chronic kidney disease with creatinine, cystatin C, and urine albumin-to-creatinine ratio and association with progression to end-stage renal disease and mortality. JAMA 2011, 305, 1545–1552. [Google Scholar] [CrossRef] [Green Version]
  24. Nichols, T.A.; Drayton, S.J.; Borckardt, J.; Taber, D.J. Lithium dosing equations: Are they accurate? Ann. Pharmacother. 2014, 48, 596–600. [Google Scholar] [CrossRef]
  25. Yoshida, K.; Uchida, H.; Suzuki, T.; Watanabe, M.; Yoshino, N.; Houchi, H.; Mimura, M.; Fukuoka, N. Prediction Model of Serum Lithium Concentrations. Pharmacopsychiatry 2018, 51, 82–88. [Google Scholar] [CrossRef] [PubMed]
  26. Cooper, T.B.; Bergner, P.E.; Simpson, G.M. The 24-h serum lithium level as a prognosticator of dosage requirements. Am. J. Psychiatry 1973, 130, 601–603. [Google Scholar] [CrossRef]
  27. Nolen, W.A.; Licht, R.W.; Young, A.H.; Malhi, G.S.; Tohen, M.; Vieta, E.; Kupka, R.W.; Zarate, C.; Nielsen, R.E.; Baldessarini, R.J.; et al. What is the optimal serum level for lithium in the maintenance treatment of bipolar disorder? A systematic review and recommendations from the ISBD/IGSLI Task Force on treatment with lithium. Bipolar Disord. 2019, 21, 394–409. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  28. Finley, P.R. Drug Interactions with Lithium: An Update. Clin. Pharmacokinet. 2016, 55, 925–941. [Google Scholar] [CrossRef]
  29. Hsu, C.W.; Lee, S.Y.; Yang, Y.H.; Wang, L.J. Brand-Name Antidepressants Outperform Their Generic Counterparts in Preventing Hospitalization for Depression: The Real-World Evidence from Taiwan. Int. J. Neuropsychopharmacol. 2020, 23, 653–661. [Google Scholar] [CrossRef]
  30. Jonsson, P.; Wohlin, C. An evaluation of k-nearest neighbour imputation using Likert data. In Proceedings of the 10th International Symposium on Software Metrics, Washington, DC, USA, 11–17 September 2004; pp. 108–118. [Google Scholar]
  31. Rassen, J.A.; Shelat, A.A.; Myers, J.; Glynn, R.J.; Rothman, K.J.; Schneeweiss, S. One-to-many propensity score matching in cohort studies. Pharmacoepidemiol. Drug Saf. 2012, 21 (Suppl. 2), 69–80. [Google Scholar] [CrossRef]
  32. Baştanlar, Y.; Ozuysal, M. Introduction to machine learning. Methods Mol. Biol. 2014, 1107, 105–128. [Google Scholar] [CrossRef] [Green Version]
  33. Tomasik, J.; Han, S.Y.S.; Barton-Owen, G.; Mirea, D.-M.; Martin-Key, N.A.; Rustogi, N.; Lago, S.G.; Olmert, T.; Cooper, J.D.; Ozcan, S.; et al. A machine learning algorithm to differentiate bipolar disorder from major depressive disorder using an online mental health questionnaire and blood biomarker data. Transl. Psychiatry 2021, 11, 41. [Google Scholar] [CrossRef] [PubMed]
  34. Nunes, A.; Ardau, R.; Berghöfer, A.; Bocchetta, A.; Chillotti, C.; Deiana, V.; Garnham, J.; Grof, E.; Hajek, T.; Manchia, M.; et al. Prediction of lithium response using clinical data. Acta Psychiatr. Scand. 2020, 141, 131–141. [Google Scholar] [CrossRef]
  35. Li, M.; Das, T.; Deng, W.; Wang, Q.; Li, Y.; Zhao, L.; Ma, X.; Wang, Y.; Yu, H.; Li, X.; et al. Clinical utility of a short resting-state MRI scan in differentiating bipolar from unipolar depression. Acta Psychiatr. Scand. 2017, 136, 288–299. [Google Scholar] [CrossRef]
  36. Cortes, C.; Vapnik, V. Support-vector networks. Mach. Learn. 1995, 20, 273–297. [Google Scholar] [CrossRef]
  37. Tin Kam, H. The random subspace method for constructing decision forests. IEEE 1998, 20, 832–844. [Google Scholar] [CrossRef] [Green Version]
  38. Chen, T.; Guestrin, C. XGBoost: A scalable tree boosting system. In Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, San Francisco, CA, USA, 13–17 August 2016; pp. 785–794. [Google Scholar]
  39. Zhang, S.; Golbraikh, A.; Oloff, S.; Kohn, H.; Tropsha, A. A novel automated lazy learning QSAR (ALL-QSAR) approach: Method development, applications, and virtual screening of chemical databases using validated ALL-QSAR models. J. Chem. Inf. Model. 2006, 46, 1984–1995. [Google Scholar] [CrossRef] [Green Version]
  40. Varsou, D.D.; Melagraki, G.; Sarimveis, H.; Afantitis, A. MouseTox: An online toxicity assessment tool for small molecules through Enalos Cloud platform. Food Chem. Toxicol. 2017, 110, 83–93. [Google Scholar] [CrossRef] [PubMed]
  41. Tibshirani, R. Regression Shrinkage and Selection Via the Lasso. JSTOR 1996, 58, 267–288. [Google Scholar] [CrossRef]
  42. Kosmicki, J.A.; Sochat, V.; Duda, M.; Wall, D.P. Searching for a minimal set of behaviors for autism detection through feature selection-based machine learning. Transl. Psychiatry 2015, 5, e514. [Google Scholar] [CrossRef] [Green Version]
  43. Lundberg, S.M.; Lee, S.-I. A unified approach to interpreting model predictions. In Proceedings of the 31st International Conference on Neural Information Processing Systems, Long Beach, CA, USA, 4–9 December 2017; pp. 4768–4777. [Google Scholar]
  44. Zetin, M.; Garber, D.; De Antonio, M.; Schlegel, A.; Feureisen, S.; Fieve, R.; Jewett, C.; Reus, V.; Huey, L.Y. Prediction of lithium dose: A mathematical alternative to the test-dose method. J. Clin. Psychiatry 1986, 47, 175–178. [Google Scholar] [PubMed]
  45. Stip, E.; Dufresne, J.; Boulerice, B.; Elie, R. Accuracy of the Pepin method to determine appropriate lithium dosages in healthy volunteers. J. Psychiatry Neurosci. 2001, 26, 330–335. [Google Scholar] [PubMed]
  46. Higuchi, S.; Fukuoka, R.; Aoyama, T.; Horioka, M. Predicting serum lithium concentration using Bayesian method: A comparison with other methods. J. Pharmacobiodyn. 1988, 11, 158–174. [Google Scholar] [CrossRef] [PubMed]
  47. Malhi, G.S.; Gessler, D.; Outhred, T. The use of lithium for the treatment of bipolar disorder: Recommendations from clinical practice guidelines. J. Affect. Disord 2017, 217, 266–280. [Google Scholar] [CrossRef] [PubMed]
  48. Malhi, G.S.; Tanious, M.; Das, P.; Berk, M. The science and practice of lithium therapy. Aust. N. Z. J. Psychiatry 2012, 46, 192–211. [Google Scholar] [CrossRef] [PubMed]
  49. Eagles, J.M.; McCann, I.; MacLeod, T.N.; Paterson, N. Lithium monitoring before and after the distribution of clinical practice guidelines. Acta Psychiatr. Scand. 2000, 101, 349–353. [Google Scholar] [CrossRef] [PubMed]
  50. Chan, B.S.; Cheng, S.; Isoardi, K.Z.; Chiew, A.; Siu, W.; Shulruf, B.; Vecellio, E.; Buckley, N.A. Effect of age on the severity of chronic lithium poisoning. Clin. Toxicol. 2020, 58, 1023–1027. [Google Scholar] [CrossRef]
  51. Bocchetta, A.; Ardau, R.; Fanni, T.; Sardu, C.; Piras, D.; Pani, A.; Del Zompo, M. Renal function during long-term lithium treatment: A cross-sectional and longitudinal study. BMC Med. 2015, 13, 12. [Google Scholar] [CrossRef] [Green Version]
  52. Potkin, S.G.; Thyrum, P.T.; Bera, R.; Carreon, D.; Alva, G.; Kalali, A.H.; Yeh, C. Open-label study of the effect of combination quetiapine/lithium therapy on lithium pharmacokinetics and tolerability. Clin. Ther. 2002, 24, 1809–1823. [Google Scholar] [CrossRef]
  53. Granneman, G.R.; Schneck, D.W.; Cavanaugh, J.H.; Witt, G.F. Pharmacokinetic interactions and side effects resulting from concomitant administration of lithium and divalproex sodium. J. Clin. Psychiatry 1996, 57, 204–206. [Google Scholar]
  54. Amerio, A.; Russo, D.; Miletto, N.; Aguglia, A.; Costanza, A.; Benatti, B.; Odone, A.; Barroilhet, S.A.; Brakoulias, V.; Dell’Osso, B.; et al. Polypharmacy as maintenance treatment in bipolar illness: A systematic review. Acta Psychiatr. Scand. 2021, 144, 259–276. [Google Scholar] [CrossRef]
  55. Altamura, A.C.; Mundo, E.; Dell’Osso, B.; Tacchini, G.; Buoli, M.; Calabrese, J.R. Quetiapine and classical mood stabilizers in the long-term treatment of Bipolar Disorder: A 4-year follow-up naturalistic study. J. Affect. Disord. 2008, 110, 135–141. [Google Scholar] [CrossRef] [PubMed]
  56. Colvard, M.D.; Gentry, J.D.; Mullis, D.M. Neurotoxicity with combined use of lithium and haloperidol decanoate. Prim. Care Companion CNS Disord. 2013, 15, PCC13l01563. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  57. Hsu, C.-W.; Lee, Y.; Lee, C.-Y.; Lin, P.-Y. Neurotoxicity and nephrotoxicity caused by combined use of lithium and risperidone: A case report and literature review. BMC Pharmacol. Toxicol. 2016, 17, 59. [Google Scholar] [CrossRef] [Green Version]
  58. Netto, I.; Phutane, V.H.; Ravindran, B. Lithium Neurotoxicity Due to Second-Generation Antipsychotics Combined With Lithium: A Systematic Review. Prim. Care Companion CNS Disord. 2019, 21, 17r02225. [Google Scholar] [CrossRef] [PubMed]
  59. McCoy, S.N.; Thavundayil, J.X.; Schwartz, G.; Etienne, P. Lithium ratio and hypertension in manic-depressive patients. Am. J. Psychiatry 1982, 139, 247–248. [Google Scholar] [CrossRef] [PubMed]
  60. Cirillo, M.; Laurenzi, M.; Panarelli, W.; Trevisan, M.; Dyer, A.R.; Stamler, R.; Stamler, J. Sodium-lithium countertransport and blood pressure change over time: The Gubbio study. Hypertension 1996, 27, 1305–1311. [Google Scholar] [CrossRef] [PubMed]
  61. Anton, R.F.; Paladino, J.A.; Morton, A.; Thomas, R.W. Effect of acute alcohol consumption on lithium kinetics. Clin. Pharmacol. Ther. 1985, 38, 52–55. [Google Scholar] [CrossRef] [PubMed]
  62. Vrontaki, E.; Melagraki, G.; Mavromoustakos, T.; Afantitis, A. Searching for anthranilic acid-based thumb pocket 2 HCV NS5B polymerase inhibitors through a combination of molecular docking, 3D-QSAR and virtual screening. J. Enzyme Inhib. Med. Chem. 2016, 31, 38–52. [Google Scholar] [CrossRef]
Biomedicines 09 01558 g001 550
Figure 1. Flowchart of the selection process for this study and outline of analytic procedures.
Figure 1. Flowchart of the selection process for this study and outline of analytic procedures.
Biomedicines 09 01558 g001
Biomedicines 09 01558 g002 550
Figure 2. The suggested workflow for clinicians to use the predictive models of this study. (A) binary prediction, (B) continuous prediction.
Figure 2. The suggested workflow for clinicians to use the predictive models of this study. (A) binary prediction, (B) continuous prediction.
Biomedicines 09 01558 g002
Table
Table 1. Characteristics of lithium-treated patients, comparing inpatient development data and inpatient test data.
Table 1. Characteristics of lithium-treated patients, comparing inpatient development data and inpatient test data.
CharacteristicsDevelopment
(n = 1505)		Test
(n = 204)		t or χ²p
Lithium serum levels, mmol/L 0.69 ± 0.210.70 ± 0.22−0.510.612
Basic Information
Age, year 43.13 ± 13.7042.95 ± 13.820.180.856
Sex, female 696 (46.25)87 (42.65)0.940.333
Clinical Characteristics
Height, m 1.64 ± 0.081.64 ± 0.09−0.900.370
Weight, kg 69.64 ± 14.7370.35 ± 14.54−0.650.517
Systolic blood pressure, mmHg 122.00 ± 10.86122.30 ± 10.42−0.380.706
Diastolic blood pressure, mmHg 76.28 ± 7.4876.21 ± 7.760.140.890
Lithium Prescription
Daily dose, mg/day 867.70 ± 266.70896.30 ± 257.20−1.440.148
Dosing frequency, time/day 2.57 ± 0.782.60 ± 0.73−0.570.568
Last dose, mg 354.60 ± 125.20362.50 ± 131.30−0.840.401
Time interval, hour 13.08 ± 1.4413.11 ± 1.32−0.260.796
Concomitant Psychotropic Drugs
Mood Stabilizers
Carbamazepine 73 (4.85)11 (5.39)0.110.737
Lamotrigine 64 (4.25)7 (3.43)0.300.581
Topiramate 110 (7.31)15 (7.35)0.0010.982
Valproic acid 524 (34.82)67 (32.84)0.310.578
Antidepressants
SSRI 175 (11.63)24 (11.76)0.0030.954
SNRI 88 (5.85)10 (4.90)0.300.586
Trazodone 38 (2.52)4 (1.96)0.240.625
Mirtazapine 45 (2.99)7 (3.43)0.120.731
Bupropion 62 (4.12)8 (3.92)0.020.894
Agomelatine 48 (3.19)7 (3.43)0.030.854
Antipsychotics
Typical antipsychotics 177 (11.76)23 (11.27)0.040.839
The benzamides 49 (3.26)7 (3.43)0.020.895
The -dones 295 (19.60)49 (24.02)2.180.140
The -pines 1033 (68.64)129 (63.24)2.410.121
Aripiprazole 157 (10.43)23 (11.27)0.140.713
Anxiolytics, Sedatives, or Hypnotics
Benzodiazepines 1252 (83.19)169 (82.84)0.020.901
Non-benzodiazepines 208 (13.82)32 (15.69)0.520.472
Acetylcholinesterase inhibitors 12 (0.80)1 (0.49)0.220.636
Mental Disorders
Bipolar disorders 1108 (73.62)150 (73.53)0.0010.978
Laboratory Data
Serum creatinine, mg/dL 0.79 ± 0.160.80 ± 0.160.120.469
BUN, mg/dL 9.99 ± 2.649.97 ± 2.58−0.720.902
Abbreviations: BUN; blood urea nitrogen; SNRI, serotonin norepinephrine reuptake inhibitor; SSRI, selective serotonin reuptake inhibitor. Data was expressed as N (percentage) or mean ±standard deviation. Time interval is the time between the blood draw and last lithium dose.
Table
Table 2. The model performance of the binary and continuous outcomes between different algorithms in inpatient test data.
Table 2. The model performance of the binary and continuous outcomes between different algorithms in inpatient test data.
BinaryLogRSVMRFXGBoost
Sensitivity0.89 (0.84–0.93)0.94 (0.91–0.97)0.96 (0.95–0.97)0.90 (0.87–0.94)
Specificity0.43 (0.36–0.51)0.32 (0.24–0.41)0.22 (0.13–0.31)0.38 (0.34–0.41)
AUC-ROC0.75 (0.73–0.76)0.76 (0.74–0.77)0.78 (0.75–0.81)0.78 (0.74–0.81)
Accuracy0.73 (0.71–0.75)0.73 (0.71–0.75)0.70 (0.68–0.73)0.72 (0.70–0.74)
ContinuousLinRSVMRFXGBoost
MAE0.16 (0.16–0.16)0.14 (0.13–0.15)0.15 (0.15–0.16)0.15 (0.15–0.16)
MSE0.04 (0.04–0.04)0.03 (0.03–0.03)0.04 (0.03–0.04)0.04 (0.04–0.04)
RMSE0.20 (0.19–0.20)0.17 (0.17–0.18)0.19 (0.18–0.19)0.19 (0.19–0.20)
Accuracy0.69 (0.68–0.70)0.75 (0.71–0.79)0.68 (0.67–0.70)0.68 (0.67–0.70)
Abbreviations: AUC-ROC, area under the curve of receiver operator characteristic; LinR, linear regression; LogR, logistic regression; MAE, mean absolute error; MSE, mean-square error; RF, random forests; RMSE, root-mean-square error; SVM, support vector machine; XGBoost, extreme gradient boosting.
Table
Table 3. Top 10 features of the binary and continuous outcomes between different algorithms.
Table 3. Top 10 features of the binary and continuous outcomes between different algorithms.
BinaryEnsembleLogRSVMRFXGBoost
Top 1Daily dose *Daily doseDaily doseDaily doseDaily dose
Top 2Age *MCHCTopiramateMCHCAge
Top 3Last dose *Valproic acidNSAIDsLast doseValproic acid
Top 4The -pines *Renal diseasesHyperlipidemiaDosing frequencyHeight
Top 5Valproic acid *AgeElimination disordersAgeTime interval
Top 6WeightWeightAgeThe -pinesBenzodiazepines
Top 7SBP *Substance use disordersThe -pinesValproic acidRBC
Top 8HypertensionHypertensionTime intervalBenzodiazepinesWBC
Top 9MCHCThe -pinesSleep-wake disordersHemoglobinMCHC
Top 10Substance use disorders *Last doseLast doseSerum creatinineRDW-SD
ContinuousEnsembleLinRSVMRFXGBoost
Top 1Daily dose *Daily doseDaily doseDaily doseDaily dose
Top 2Age *Valproic acidAgeAgeAge
Top 3Valproic acid *AgeBeta blockersValproic acidValproic acid
Top 4The -pines *WeightHyperlipidemiaWeightThe -pines
Top 5Substance use disorders *TopiramateDepressive disordersRBCSBP
Top 6SBP *ARBSBPHeightWeight
Top 7Beta blockersMCHCValproic acidRDW-SDHeight
Top 8Last dose *HypertensionMild DMSBPTime interval
Top 9PotassiumOcular bleedingSexLast doseTopiramate
Top 10NSAIDsSubstance use disordersSleep-wake disordersTopiramateBenzodiazepines
Abbreviations: ARB, angiotensin receptor blockers; LinR, linear regression; LogR, logistic regression; MCHC, mean corpuscular hemoglobin concentration; Mild DM = diabetes mellitus without end organ damage; NSAIDs, non-steroidal anti-inflammatory drugs; RBC, red blood cell; RDW-SD, red cell distribution width-standard deviation; RF, random forests; SBP, systolic blood pressure; Substance use disorders = substance-related and addictive disorders; SVM, support vector machine; WBC, white blood cell; XGBoost, extreme gradient boosting. * indicated simultaneous occurrence in the top 10 ensemble features of binary and continuous machine learning models.
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Share and Cite

MDPI and ACS Style

Hsu, C.-W.; Tsai, S.-Y.; Wang, L.-J.; Liang, C.-S.; Carvalho, A.F.; Solmi, M.; Vieta, E.; Lin, P.-Y.; Hu, C.-A.; Kao, H.-Y. Predicting Serum Levels of Lithium-Treated Patients: A Supervised Machine Learning Approach. Biomedicines 2021, 9, 1558. https://doi.org/10.3390/biomedicines9111558

AMA Style

Hsu C-W, Tsai S-Y, Wang L-J, Liang C-S, Carvalho AF, Solmi M, Vieta E, Lin P-Y, Hu C-A, Kao H-Y. Predicting Serum Levels of Lithium-Treated Patients: A Supervised Machine Learning Approach. Biomedicines. 2021; 9(11):1558. https://doi.org/10.3390/biomedicines9111558

Chicago/Turabian Style

Hsu, Chih-Wei, Shang-Ying Tsai, Liang-Jen Wang, Chih-Sung Liang, Andre F. Carvalho, Marco Solmi, Eduard Vieta, Pao-Yen Lin, Chien-An Hu, and Hung-Yu Kao. 2021. "Predicting Serum Levels of Lithium-Treated Patients: A Supervised Machine Learning Approach" Biomedicines 9, no. 11: 1558. https://doi.org/10.3390/biomedicines9111558

APA Style

Hsu, C.-W., Tsai, S.-Y., Wang, L.-J., Liang, C.-S., Carvalho, A. F., Solmi, M., Vieta, E., Lin, P.-Y., Hu, C.-A., & Kao, H.-Y. (2021). Predicting Serum Levels of Lithium-Treated Patients: A Supervised Machine Learning Approach. Biomedicines, 9(11), 1558. https://doi.org/10.3390/biomedicines9111558

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop