Next Article in Journal
Development of a Machine Learning Model to Predict Non-Durable Response to Anti-TNF Therapy in Crohn’s Disease Using Transcriptome Imputed from Genotypes
Next Article in Special Issue
Personalized Diagnosis and Treatment for Neuroimaging in Depressive Disorders
Previous Article in Journal
The Role of Allergen-Specific Immunotherapy in ENT Diseases: A Systematic Review
Previous Article in Special Issue
The Effectiveness of Predicting Suicidal Ideation through Depressive Symptoms and Social Isolation Using Machine Learning Techniques
 
 
Search for Articles:
Title / Keyword
Author / Affiliation / Email
Journal
Article Type
 
 
Advanced Search
 
Section
Special Issue
Volume
Issue
Number
Page
 
Journals
JPM
Volume 12
Issue 6
10.3390/jpm12060945
jpm-logo
Submit to this Journal Review for this Journal Propose a Special Issue
► Article Menu

Article Menu

share Share announcement Help format_quote Cite question_answer Discuss in SciProfiles thumb_up
...
Endorse textsms
...
Comment
first_page
settings
Order Article Reprints
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

A Machine Learning Approach for Predicting Wage Workers’ Suicidal Ideation

by
Hwanjin Park
1 and
Kounseok Lee
2,*
1
Department of Occupational & Environmental Medicine, Kangbuk Samsung Hospital, Sungkyunkwan University School of Medicine, Seoul 03181, Korea
2
Department of Psychiatry, Hanyang University Medical Center, Seoul 04763, Korea
*
Author to whom correspondence should be addressed.
J. Pers. Med. 2022, 12(6), 945; https://doi.org/10.3390/jpm12060945
Submission received: 29 April 2022 / Revised: 5 June 2022 / Accepted: 6 June 2022 / Published: 9 June 2022
(This article belongs to the Special Issue Advances in the Use of Machine Learning for the Clinical Research of Mood Disorders)
Versions Notes

Abstract

:
(1) Background: Workers spend most of their days working. One’s working environment can be a risk factor for suicide. In this study, we examined whether suicidal ideation can be predicted using individual characteristics, emotional states, and working environments. (2) Methods: Nine years of data from the Korean National Health and Nutrition Survey were used. A total of 12,816 data points were analyzed, and 23 variables were selected. The random forest technique was used to predict suicidal thoughts. (3) Results: When suicidal ideation cases were predicted using all of the independent variables, 98.9% of cases were predicted, and 97.4% could be predicted using only work-related conditions. (4) Conclusions: It was confirmed that suicide risk could be predicted efficiently when machine learning techniques were applied using variables such as working environments.

1. Introduction

Adult workers spend most of their days working. Work is good for one’s mental health, but a poor work environment can cause physical and mental health problems [1]. When a worker has mental health problems, various difficulties such as low self-confidence, high tension, frequent mistakes, low energy, and conflicts between colleagues can occur, which can ultimately lead to a decrease in work performance. If mental health problems eventually lead to suicide, valuable personal and social assets will be lost not only to workers, but also to businesses. Therefore, management of the mental health of workers and suicide prevention should be carried out in the workplace.
The working environment refers to all factors that affect work activities, including various managerial factors, human relationships, and physical conditions related to job performance, as well as the physical environment in terms of job performance. In other words, the working environment can be divided into the physical environment, including the type of employment and type of business, the labor structure, which includes the number of employees, working hours, and shift work, and the socio-psychological environment, which is affected by the characteristics of the work being performed [2].
It was found that the longer the working hours, the higher was the risk of depression and suicidal ideation [3,4,5]. In contrast, working shorter than normal hours significantly reduced suicidal ideations, suggesting that priority working hours are a major risk factor [3]. The risk of absenteeism due to accidents and illness was three times higher in those who did not work more than 40 h a week [6]. Therefore, the working hours of workers can be an important factor in the working environment. Long working hours can pose a threat to the health and safety of workers, their families, businesses, and the public, even if they are voluntary choices made by workers. For this reason, the International Labor Organization suggests that the appropriate working hours are eight hours a day or 48 h a week [7].
In addition, irregular working environment factors, such as overtime and irregular working hours, can cause changes in an individual’s physiological rhythm and threaten their health [8]. In the case of nonregular workers, the rate of use of sick leave, which can be used in cases of illness, was significantly lower than that of regular workers [9]. Another study suggested that suicidal ideation was significantly higher among nonregular workers than among regular workers [10].
Job characteristics include job control and autonomy, and it is known that exposure to excessive noise, sunlight, and specific substances (e.g., pesticides) can affect the work environment [11]. It has also been shown that occupational skill level according to the type of occupation affects one’s suicide risk [12].
Although these characteristics can be used to evaluate the risk of suicide in a worker group, there have been no studies on the overall risk of several factors. Suicide does not occur due to a single cause; rather it occurs as a result of the complex contribution of several factors such as biological vulnerability, psychological state, and clinical characteristics [13].
Although the early identification and treatment of high-risk patients is a key suicide prevention strategy [14], high-risk patients are often not recognized by healthcare providers [15]. To overcome this issue, a suicide risk screening study using machine learning has been conducted [16,17,18]. Machine learning algorithms can change and improve as they are exposed to new data, and these detection patterns have many potential advantages over passively guided approaches to model specifications, particularly in terms of efficiency, complexity, and flexibility [19]. Machine learning techniques have the advantage of being able to investigate a wide range of complex associations between a large number of potential factors to generate algorithms that optimize predictions [20].
To date, no studies have applied machine learning techniques to predict suicide risk in workers. Therefore, in this study, the demographic characteristics, medical history, and personal lifestyle differences of a group with suicidal thoughts were first examined. Their emotional status, stress levels, subjective health status, and depression factors were then examined to determine whether there were differences. In addition, we aimed to determine how work characteristics such as occupational classification, working hours, and work type affect suicidal ideation. This study aimed to investigate whether the suicide risk of workers could be efficiently predicted using machine learning techniques based on working environments and emotional factors.

2. Materials and Methods

2.1. Study Population

This study used data from the Korean National Health and Nutrition Examination Survey (KHANES). Data from a total of 9 years (2007–2013, 2015, and 2017) were used [16]. In 2014, 2016, and 2018, suicidal ideation among adults was not investigated, and part-time work (full-time or part-time) was not investigated in 2019; therefore, these types of data were excluded from the study. The KHANES is a national survey of the health and nutritional status of noninstitutionalized civilians in Korea [21]. This study targeted 12,865 wage earners aged 19 years or older from nine years of data. The subjects were administered a questionnaire that included the question: “During the past year, have you ever felt that you were willing to die?” Responses were acquired from 12,816 people, excluding 49 who did not respond. This study was approved by the Institutional Review Board of the Korea Disease Control and Prevention Agency (IRB No 2018-01-03-P-A).

2.2. Work-Related Conditions

Occupations were classified according to the major classification codes of Korea’s standard occupational classification system. Among the KHANES data, the work-related variables commonly investigated in the 9 years of data were as follows:
  • Occupation type: (1) managers, experts, and related workers; (2) office workers; (3) service and sales workers; (4) skilled workers in agriculture, forestry, and fisheries; (5) technicians, equipment operators, machine assembly and operation workers; (6) simple labor workers.
  • Occupational status: (1) full-time; (2) temporary; (3) day worker.
  • Working hour system: (1) full-time worker; (2) part-time worker.
  • Type of shiftwork: (1) day work; (2) evening shift (14:00~24:00); (3) night shift (21:00–08:00); (4) regular day and night shift work; (5) 24 h shift work, split work (more than two working hours per day), irregular shift work, and other forms of shift work.
  • Working hours: average working hours per week.
  • Classification of regular and nonregular workers, which are thought to have an impact on suicidal ideation, were excluded as the survey was conducted only at 13, 15, and 17 years.

2.3. Data Processing and Machine Learning

This study evaluated the suitability of the suicidal ideation prediction model for wage workers by selecting 23 analyzable variables, presented in Table 1, that are thought to be related to suicidal ideation. For the second model, we conducted an analysis that included only work-related variables, including occupations, occupational status, working hour system, type of shift work, and weekly working hours. We analyzed whether suicidal ideation could be predicted using only the work-related conditions of wage workers.
Among machine learning algorithms, random forest (RF) is a method for finding optimal results through randomly combining results and constructing several decision trees. Random forest is widely used due to its good predictive performance, especially in mental health-related studies [16,22]. Therefore, in this study, a RF was constructed to evaluate various variables, work-related variables, and the suitability of the suicidal ideation prediction model for wage workers.
The subjects were analyzed by dividing the training and test data by 70% and 30%, respectively. Repeated 10-fold cross-validation was used to develop the analysis model. In the 10-fold cross validation, the data is divided into 10 folds of the same size, and the remaining 9 folds are used for training while each fold is used as a validation set. This is repeated five times to avoid overfitting and to generalize the prediction model. To find the optimal parameter of the RF, the analysis was conducted with the grid search method and the optimized hyperparameter of the RF was applied to the final model. The fitted model was used to predict the suicidal ideation of wage workers and it was compared with actual suicidal ideation using a confusion matrix with the training data. The performance of the predictive model was evaluated by calculating the accuracy, sensitivity, specificity, positive predictive value, negative predictive value, and area under the curve. Area under the curve (AUC) means the area under the roc curve drawn with a false positive rate as the x-axis and true positive rate as the y-axis. An AUC value of 0.5 indicated no discriminative value. An AUC value ≥0.75 is considered clinically useful [23]. R version 4.1.2 was used for all analyses with the “mice” package in R for processing missing data [24], the “caret” package for cross validation, and the “ROCR” package for the ROC curve.

3. Results

3.1. Differences in General Characteristics of Workers according to Suicidal Ideation and Work-Related Conditions

A suicide-related questionnaire was administered to 12,816 subjects, and it included the question: “During the past year, have you ever felt that you were willing to die?” A total of 915 people answered yes (7.14% of the subjects). The proportion of women who had suicidal thoughts in the past year was 66.7%, which was higher than that of women who had not (47.3%) (p < 0.001). The lower the household income (p < 0.001) and the lower the education level, the higher the rate of suicidal ideation (p < 0.001). Subjects in the suicidal ideation group were divorced (7.3%), separated, or widowed (9.1%), and the ratios of these factors were higher than those in the nonsuicidal group. Marital status was also significantly related to suicidal ideation (p < 0.001). Hypertension (p = 0.165), stroke (p = 0.469), cancer (p = 0.626), and diabetes (p = 0.994) were not associated with suicidal ideation, but hyperlipidemia (11.3%, p = 0.038), myocardial infarction, and angina pectoris (2.2%, p = 0.023) were, and the rates of these in the suicidal ideation group was significantly higher than that in the group without suicidal ideation. In the suicidal ideation group, the proportion (8.4%) of the group who drank more than four times a week was higher than that of the group without suicidal thoughts (5.8%, p < 0.001), and smoking (p = 0.049) was significantly associated with suicidal ideation. In the suicidal ideation group, the suicidal ideation rate was significantly higher in the group who felt extreme stress in their daily life (16.8%, p < 0.001). Bad (26.5%) or very bad (5.1%) subjective health status reports were also significantly higher in the suicidal group (p < 0.001). The rate of experiencing depressive symptoms lasting more than two weeks was overwhelmingly higher in the suicidal group (51.3%) compared to in the nonsuicidal group (7.1%) (p < 0.001), and the lower the EQ-5D score, the higher the rate of suicidal thoughts. This ratio was also significantly high (p < 0.001). BMI was not significantly associated with suicidal ideation (p = 0.395).
Regarding work-related conditions, in the suicidal ideation group, simple labor workers had the highest rate of suicidal thoughts (30.8%). In the nonsuicidal group, the proportion of officials, experts, and related workers was highest at 25.7% (p < 0.001). In the group with suicidal thoughts, the ratio of regular workers was lower and the ratio of temporary and daily workers was higher than in the group without suicidal thoughts (p < 0.001). The proportion of part-time workers was significantly higher in the suicidal ideation group (26.0%) than in the nonsuicidal group (19.1%, p < 0.001). In the suicidal ideation group, the proportion of shift workers was higher, as was the ratio of evening shift workers (9.8%) to night shift workers (3.1%; p < 0.001). Working hours per week were shorter in the group with suicidal thoughts (38.97 ± 17.65) than in the group without suicidal ideations (41.25 ± 15.74, p < 0.001, Table 1).

3.2. Prediction Model—Random Forest

In this study, a machine learning technique was used to build a model to predict suicidal ideation among adult wage workers. After analyzing the predictive model for suicidal ideations related to all variables, including depressive symptoms, the predictive model for suicidal ideations related to working conditions, including only the variables related to working conditions, was analyzed. Random forest analysis was performed on both the training data and test data, and the results for the training data are presented. The AUC of the random forest model, which included all variables, including depressive symptoms, was 0.922, and when only work-related conditions were analyzed, the AUC was 0.818, showing good performance. When looking at the confusion matrices, when all variables were included, 100 people out of 641 people who actually had suicidal thoughts were predicted, and 8331 people who did not have suicidal thoughts were accurately predicted. Thus, the accuracy of the model was 0.989, the sensitivity was 0.844, the specificity was 1.000, the positive predictive value was 1.000, and the negative predictive value was 0.988 (Table 2).
When analyzing only work-related conditions, 408 out of 641 people who actually had suicidal thoughts were predicted, and all 8331 people who did not have suicidal thoughts were predicted. Thus, the accuracy of the model was 0.974, the sensitivity was 0.637, the specificity was 1.000, the positive predictive value was 1.000, and the negative predictive value was 0.973 (Table 3).

4. Discussion

This study predicts suicidal ideation in adult wage workers using work-related indicators, working environment factors, emotional state information, and so on. Compared with the group without suicidal thoughts, the participants with suicidal thoughts had higher household incomes, stress levels, and symptoms of depression. They were also more likely to smoke and have chronic diseases. In terms of working environment, it was found that simple, part-time, and shift workers had higher suicidal thoughts. When suicidal ideation was predicted using all these independent variables, 98.9% of subjects with suicidal thoughts could be predicted, and 97.4% could be predicted using only work-related conditions. Depression was the biggest contributing factor to suicidal ideations, and it is known that the global labor loss due to emotional problems is $2.5 trillion, aggravating the economic loss of companies [25].
The work environment acts as a stress factor during work and can lead to depression and suicide; therefore, management of job stress is important to lower job-related suicide [26]. Working environment variables, such as long hours and shift work, can cause major job stress. Such job stress increases corticosteroids in the body, and the continuous increase in corticosteroids directly affects the brain and inhibits the regulation of emotions by downregulating glucocorticoid receptors, making workers vulnerable to depression. This may lead to suicide through direct or indirect complex actions involving other factors [27]. This study confirmed that occupational status, working hours, and working type were all affected. Although unavoidable circumstances are inevitable, in the case of changeable environmental factors, the emotional health of workers can be improved by establishing legal standards.
Although it has been reported that depression and suicide risk increase when working hours are high [3,4], in this study, when compared to the average working hours, the working hours of the group at risk of suicide were relatively short. First, suicidal ideation appears to be related to the significantly lower job stability in the group with suicidal thoughts [28]; the proportion of temporary or daily workers with low job security was 43% (28% in the nonsuicidal group). Second, a previous study showed a significant increase in depression when working hours were extremely long, and suicidal ideation was also found to increase when working more than 69 h [4]. These points are different from the results of this study.
Quality of life in terms of personal health is an indicator of one’s health. It is related to an individual’s health in terms of physical, emotional, and social aspects. Notably, quality of life was not good in the group with suicidal thoughts [29]. Although subjective, decreased overall life satisfaction may be a risk factor for suicidal ideations. In addition, in the group with suicidal thoughts, the perception of stress was relatively high: they felt more depressed for more than 2 weeks, and they felt that their subjective health was not good. Mental health problems such as worker depression can lead to problems such as absenteeism, decreased and exhausted work motivation, tension and relationship conflicts among colleagues, increased accidents at work, and decreased productivity in companies [30]. For workers, depression eventually contributes to long-term absenteeism and early retirement [31,32]. It was also found that workers with major depressive disorder lost 33.4% of their annual salary as a result of decreased productivity due to absenteeism and presenteeism, whereas the control group lost 2.5% of their annual salary [33]. As such, workers’ depression can affect not only their personal health but also lower corporate productivity and, ultimately, lower national competitiveness.
When analyzing by occupation, it was confirmed that the simple labor workers had a relatively high suicidal ideation rate. In a previous study, suicidal ideation rates according to the job status of wage workers were 2.0%, 5.0%, and 7.0% for regular, temporary, and daily workers, respectively. For men, the rate of suicidal ideation was the highest among daily workers (9.6%), and for women, it was the highest among temporary workers (5.6%) [34].
Work-related difficulties, such as excessive job demands, relationship conflicts, and emotional labor, can increase job stress, job satisfaction, emotional exhaustion, and negative health symptoms, such as depression [35,36]. However, factors influencing suicide among workers may include personal life stress and individual sensitivity, in addition to the working environment and job stress.
Suicide prevention programs in the workplace should be structured to prevent each of these stressors and enable the early detection and treatment of suicidal ideations [37]. This study confirmed the possibility of predicting suicidal ideation using common variables and work environments. Among other things, it is necessary to understand the mental health of workers with depression or suicidal thoughts. Once issues have been identified, it can be helpful to take measures that are suitable for each worker or to improve workplace environmental factors.
The limitations of this study are as follows: First, only cross-sectional factors were used, and an overall evaluation of work environments was not performed. In situations where it is impossible to evaluate various working environments with a single evaluation index, it will be helpful to index the work environment by evaluating these environmental aspects through indicators that can be evaluated in the future. Second, only the cross-sectional area evaluation results were used. In the future, it will be necessary to confirm these findings via a cohort study of workers. Third, only suicidal ideation was evaluated. Suicidal behavior can be divided into suicidal thoughts, plans, and attempts. In the end, suicidal thoughts can lead to suicide plans and even suicide attempts, but it should be taken into account that not all suicidal thoughts lead to suicide attempts. Meanwhile, studies using machine learning or artificial intelligence to predict suicide have been conducted. However, these methods may be limited by the quality of the input data, and their clinical utility has not yet been established [18].
The prediction of suicide using machine learning is mainly based on the results of suicide-related questionnaires or emotional evaluations [16,38]. However, the acquisition of such evaluation data may be difficult in practice, as disadvantages in work may be expected if it is targeted to actual workers. The strength of this study is that it utilized typical variables and large-scale data obtained through conventional investigations. This is a national project, and a similar type of investigation will be conducted in the future, making it possible to repeat and improve learning through open data. As for the variables used in the study, evaluation results from environmental and simple emotional aspects were used as opposed to special measurements or evaluations. Above all, this study has great significance in that it predicts the risk of suicide by synthesizing factors such as work-related environmental variables and the emotional aspects of workers. In the future, it will be possible to select risk factors through large-scale learning using various risk factors and create algorithms that utilize factors that characterize the individual status of workers, such as their language, facial expressions, or social media data.

5. Conclusions

This study attempted to predict suicide accidents among wage workers using variables from the National Health and Nutrition Examination Survey. When suicidal ideation cases were predicted using 23 work-, life-, and emotion-related variables with the random forest method, 98.9% were predicted more accurately (F1 score was 0.915), and 97.4% could be predicted using only work-related variables. It was confirmed that when the machine learning technique was applied to workers using working environment factors, suicide could be efficiently predicted. In the future, it will be necessary to identify risk factors using various variables and to make efforts to protect the emotional health of workers.

Author Contributions

Conceptualization, H.P. and K.L.; methodology and validation, H.P.; formal analysis, K.L.; investigation, H.P.; resources and data curation, H.P. and K.L.; writing—original draft preparation, H.P. and K.L.; writing—review and editing, H.P. and K.L.; visualization, H.P.; supervision, K.L.; funding acquisition, K.L. All authors have read and agreed to the published version of the manuscript.

Funding

This research was funded by the Technology Innovation Program (20012931) and Core Technology Development Project (P0018663) funded by the Ministry of Trade, Industry and Energy (MOTIE, Korea).

Institutional Review Board Statement

The study was conducted in accordance with the Declaration of Helsinki, and approved by the Institutional Review Board of the Korea Disease Control and Prevention Agency (IRB No 2018-01-03-P-A).

Informed Consent Statement

Informed consent was obtained from all subjects involved in the study.

Data Availability Statement

Not applicable.

Conflicts of Interest

The authors declare no conflict of interest. The funders had no role in the design of the study; in the collection, analyses, or interpretation of data; in the writing of the manuscript, or in the decision to publish the results.

References

  1. LaMontagne, A.D.; Martin, A.; Page, K.M.; Reavley, N.J.; Noblet, A.J.; Milner, A.J.; Keegel, T.; Smith, P.M. Workplace mental health: Developing an integrated intervention approach. BMC Psychiatry 2014, 14, 131. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  2. Laaksonen, M.; Pitkäniemi, J.; Rahkonen, O.; Lahelma, E. Work arrangements, physical working conditions, and psychosocial working conditions as risk factors for sickness absence: Bayesian analysis of prospective data. Ann. Epidemiol. 2010, 20, 332–338. [Google Scholar] [CrossRef] [PubMed]
  3. Choi, B. Job strain, long work hours, and suicidal ideation in US workers: A longitudinal study. Int. Arch. Occup. Environ. Health 2018, 91, 865–875. [Google Scholar] [CrossRef] [PubMed]
  4. Choi, E.; Choi, K.W.; Jeong, H.-G.; Lee, M.-S.; Ko, Y.-H.; Han, C.; Ham, B.-J.; Chang, J.; Han, K.-M. Long working hours and depressive symptoms: Moderation by gender, income, and job status. J. Affect. Disord. 2021, 286, 99–107. [Google Scholar] [CrossRef] [PubMed]
  5. Yoon, Y.; Ryu, J.; Kim, H.; Jung-Choi, K. Working hours and depressive symptoms: The role of job stress factors. Ann. Occup. Environ. Med. 2018, 30, 46. [Google Scholar] [CrossRef] [PubMed]
  6. Breslin, F.C.; Tompa, E.; Zhao, R.; Pole, J.D.; Amick Iii, B.C.; Smith, P.M.; Hogg-Johnson, S. The relationship between job tenure and work disability absence among adults: A prospective study. Accid. Anal. Prev. 2008, 40, 368–375. [Google Scholar] [CrossRef] [PubMed]
  7. Messenger, J. Working time and the future of work. ILO Future Work Res. Pap. Ser. 2018, 6, 33–37. [Google Scholar]
  8. Presser, H.B.; Gornick, J.C.; Parashar, S. Gender and nonstandard work hours in 12 European countries. Mon. Lab. Rev. 2008, 131, 83. [Google Scholar]
  9. Virtanen, M.; Kivimäki, M.; Joensuu, M.; Virtanen, P.; Elovainio, M.; Vahtera, J. Temporary employment and health: A review. Int. J. Epidemiol. 2005, 34, 610–622. [Google Scholar] [CrossRef] [Green Version]
  10. Lee, W.; Ha, J. The association between nonstandard employment and suicidal ideation: Data from the first~ fourth Korea National Health and Nutrition Examination Surveys. Korean J. Occup. Environ. Med. 2011, 23, 89–97. [Google Scholar] [CrossRef]
  11. Howard, M.C.; Follmer, K.B.; Smith, M.B.; Tucker, R.P.; Van Zandt, E.C. Work and suicide: An interdisciplinary systematic literature review. J. Organ. Behav. 2022, 43, 260–285. [Google Scholar] [CrossRef]
  12. Milner, A.; Spittal, M.J.; Pirkis, J.; LaMontagne, A.D. Suicide by occupation: Systematic review and meta-analysis. Br. J. Psychiatry 2013, 203, 409–416. [Google Scholar] [CrossRef] [PubMed]
  13. Turecki, G.; Brent, D.A.; Gunnell, D.; O’Connor, R.C.; Oquendo, M.A.; Pirkis, J.; Stanley, B.H. Suicide and suicide risk. Nat. Rev. Dis. Primers 2019, 5, 74. [Google Scholar] [CrossRef] [PubMed]
  14. Center for Mental Health Services. National Strategy for Suicide Prevention: Goals and Objectives for Action; Center for Mental Health Services: Rockville MD, USA, 2001.
  15. Luoma, J.B.; Martin, C.E.; Pearson, J.L. Contact with mental health and primary care providers before suicide: A review of the evidence. Am. J. Psychiatry 2002, 159, 909–916. [Google Scholar] [CrossRef]
  16. Kim, S.; Lee, H.-K.; Lee, K. Detecting suicidal risk using MMPI-2 based on machine learning algorithm. Sci. Rep. 2021, 11, 15310. [Google Scholar] [CrossRef]
  17. Kim, S.; Lee, H.-K.; Lee, K. Which PHQ-9 items can effectively screen for suicide? Machine learning approaches. Int. J. Environ. Res. Public Health 2021, 18, 3339. [Google Scholar] [CrossRef]
  18. McHugh, C.M.; Large, M.M. Can machine-learning methods really help predict suicide? Curr. Opin. Psychiatry 2020, 33, 369–374. [Google Scholar] [CrossRef]
  19. Linthicum, K.P.; Schafer, K.M.; Ribeiro, J.D. Machine learning in suicide science: Applications and ethics. Behav. Sci. Law 2019, 37, 214–222. [Google Scholar] [CrossRef]
  20. Walsh, C.G.; Ribeiro, J.D.; Franklin, J.C. Predicting risk of suicide attempts over time through machine learning. Clin. Psychol. Sci. 2017, 5, 457–469. [Google Scholar] [CrossRef]
  21. Kweon, S.; Kim, Y.; Jang, M.-j.; Kim, Y.; Kim, K.; Choi, S.; Chun, C.; Khang, Y.-H.; Oh, K. Data resource profile: The Korea national health and nutrition examination survey (KNHANES). Int. J. Epidemiol. 2014, 43, 69–77. [Google Scholar] [CrossRef] [Green Version]
  22. Gibbons, R.D.; Weiss, D.J.; Frank, E.; Kupfer, D. Computerized adaptive diagnosis and testing of mental health disorders. Annu. Rev. Clin. Psychol. 2016, 12, 83–104. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  23. Flach, P.A.; Hernández-Orallo, J.; Ramirez, C.F. A coherent interpretation of AUC as a measure of aggregated classification performance. In Proceedings of the International Conference on Machine Learning, Bellevue, WA, USA, 28 June–2 July 2011. [Google Scholar]
  24. R Core Team. R: A Language and Environment for Statistical Computing. 2018. Available online: https://www.R-project.org/ (accessed on 14 March 2022).
  25. CeE, B.D.; Jané-Llopis, E.; Abrahams-Gessel, S.; Bloom, L.; Fathima, S.; Feigl, A.; Gaziano, T.; Mowa, M.; Pandya, A.; Prettner, K. The Global Economic Burden of Non-Communicable Diseases; World Economic Forum and the Harvard School of Public Health: Geneva, Switzerland, 2011. [Google Scholar]
  26. Routley, V.H.; Ozanne-Smith, J.E. Work-related suicide in Victoria, Australia: A broad perspective. Int. J. Inj. Control Saf. Promot. 2012, 19, 131–134. [Google Scholar] [CrossRef] [PubMed]
  27. Woo, J.-M.; Postolache, T.Τ. The impact of work environment on mood disorders and suicide: Evidence and implications. Int. J. Disabil. Hum. Dev. 2008, 7, 185–200. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  28. Park, S.-M. Effects of work conditions on suicidal ideation among middle-aged adults in South Korea. Int. J. Soc. Psychiatry 2019, 65, 144–150. [Google Scholar] [CrossRef] [PubMed]
  29. Larson, C.O. Use of the SF-12 instrument for measuring the health of homeless persons. Health Serv. Res. 2002, 37, 733–750. [Google Scholar] [CrossRef] [Green Version]
  30. Harnois, G.; Gabriel, P. Mental Health and Work: Impact, Issues and Good Practices; World Health Organization: Geneva Switzerland, 2000. [Google Scholar]
  31. Henderson, M.; Glozier, N.; Elliott, K.H. Long term sickness absence. BMJ 2005, 330, 802–803. [Google Scholar] [CrossRef]
  32. Karpansalo, M.; Kauhanen, J.; Lakka, T.A.; Manninen, P.; Kaplan, G.A.; Salonen, J.T. Depression and early retirement: Prospective population based study in middle aged men. J. Epidemiol. Community Health 2005, 59, 70–74. [Google Scholar] [CrossRef] [Green Version]
  33. Woo, J.-M.; Kim, W.; Hwang, T.-Y.; Frick, K.D.; Choi, B.H.; Seo, Y.-J.; Kang, E.-H.; Kim, S.J.; Ham, B.-J.; Lee, J.-S. Impact of depression on work productivity and its improvement after outpatient treatment with antidepressants. Value Health 2011, 14, 475–482. [Google Scholar] [CrossRef] [Green Version]
  34. MOHW. White Paper on Suicide Prevention. 2021. Available online: https://www.dinf.ne.jp/d/2/684.html (accessed on 18 March 2022).
  35. Iacovides, A.; Fountoulakis, K.N.; Kaprinis, S.; Kaprinis, G. The relationship between job stress, burnout and clinical depression. J. Affect. Disord. 2003, 75, 209–221. [Google Scholar] [CrossRef]
  36. Lewig, K.A.; Dollard, M.F. Emotional dissonance, emotional exhaustion and job satisfaction in call centre workers. Eur. J. Work Organ. Psychol. 2003, 12, 366–392. [Google Scholar] [CrossRef]
  37. Milner, A.; Page, K.; Spencer-Thomas, S.; LaMontagne, A.D. Workplace suicide prevention: A systematic review of published and unpublished activities. Health Promot. Int. 2014, 30, 29–37. [Google Scholar] [CrossRef] [PubMed]
  38. Oh, J.; Yun, K.; Hwang, J.-H.; Chae, J.-H. Classification of suicide attempts through a machine learning algorithm based on multiple systemic psychiatric scales. Front. Psychiatry 2017, 8, 192. [Google Scholar] [CrossRef] [PubMed] [Green Version]
Table
Table 1. General characteristics of subjects according to suicidal ideation and work type.
Table 1. General characteristics of subjects according to suicidal ideation and work type.
With Suicidal
Ideation (n = 915)		Without Suicidal
Ideation (n = 11,901)		p Value
SexMale305 (33.3%)6276 (52.7%)<0.001
Female610 (66.7%)5625 (47.3%)
Age 45.12 ± 15.3443.96 ± 13.790.014
Family incomeLower192 (21.0%)1169 (9.8%)<0.001
Low-intermediate209 (22.8%)2159 (18.1%)
Middle212 (23.2%)2736 (23.0%)
Upper-intermediate158 (17.3%)2870 (24.1%)
Upper144 (15.7%)2967 (24.9%)
EducationLower than elementary school200 (21.9%)1348 (11.3%)<0.001
Middle school113 (12.3%)979 (8.2%)
High school324 (35.4%)4150 (34.9%)
Higher than university 278 (30.4%)5424 (45.6%)
Marriage statusUnmarried232(25.4%)2566 (21.6%)<0.001
Married533 (58.3%)8384 (70.4%)
Divorced67 (7.3%)426 (3.6%)
Other83 (9.1%)525 (4.4%)
Hypertension historyNo771 (84.3%)10,226 (85.9%)0.165
Yes144 (15.7%)1675 (14.1%)
Dyslipidemia historyNo812 (88.7%)10,808 (90.8%)0.038
Yes103 (11.3%)1093 (9.2%)
Stroke historyNo904 (98.8%)11,787 (99.0%)0.469
Yes11 (1.2%)114 (1.0%)
Myocardial infarction or angina historyNo895 (97.8%)11,748 (98.7%)0.023
Yes20 (2.2%)153 (1.3%)
Diabetes mellitus historyNo870 (95.1%)11,315 (95.1%)0.994
Yes45 (4.9%)586 (4.9%)
Cancer historyNo896 (97.9%)11,624 (97.7%)0.626
Yes19 (2.1%)277 (2.3%)
Alcohol consumptionNot a drinker182 (19.9%)2110 (17.7%)<0.001
One or less/month183 (20.0%)2187 (18.4%)
One time/month112 (12.2%)1360 (11.4%)
2~4 times/month210 (23.0%)3377 (28.4%)
2~3 times/week151 (16.5%)2181 (18.3%)
4 or more/week77 (8.4%)686 (5.8%)
SmokingDaily228 (24.9%)2560 (21.5%)0.049
Occasional24 (2.6%)361 (3.0%)
Not a smoker663 (72.5%)8980 (75.5%)
Subjective stressVery much154 (16.8%)375 (3.2%)
Many426 (46.6%)2681 (22.5%)
Little310 (33.9%)7413 (62.3%)
Rarely25 (2.7%)1432 (12.0%)
Subjective health statusVery good20 (2.2%)572(4.8%)<0.001
Good171 (18.7%)3709 (31.2%)
Moderate434 (47.4%)6197 (52.1%)
Bad243 (26.5%)1306 (11.0%)
Very bad47 (5.1%)117 (1.0%)
Depressed mood over 2 weeksNo446 (48.7%)11,059 (92.9%)<0.001
Yes469 (51.3%)842 (7.1%)
EQ5D score 0.91 ± 0.120.97 ± 0.07<0.001
Body mass index 23.64 ± 3.7523.67 ± 3.420.395
OccupationManagers, experts, and related workers184 (20.1%)3061 (25.7%)<0.001
Office worker153 (16.7%)2586 (21.7%)
Service and salespersonnel183 (20.0%)1952 (16.4%)
Skilled workers inagriculture, forestry, and fishing2 (0.2%)62 (0.5%)
Technician, device/machine operator, and assembly worker111 (12.1%)1995 (16.8%)
Simple labor worker282 (30.8%)2245 (18.9%)
Occupational statusFull-time employee519 (56.7%)8679 (72.9%)<0.001
Temporary227 (24.8%)2149 (19.1%)
Day job169 (18.5%)1073 (9.0%)
Working hour systemFull-time677 (74.0%)9629 (80.9%)<0.001
Part-time238 (26.0%)2272 (19.1%)
Shiftwork typeDay shift725 (79.2%)9806 (82.4%)<0.001
Evening shift (14:00~24:00)90 (9.8%)810 (6.8%)
Night shift (21:00~8:00)28 (3.1%)228 (1.9%)
Regular day and nightshift work36 (3.9%)550 (4.6%)
Etc.36 (3.9%)507 (4.3%)
Weekly working hours 38.97 ± 17.6541.25 ± 15.74<0.001
Values are presented as n (%) or mean ± SD. EQ-5D: EuroQoL-5D.
Table
Table 2. Confusion matrix and prediction scores of random forest model with all variables.
Table 2. Confusion matrix and prediction scores of random forest model with all variables.
ObservationPrediction (n)SensitivitySpecificityAccuracyPositive
Predictive Value		Negative Predictive ValueF1 ScoreAUC
YesNo(%)(%)(%)(%)(%)
Yes54110084.410098.910098.80.9150.922
No08331
Table
Table 3. Confusion matrix and prediction scores of random forest model with work-related variables.
Table 3. Confusion matrix and prediction scores of random forest model with work-related variables.
ObservationPrediction (n)SensitivitySpecificityAccuracyPositive
Predictive Value		Negative
Predictive Value		F1 ScoreAUC
YesNo(%)(%)(%)(%)(%)
Yes40823363.6510097.410097.30.7780.818
No08331
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Share and Cite

MDPI and ACS Style

Park, H.; Lee, K. A Machine Learning Approach for Predicting Wage Workers’ Suicidal Ideation. J. Pers. Med. 2022, 12, 945. https://doi.org/10.3390/jpm12060945

AMA Style

Park H, Lee K. A Machine Learning Approach for Predicting Wage Workers’ Suicidal Ideation. Journal of Personalized Medicine. 2022; 12(6):945. https://doi.org/10.3390/jpm12060945

Chicago/Turabian Style

Park, Hwanjin, and Kounseok Lee. 2022. "A Machine Learning Approach for Predicting Wage Workers’ Suicidal Ideation" Journal of Personalized Medicine 12, no. 6: 945. https://doi.org/10.3390/jpm12060945

APA Style

Park, H., & Lee, K. (2022). A Machine Learning Approach for Predicting Wage Workers’ Suicidal Ideation. Journal of Personalized Medicine, 12(6), 945. https://doi.org/10.3390/jpm12060945

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop