Advantages of Machine Learning in Forensic Psychiatric Research—Uncovering the Complexities of Aggressive Behavior in Schizophrenia

Abstract

Linear statistical methods may not be suited to the understanding of psychiatric phenomena such as aggression due to their complexity and multifactorial origins. Here, the application of machine learning (ML) algorithms offers the possibility of analyzing a large number of influencing factors and their interactions. This study aimed to explore inpatient aggression in offender patients with schizophrenia spectrum disorders (SSDs) using a suitable ML model on a dataset of 370 patients. With a balanced accuracy of 77.6% and an AUC of 0.87, support vector machines (SVM) outperformed all the other ML algorithms. Negative behavior toward other patients, the breaking of ward rules, the PANSS score at admission as well as poor impulse control and impulsivity emerged as the most predictive variables in distinguishing aggressive from non-aggressive patients. The present study serves as an example of the practical use of ML in forensic psychiatric research regarding the complex interplay between the factors contributing to aggressive behavior in SSD. Through its application, it could be shown that mental illness and the antisocial behavior associated with it outweighed other predictors. The fact that SSD is also highly associated with antisocial behavior emphasizes the importance of early detection and sufficient treatment.

1. Introduction

With the rapid technological progress of the past few years, artificial intelligence (AI) is increasingly being put to use in medical research. Often equated with human-like robots by the general public, AI is ultimately any system that adapts its performance based on its perception of the environment. This includes advanced statistics such as machine learning (ML), which allows a variety of variables and their relationship to one another to be analyzed through complex mathematical algorithms, as well as the quantification of the quality of a statistical model [1,2,3,4]. When it comes to psychiatric research though, statistical analyses are usually conducted using null hypothesis significance tests (NHSTs) or simple linear regressions. This results in the following certain limitations: (I) mainly linear relationships can be determined, and, with NHSTs, it is not even possible to examine the relationships between the variables themselves; (II) in order to avoid alpha error accumulation, only a limited number of variables can be analyzed; and (III) the research question must be precisely defined and rather constrained, as it can only be determined whether a (null) hypothesis is true or not. However, this approach does not accommodate psychiatric syndromes with their complex and often highly interdependent multifactorial relationships. The genesis of psychiatric diseases and pathological behavioral disorders is by no means a linear process influenced by only single, independent factors. This is especially true for the generally under-researched field of forensic psychiatry, where the interplay of psychopathology, offending, and aggression has yet to be comprehensively understood. Consequently, to investigate such phenomena, modern statistical methods such as ML are necessary and already applied in psychiatric research areas regarding pharmaceuticals or neuroimaging [5,6,7,8,9]. The following analysis should serve as an example of the practical use of ML in the field of forensic psychiatry, specifically aggression and schizophrenia spectrum disorders (SSDs). Factors linked to aggressive behavior outside the clinical setting have recently been evaluated by means of ML and include higher PANSS scores as well as younger age at SSD diagnosis [10,11,12]. For this study, we chose an explorative approach, as aggression is considered to be a multifactorial, complex phenomenon, mediated through a broad variety of parameters from different domains. This study now aims to determine the most predictive factors of aggression within the institutional setting, based on a unique group of forensic offenders with SSD, (objective I) and to quantify the performance of the calculated model (objective II).

2. Materials and Methods

The files of 370 delinquent patients diagnosed with SSD according to ICD-9 (295.x) [13] and ICD-10 (F20–29.x) [14], who were admitted to the Center for Inpatient Forensic Therapies of the University Hospital of Psychiatry Zurich, were assessed retrospectively. This comprehensive dataset included items from the following domains: social-demographic data, childhood/youth experiences, psychiatric history, past criminal history, social/sexual functioning, details on the offense leading to forensic hospitalization, prison data, and particularities of the current hospitalization and psychopathological symptoms. The latter was defined by an adapted positive and negative symptom scale (PANSS), whereby symptoms were divided into the usual 30 sub-categories and rated on a three-tier scale instead of a seven-tier one (completely absent, discretely present, or substantially present). The dataset has already been used in other studies as part of a larger, ongoing project with the goal of providing insights into the complex field of SSD and offending. Although the same database provides the basis for several analyses covering a wide range of objectives in this research area, and although there are a few overlapping parameters, it still contains a substantial number of unique variables, thus resulting in different theoretical and practical conclusions and implications. An overview of the basic characteristics of the population is provided in

Table 1. Sociodemographic characteristics ¹.

Characteristics	Total n/N (%)	No Aggression n/N (%)	Aggression n/N (%)
Male sex	327/352 (92.9)	219/239 (91.6)	108/113 (95.6)
Age at admission (mean, SD)	33.98 (10.206)	34.62 (10.014)	32.64 (10.519)
Native Country Switzerland	156/352 (44.3)	106/239 (44.4)	50/113 (44.2)
Single (at offense)	285/346 (82.4)	188/233 (80.7)	97/113 (85.8)

¹ SD = standard deviation; N = total study population; n = subgroup with characteristic.

.

Parts of the following section were published in advance in a study by Günther et al. [15] and is here partly replicated and extended by the methodology of the current research question. For further information regarding data collection and processing, please refer to previous publications [15,16,17]. Due to the explorative nature of this study, supervised machine learning (ML) seemed to be the optimal approach to identify the most relevant predictive factors out of numerous parameters and to determine the model providing the best predictive power. An overview of the statistical steps can be seen in Figure 1 and is further described below. All the steps were performed using R version 3.6.3. (R Project, Vienna, Austria) and the MLR package v2.171 (Bischl, Munich, Germany). CI calculations of the balanced accuracy were conducted using MATLAB R2019a (MATLAB and Statistics Toolbox Release 2012, The MathWorks, Inc., Natick, Massachusetts, United States) with the add-on “computing the posterior balanced accuracy” v1.0.

All raw data were first processed for machine learning (see Figure 1, Step 1): Several categorical variables were converted to binary code, while continuous and ordinal variables were not adjusted. Due to the retrospective nature of the study and a large number of variables included, there were missing values among variables. This especially applied to information on the broader biographical history of patients, although forensic records were comprehensive. Variables with more than 33% missing values were eliminated, leaving a set of 508 variables. The outcome variable “aggressive behavior during current hospitalization” was dichotomized into (1) “aggressive behavior” and (0) “no aggressive behavior”. Acts of aggression were defined as either verbal or physical attacks aimed toward staff or other patients, as well as damage of property. After the exclusion of all patients with missing information regarding their aggressive behavior from further analysis, a total of 352 patients remained. Out of these patients, 113 (32.1%) were involved in an aggressive event, while 239 (67.9%) were not (see Table 1). “No Aggression” was defined as the positive class, “Aggression” as the negative class.

After the completion of data preparation, the database was divided into one training and one validation subset (see Figure 1, Step 2). The training subset, including 70% of all cases (n = 246), was used for variable reduction and model building/selection. To enable the flexible application of all ML algorithms, imputation of missing values was carried out and imputation weights saved for later were reused on the validation subset (see Figure 1, Step 3a). As the outcome variable was unevenly distributed (12.4:87.6%), a random up-sampling at a rate of 2 was conducted, leading to a more balanced outcome (see Figure 1, Step 3b). A major objective of the present study was to identify the most important predictor variables from 507 possible variables. Additionally, a decrease in variables can counteract overfitting and maintain computing times in initial model building at an acceptable level. For this purpose, variable reduction to the 10 most important predictors was performed using randomForestSRC implemented in the MLR package (see Figure 1, Step 3c). As the database was relatively small for ML purposes and our focus lay on variable extraction and prediction, we applied discriminative model building with logistic regression, trees, random forest, gradient boosting, KNN (k-nearest neighbor), support vector machines (SVM), and as an easily applicable generative model building, naive Bayes (see Figure 1, Step 3d). No hyperparameters were optimized. The model performance of each model was calculated and assessed in terms of its balanced accuracy (the average of true positive and true negative rate, better suited for model evaluation and calculation of confidence intervals in imbalanced data) and goodness of fit (measured with the receiver operating characteristic, balanced curve area under the curve method, ROC balanced AUC). Specificity, sensitivity, positive predictive value (PPV), and negative predictive value (NPV) were also evaluated. As our training dataset was artificially balanced, the model with the highest AUC was chosen for final model validation with the test subset (see Figure 1, Step 3e). The set of identified variables was tested for multicollinearity to avoid dependencies between the variables. Finally, a nested resampling approach was employed, thus preventing the common obstacle of overfitting in ML. This was achieved using a nested resampling model with the inner loop performing imputation, oversampling, variable filtration, and model building within 5-fold cross-validation, and the outer loop for performance evaluation also embedded in 5-fold cross-validation—a technique for artificially creating different subsamples of a dataset (see Figure 1, Step 4).

As a next step, the validation subset, including 30% of all cases (n = 106), was applied to evaluate the statistical model selected before (see Figure 1, Steps 5–7). As briefly mentioned above, the previously stored imputation weights were reused on the validation subset (see Figure 1, Step 5). Then the model selected through the application of the training subset was applied for validation (see Figure 1, Step 6). The identified variables were finally tested for multicollinearity and ranked by their indicative power (see Figure 1, Step 7).

3. Results

3.1. Model Calculation

An overview of the performance parameters of the different calculated models during the nested resampling procedure can be found in

Table 2. Machine learning models and performance in nested cross-validation on training dataset ¹.

Statistical Procedure	Balanced Accuracy (%)	AUC	Sensitivity (%)	Specificity (%)	PPV (%)	NPV (%)
Logistic Regression	74.9	0.85	77.8	72.1	85.7	60.6
Tree	74.7	0.80	72.6	76.8	86.2	56.7
Random Forest	75.3	0.83	74.9	74.9	87.3	59.9
Gradient Boosting
KNN	77.7	0.85	78.6	76.8	88.0	63.1
SVM	77.6	0.87	78.2	66.9	87.3	66.3
Naive Bayes	75.9	0.85	87.9	76.1	87.8	59.8

¹ AUC = area under the curve (level of discrimination); PPV = positive predictive value; NPV = negative predictive value; KNN = k-nearest neighbors; SVM = support vector machines.

. With a balanced accuracy of 77.6% and an AUC of 0.87, the support vector machines (SVM) outperformed all the other ML algorithms (see Table 2).

The absolute and relative distribution of the 10 most predictive variables identified during nested resampling and used for the model buildings can be seen in

Table 3. Absolute and relative distribution of relevant predictor variables ¹.

Variable Code	Variable Description	Aggressive Incidents	No Aggressive Incidents
DZ1	Did the patient complain about the hospital staff?	73/111 (65.8)	45/238 (18.9)
DZ2	Did the patient show negative behavior toward other patients?	76/112 (67.9)	40/237 (16.9)
DZ7	Did the patient show dis/antisocial behavior?	90/111 (81.1)	73/238 (30.7)
DZ10	Did the patient break the rules of the ward (e.g., substance abuse)?	61/112 (54.5)	36/238 (15.1)
R22c (mean, SD)	Time spent at a high-security level during current forensic hospitalization	48.36 (59.65))	33.84 (45.22)
PA_A (mean, SD)	Adapted PANSS at admission: Total score	30.19 (12.34)	22.05 (11.35)
PA7	Adapted PANSS at admission: Hostility
	symptom absent	27/113 (23.9)	160/238 (67.2)
	symptom discreet	22/113 (19.5)	45/238 (18.9)
	symptom substantial	64/113 (56.6)	33/238 (13.9)
PA18	Adapted PANSS at admission: Tension
	symptom absent	25/113 (22.1)	131/238 (55)
	symptom discreet	25/113 (22.1)	54/238 (22.7)
	symptom substantial	63/113 (55.8)	53/238 (22.3)
PA22	Adapted PANSS at admission: Uncooperativeness
	symptom absent	22/113 (19.5)	144/238 (60.5)
	symptom discreet	38/113 (33.6)	58/238 (24.4)
	symptom substantial	53/113 (46.9)	36/238 (15.1)
PA28	Adapted PANSS at admission: Poor impulse control
	symptom absent	17/113 (15)	155/238 (65.1)
	symptom discreet	33/113 (29.2)	40/238 (16.8)
	symptom substantial	63/113 (55.8)	43/238 (18.1)

¹ SD = Standard deviation; PANSS = positive and negative syndrome scale.

. They can be grouped in the following two areas: (1) problematic or antisocial behavior during current hospitalization, and (2) psychopathology. In the initial model including ten variables, the time spent at a high-security level during current forensic hospitalization was the dominant variable. However, the variable was omitted as it was considered circular. Further analysis was, therefore, conducted with the remaining nine most predictive variables.

The quality of the final model in the validation step is shown in

Table 4. Final SVM model performance measures on validation dataset.

Performance Measures	% (95% CI)
Balanced Accuracy	73.5 (64.4–82.1)
AUC	0.84 (0.75–0.93)
Sensitivity	83.5 (83.3–83.8)
Specificity	59.4 (58.8–59.9)
PPV	83.5 (83.2–83.8)
NPV	59.4 (58.8–59.9)

. As expected, the balanced accuracy of 73.5 and the AUC of 0.84 were less than the results of the initial training model but they were still meaningful. With a sensitivity of 84% and a specificity of 59%, the patients involved in aggressive incidents were identified correctly in almost three-quarters of events, while three-thirds of all the non-aggressive patients were identified correctly (see Table 4).

3.2. Determinants of Aggressive Inpatient Behavior

The distribution of the importance of variables of the final validation model is presented in Figure 2 as a one-sided tornado graph. Negative behavior toward other patients was identified as the most indicative factor in distinguishing aggressive and non-aggressive patients, followed by breaking of ward rules, the PANSS score at admission, and poor impulse control as well as hostility, also according to the PANSS. Complaints about hospital staff, dis/antisocial utterances or attitudes, tension, and uncooperativeness were also identified as factors influencing the model.

4. Discussion

The purpose of this study was to identify the factors that distinguish between offender patients with SSD who show aggressive behavior within a hospitalized setting and those who do not. The idea was to exploratively identify the most predictive factors in inpatient violence. By applying ML algorithms to a large database, we were able to create an appropriate model consisting of nine factors. With a balanced accuracy of 73.5 and an AUC of 0.84, the model was able to correctly identify aggressive patients in nearly three-quarters of events and non-aggressive patients in two-thirds of events. The variables related to psychopathology and antisocial behavior proved to be the most predictive regarding inpatient aggression. The aggressive patients in our population showed an increased occurrence of negative behavior toward other patients. Patient/patient interaction is known to trigger aggressive events on general psychiatric wards in about a quarter of cases [16,17,18]. It seems obvious that this factor is all the more important in a forensic psychiatric hospital, where severely ill patients with a high potential for violence come together in a confined space with little opportunity for avoidance. This finding emphasizes the importance of de-escalating skills among staff. Remarkably, in contrast to previous results regarding acute general psychiatric wards, negative behavior toward staff was not identified as one of the ten most predictive parameters [19,20,21,22]. This seems somewhat contradictory to the fact that both failure to comply with the ward rules and complains about hospital staff were highly relevant in distinguishing aggressive from non-aggressive patients: Pointing out or insisting on adherence to ward rules and disciplining noncompliance often causes friction between staff and patients. While such situations arise on a regular basis within a highly institutionalized setting, such as forensic psychiatry, they do not seem to be obligatorily linked to the development of a negative attitude toward the staff and may be tolerated in the presence of a sustainable therapeutic relationship. Poor impulse control, tension, hostility, and uncooperativeness, measured using the corresponding PANSS scales, as well as the overall PANSS score at admission, were also identified as key factors related to inpatient aggression. While the results regarding a lack of impulse control, tension, and hostility are in line with previous findings regarding inpatient aggression in SSD patients as well as aggressive events prior to hospitalization, the link between overall symptomatology represented by the total PANSS score and aggression remains controversial [10,23,24,25]. This suggests that it is not the severity of disease alone that determines the development of aggression, but rather the interplay of the various factors present [23,25]. In summary, the factors that constitute aggression during hospitalization can be reduced to two domains, psychopathology, and antisocial behavior. Interestingly, these two domains outweighed all the other factors, for example, the parameters related to child development, social contacts, and family situation. This was surprising as childhood poverty, for instance, has been previously identified as a risk factor for violent offending [10,26,27]. This is similarly true for comorbid substance abuse, which has been identified as a risk factor for inpatient violence, especially in SSD patients, but did not prove to be of high influence in our population [11,26,27,28,29]. One possible explanation for these phenomena is that the highly structured and closely supervised setting of the forensic psychiatric institution compensates for social and biographical factors, as patients have little exposure to a potentially harmful original social environment (e.g., negative peer group, domestic conflicts, availability of drugs). In addition, it is worth considering that as SSD progresses and becomes more chronic, factors related to psychopathology may become more prominent than they are at an earlier stage of the disease and then outweigh factors with greater influence. As outlined above, in an initial analysis, the time spent at a high-security level during current hospitalization was identified as the most predictive factor outweighing the other variables by far. Since this was a circular argument, the item was omitted. Nevertheless, it should be noted that aggressive behavior in the context of hospitalization leads to a longer length of stay in high-security settings, which has both personal consequences for the patient regarding their rehabilitation and economic consequences for the healthcare system [30,31,32]. When interpreting these findings, the following two hypotheses deserve to be discussed: On the one hand, a conglomerate of SSD and antisocial traits might be present in the patients who display aggression in a highly institutionalized setting. The role of a potential comorbid antisocial personality disorder in SSD patients in the development of aggressive behavior has been extensively discussed in the literature [33,34,35,36]. On the other hand, antisociality may not be an expression of a comorbid personality disorder, but an expression of the underlying SSD. It is well known that positive psychotic symptoms such as hallucinations or threat/control–override symptoms can contribute to violent behavior [37,38,39]. This seems contradictory to the fact that the PANSS, regarding positive symptoms, was not identified as a risk factor in this study and further exploration is needed to distinguish between psychopathology and antisociality. Regarding the limitations of the present study, the most obvious lies in its retrospective design and, therefore, the small possibility of collecting selected parameters in a standardized manner. This poses a particular difficulty for the parameters that are hard to define, such as “antisocial behavior”, since individual assessments of certain events may differ among different professionals. To minimize the possible bias effects and to draw robust causal inferences, a replication of the present findings in a prospective design is recommended. Furthermore, while a sample size of 370 patients is rather large regarding the field of forensic psychiatry, it has to be acknowledged that the dataset is a rather small one regarding medical research in other disciplines using ML algorithms. It is, therefore, recommended to apply the model to others. While the model was able to correctly identify aggressive patients in three-quarters of events, it failed to identify a third of all the non-aggressive patients as such. Therefore, one must be cautious when drawing definitive conclusions from the model, especially so if it is applied to clinical practice, where the label “aggressive patient” may affect a course of treatment, e.g., through a lower threshold for coercive measures. Thus, the question arises as to what value the AUC must assume in order to be considered an acutely acceptable performance measure. This discourse must be conducted intensively, particularly in the sensitive field of forensic psychiatry, since aggressive events occurring in such a setting have far-reaching consequences for the patients concerned (for example, compulsory medication, restrictions on freedom in the form of isolation and restraint as well as prolongation of hospitalization) [40].

5. Conclusions

Our findings expand the current research on factors influencing aggression within forensic inpatient treatment in offender patients with SSD. The present study is a good example of the practical use of artificial intelligence and illustrates that ML is instrumental in analyzing a large dataset and understanding the complex interplay between the factors that contribute to aggressive behavior in SSD. By applying ML, the 9 most predictive variables could be singled out from 507 items, and their interactions could be analyzed in an exploratory manner. A similar analysis with all 507 items would not have been feasible using linear regressions or even multivariate analyses, as the item number exceeds the capacities of those models, and the interplay of variables cannot be explored. In this study, we could show that mental illness and the antisocial behavior associated with it outweighed all the other factors. That these two groups have emerged as predictor domains is encouraging in that they are clinically elicitable using fairly simple methods. Biographical factors such as childhood trauma, on which psychiatrists are often focused when trying to explain aggressive behavior, are, in contrast, rather difficult to assess if the patients are not transparent and are also static, meaning they are perennially present regardless of the patients’ individual development. Of course, these findings do not allow other known risk factors to be disregarded—yet they are outweighed by mental illness and antisocial behavior. The fact that SSD is also highly associated with antisocial behavior emphasizes the importance of early detection and sufficient treatment. The prevention of aggressive behavior toward fellow patients and staff members is a major concern in everyday clinical practice. Above all, but not exclusively, this applies to forensic psychiatric institutions in which a pre-selected group of patients with a particularly high risk of violence is treated. If the predictive risk domains are screened for, a tailor-made treatment approach could be designed for patients with an elevated risk. This may include closer monitoring by staff and case management by well-experienced therapists, who could even proactively develop skills counteracting aggressive impulses at an early stage with high-risk patients before the occurrence of such events.

Based on the present findings, the authors are currently developing a clinical screening tool for problematic inpatient behavior. Its application should enable clinicians to identify high-risk patients at an early stage, modify their treatment accordingly (for example, intensified monitoring), and ultimately prevent aggressive events during hospitalization. However, keeping the ethical implications described above in mind, one has to be mindful of the fact that despite this being a fairly large dataset in the niche subject of forensic psychiatry, these data can, for now, only serve as pilot data and need further application and exploration before a robust tool for detecting those patients with a high risk of aggressive behavior during hospitalization can be developed. In the future, this could not only protect staff and fellow patients from attacks, but also benefit the affected persons themselves, e.g., by reducing the need for coercive measures, shorter hospitalization, and the possible involvement of the judicial system.