Application of XGBoost algorithm in Social Adaptation of Prisoners

Prisoners, as marginalized individuals due to their involvement in illegal activities, undergo educational reform and correction within the prison system, as their behavior itself represents a failure of socialization. [1] However, following their socialization failure, prisoners often face increased societal discrimination. Cognitive transformation theory identified four main components of desistance or types of cognitive openness to change, exposure and reaction to ‘hooks of change’, the replacement of self and the transformation of the ex-offender's views regarding the deviant behaviour. Improving the social adaptability of criminals, reducing the possibility of them engaging in antisocial behaviors such as illegal and criminal activities again after being released from prison, and becoming law-abiding citizens have always been the core contents of correctional work for prison criminals. In order to find more scientific and effective methods to enhance the social adaptability of criminals, a large amount of research and criminal correction practices at home and abroad have focused on the influencing factors and mechanisms of criminal social adaptability [2]. Based on social comparison theory and symbolic interaction theory, the hope is the important contributor for improving social adaption [3]. Interpersonal trust centers around individual trust trait as its foundation, with the dynamic equilibrium between trust risk and trust expectation acting as intermediaries, ultimately resulting in trust behavior and improving social adaption [4].

Conventional psychological research techniques, such as survey-based approaches, aim to uncover underlying psychological principles and analyze the interconnections among variables using structural equation modeling[5]. Nonetheless, these methods have limitations in precisely measuring the extent of influence among different factors and accurately predicting outcomes. Conversely, artificial intelligence algorithms can construct algorithmic models based on the actual data, enhancing predictive accuracy by employing diverse algorithm types and iterative techniques. By identifying the optimal algorithm model, researchers can quantify the impact of each variable, comprehend its significance in the research context, and achieve more timely and effective forecasting and adjustment of prisoners’ social adaption. Among various artificial intelligence algorithms, XGBoost (eXtreme Gradient Boosting) has gained popularity and achieved top positions in Kaggle competitions.

Additionally, it is noteworthy that this paper has garnered over 3500 citations (according to Google Scholar, March 2, 2024) [6]. Decision trees are employed as base learners, and during each iteration, the error calculated is utilized to rectify the preceding predictor (learner), while the change in model performance is assessed using the objective function. However, in elaborating on the principles of gradient boosting, the XGBoost objective function, as defined in Eq. 1), incorporates additional regularization to mitigate overfitting, a prevalent issue in ensemble models. [7]

Nevertheless, as Eq. 1) comprises functions as parameters, traditional optimization methods are inadequate for optimizing the objective function. Instead, the model must be progressively trained, necessitating the utilization of the second-order Taylor approximation. To employ the Taylor approximation, the objective function had to be reformulated as Eq. 2).

In the XGBoost algorithm, with the previous classifiers locking a new weak classifier is added at each iteration to make the performance of the current model better. This process continues, with each new classifier considering areas where the previous ones were not performing well. The general flow of the XGBoost algorithm is illustrated in figure 1.

XGBoost enhances the performance of the current model by fitting an additional weak classifier, without modifying the previous classifier. This iterative process continues, with each new classifier addressing areas where the previous classifiers exhibited suboptimal performance. Figure 1 illustrates the general flow of the Boosting algorithm. Extensive testing and validation have demonstrated XGBoost's exceptional performance and accuracy in various real-world applications. For example, in sentiment analysis [8], which involves analyzing and understanding emotions, opinions, and attitudes expressed in text data, XGBoost has been successfully employed. It effectively classifies and analyzes sentiment, enabling a deeper comprehension of people's reactions and opinions.

Moreover, XGBoost has proven valuable in depression prediction [9] and student performance analysis [10]. It aids in the early identification and prediction of depressive symptoms or disorders. Through its advanced algorithms and ensemble learning techniques, XGBoost can analyze multiple factors and indicators to provide accurate predictions of an individual's risk or likelihood of experiencing depression. These practical implementations highlight the robustness and versatility of XGBoost as a powerful tool in diverse domains. It demonstrates the ability to tackle complex problems and deliver reliable results.

In this study, the demographic variables include age, marriage, children, parent status, education, occupation, work years, position level, type of crime, length of sentence monthly income, time served in prison and number of previous incarcerations. In the original algorithm model, data were converted into text feature and numerical variables, and the process was illustrated in figure 2.

For the algorithm model, we employed the grid search method to identify the optimal hyperparameter values. In the grid search, considering the 66 independent variables, we determined that the maximum depth (max_depth) should not exceed the number of independent variables. Therefore, the optimal value for max_depth was found to be 5. After analyzing the parameter values in the grid search, we identified the optimal solution with a learning rate of 0.01, eta of 0.001, and n_estimators of 400. This indicates that the model utilized shallow depth and assigned small weights to each decision tree, while employing a strategy of fitting the data with a greater number of decision trees. To prevent overfitting, we set the model's subsample at 0.7, colsample_bytree at 0.8, and min_child_weight as 3, as these values were determined to be optimal.

Among the questionnaire items, the item “I believe that I can successfully integrate into society after being released from prison” had an importance of 14% in the feature importance. It is hoped that by alleviating the negative impacts of adverse events and promoting the generation of positive psychological and behavioral responses, the level of psychological adaptation can be enhanced [11]. The item “I feel that I am being looked down upon by others” had an importance of 14% in the feature importance. Discrimination perception widens the psychological distance between individuals and others, leading to feelings of loneliness and interpersonal alienation, which then breed more problems through internalization and externalization [12].

In this study, demographic variables and questionnaire items were used as independent variables to train an XGBoost model for predicting prisoners’ social adaptation, achieving an accuracy of 96.15% in the three-class classification. This indicates the effectiveness of the XGBoost algorithm in studying this problem. Subsequent researchers can further optimize the algorithm to enable more profound conclusions regarding this issue.