Risk Assessment: An Art or Science? Predicting Recidivism at the Time of Sentencing

Motivated by recent legislation that bases sentencing of criminals on their likelihood to recommit a crime, we developed models that would make a prediction about whether or not individual criminals would commit another crime post-release. Using various supervised learning techniques, we looked at features of relevant criminal, locational, and demographic information to make predictions based on the known outcomes of, in some ways, similar individuals. We also created a ”balanced set” with equal numbers of positive and negative examples and trained and tested on that set. From there, we moved on to a hybrid unsupervised/supervised model. In this approach, we classified the data we had using k-Means Clustering before training our supervised learning algorithms on each data-cluster and looking at the average predictive error between clusters. Finally, we calculated Mutual Information statistics to infer which features were most informative of recidivism in an effort to decompose our hypothesized biases. In the end, we found that in the full data set the supervised learning models on their own could barely perform better than a null-hypothesis that no-one recommits, but when paired with the k-Means clustering, we saw slight but significant improvement in the prediction accuracy.