From Variable Importance to Economic Scenarios: Interpreting Machine Learning Models for Firm Failure Risk Management

Firm failure risk is an important topic for managers, credit institutions, investors and policymakers. In recent years, machine learning models have been increasingly used in this field. Although they often achieve high predictive performance, they typically lack interpretability. Existing literature on machine learning model interpretability focuses on individual variable importance with limited practical guidance for firm management. We use a large firm-level dataset of Romanian companies (360.249 observations) to train various XGBoost models and to examine their interpretability. We use a wide range of explanatory variables: firm size, industry classification, financial statement items scaled by total assets and 44 financial ratios. We use AUROC as performance measure, while firm failure is proxied by the transition from positive to negative equity. We calculate variable importance using the permutation feature importance method. Our results show that variable importance in high-dimensional settings can be unstable, as there are considerable redundancies and compensation effects between explanatory variables. Furthermore, individual variable changes have limited effects on predicted probabilities of failure, hence having little practical relevance. We illustrate that using coherent scenario-based modifications to financial statements has substantially larger and economically meaningful effects on predicted failure risk. Compared to variable importance ranking and score assignment, scenario-based analysis provides clearer interpretations and more relevant insights for management. Our contribution aims to shift the focus from individual variable importance to economically meaningful insights derived from machine learning models.