Imbalance-Aware Bayesian Multinomial Logistic Regression Via Adaptive Entropy-Based Observation Weighting

Class imbalance poses a fundamental challenge in multi-class classification, where standard maximum-likelihood methods systematically favor the majority classes. We develop an Imbalance-Aware Bayesian Multinomial Logistic Regression (IA-BMLR) model that adapts a weighted loss framework, originally developed for variational autoencoders, to Bayesian discriminative classification. The weighting approach combines inverse-frequency class weights with entropy-based weights derived from a Local Entropy Score (LES) that emphasizes observations in ambiguous, high-entropy regions. We treat the entropy weight parameter as a learnable variable with a half-normal prior, rather than a fixed parameter, thereby enabling data-driven calibration via Bayesian inference. We evaluate IA-BMLR on nine benchmark datasets that span severe to mild class imbalance. On the Yeast (ten classes, imbalance ratio = 92.6) and the Lymphography (four classes, imbalance ratio = 40.5) datasets, IA-BMLR achieves a G-Mean of 0.302 and 0.456, respectively. In contrast, all baseline methods, including XGBoost, Random Forest, SVC, Multinomial logistic regression, and unweighted Bayesian Multinomial logistic regression, achieve an exact zero, indicating prediction failure for at least one minority class. IA-BMLR achieves the highest G-Mean across three of nine datasets, demonstrating its reliability in minority-class prediction while maintaining competitive performance on standard metrics.