Selective naive Bayes predictor with mixtures of truncated exponentials

Abstract

Naive Bayes models have been successfully used in classification problems where the class variable is discrete. Naive Bayes models have been applied to regression or prediction problems, i.e. classification problems with continuous class, but usually under the assumption that the joint distribution of the feature variables and the class is multivariate Gaussian. In this paper we are interested in regres- sion problems where some of the feature variables are discrete while the others are continuous. We propose a Naive Bayes predictor based on the approximation of the joint distribution by a Mixture of Truncated Exponentials (MTE). We have designed a procedure for selecting the variables that should be used in the construction of the model. This scheme is based on the mutual information between each of the candidate variables and the class. Since the mutual information can not be computed exactly for the MTE distribution, we introduce an unbiased estimator of it, based on Monte Carlo methods. We test the performance of the proposed model in three real life problems, related to higher education management.