In machine learning, naive Bayes classifiers are a family of simple “probabilistic classifiers” based on applying Bayes’ theorem with strong (naive) independence assumptions between the features. (from wikipedia) Multinomial naive bayes is one of classic naive bayes variants mostly used in document classfication. Each sample represents by the feature vector in which certain events have been counted in multinomial model.
The two parts of multinomial navie bayes algorithm including training and prediction is given as below:
1. Training Stage: count the number of times word term or feature appears across all the training samples, Nyi. And also the number of distinct features in all sample as well as the total number of all features Nyin certain class is counted, respectively. The maximum likelihood probability is the ratio of Nyito Ny, which is one matrix. The prior probability is the ratio of the number of each class to the total number of sample, which is one vector. Additive Laplace smoothing would be used in real applicaton for features not present in the training sample so that zero probabilities can be prevented in the furture classfilier.
- Prediction Stage: for each test sample, the argmax function of the likelihood probability matrix multiply by its feature vector as well as the prior probability vector will get its classification result. And the matrix multiplication can be transformed into matrix addition by the logarithm pre-process.