Michael's Wiki

Naive Bayes Models

$p(author | document) = \frac{p(author)p(document | author)}{p(document)}$

$p(document | author) = \prod_{features} p(feature | author) $

Multinomial Model

Data: $\{x_1, x_2, \dots ,x_n\} \in \mathbb{Z}$ $x_i$ = # count of feature i in the document Vocabulary is $V$. Document length is $|D|$.

Learning: $p(author) =$ (# test documents of author)/(# test documents total)

$p(f_i | author) \sim Multinomial(n=1,\theta_i)$

$\theta_i$ = (# $f_i$ in test documents of author)/(# features in test documents of author)

  • * $\sum_{i = 1}^{V} \theta_i = 1$

$p(doc=\bar{x} | author) \sim Multinomial(doc | author, n=|doc|,\theta) * Laplace(\theta | \mu, b) $