P(y) P(sun|y) P(cool|y) P(high|y) P(strong|y) = .005 P(n) P(sun|n) P(cool|n) P(high|n) P(strong|n) = .021 So, vNB = n
Nave Bayes: Subtleties
2. What if none of the training instances with target value vj have attribute ai P(ai|vj) = 0, and… P(vj) #i P(ai |vj) = 0 Solution is Bayesian estimate: P(ai|vj) = (nc + mp)/(n + m) where

Learning to Classify Text
Why Learn which news articles are of interest Learn to classify web pages by topic Nave Bayes is among most effective algorithms What attributes shall we use to represent text documents
n is number of training examples for which v = vj, nc number of examples for which v = vj and a = ai p is prior estimate for P(ai|vj) m is weight given to prior (i.e., number of "virtual" examples)
Learning to Classify Text
Target concept Interesting : Document ! { +, - } 1. Represent each document by vector of words one attribute per word position in document 2. Learning: Use training examples to estimate P(+) P(-) P(doc| +) P(doc| -)
Nave Bayes for Text
Nave Bayes conditional independence assumption P(doc |vj) = #i=1length(doc) P(ai =wk|vj) where P(ai =wk|vj) is probability that word in position i is wk, given vj One more assumption: P(ai =wk|vj) = P(am =wk|vj) $i, m "Bag of words" assumption.
Learning Algorithm
LEARN_NAVE_BAYES_TEXT(Examples, V ) 1. collect all words and other tokens that occur in Examples Vocabulary " all distinct words and other tokens in Examples 2. Calculate the required P(vj) and P(wk|vj) probability terms For each target value vj in V do
Classification Algorithm
CLASSIFY_NAVE_BAYES_TEXT (Doc) positions " all word positions in Doc that contain tokens found in Vocabulary Return vNB, where
– docsj " subset of Examples for which the target value is vj – P(vj) " |docsj|/|Examples| – Textj " a single document created by concatenating all members of docsj – n " total number of words in Textj (counting duplicate words multiple times) ("tokens" vs. "tokens") – for each word wk in Vocabulary

上一页下一页