Clustered Naive Bayes

Abstract

Humans effortlessly use experience from related tasks to improve their performance at novel tasks. In machine learning, we are often confronted with data from "related" tasks and asked to make predictions for a new task. How can we use the related data to make the best prediction possible? In this thesis, I present the Clustered Naive Bayes classifier, a hierarchical extension of the classic Naive Bayes classifier that ties several distinct Naive Bayes classifiers by placing a Dirichlet Process prior over their parameters. A priori, the model assumes that there exists a partitioning of the data sets such that, within each subset, the data sets are identically distributed. I evaluate the resulting model in a meeting domain, developing a system that automatically responds to meeting requests, partially taking on the responsibilities of a human office assistant. The system decides, based on a learned model of the user's behavior, whether to accept or reject the request on his or her behalf. The extended model outperforms the standard Naive Bayes model by using data from other users to influence its predictions.