Advanced Search
DSpace@MIT

Machine learning on Web documents

Research and Teaching Output of the MIT Community

Show simple item record

dc.contributor.advisor David R. Karger. en_US
dc.contributor.author Shih, Lawrence Kai, 1974- en_US
dc.contributor.other Massachusetts Institute of Technology. Dept. of Electrical Engineering and Computer Science. en_US
dc.date.accessioned 2005-09-26T19:49:54Z
dc.date.available 2005-09-26T19:49:54Z
dc.date.copyright 2004 en_US
dc.date.issued 2004 en_US
dc.identifier.uri http://hdl.handle.net/1721.1/28331
dc.description Thesis (Ph. D.)--Massachusetts Institute of Technology, Dept. of Electrical Engineering and Computer Science, 2004. en_US
dc.description Includes bibliographical references (leaves 111-115). en_US
dc.description.abstract The Web is a tremendous source of information: so tremendous that it becomes difficult for human beings to select meaningful information without support. We discuss tools that help people deal with web information, by, for example, blocking advertisements, recommending interesting news, and automatically sorting and compiling documents. We adapt and create machine learning algorithms for use with the Web's distinctive structures: large-scale, noisy, varied data with potentially rich, human-oriented features. We adapt two standard classification algorithms, the slow but powerful support vector machine and the fast but inaccurate Naive Bayes, to make them more effective for the Web. The support vector machine, which cannot currently handle the large amount of Web data potentially available, is sped up by "bundling" the classifier inputs to reduce the input size. The Naive Bayes classifier is improved through a series of three techniques aimed at fixing some of the severe, inaccurate assumptions Naive Bayes makes. Classification can also be improved by exploiting the Web's rich, human-oriented structure, including the visual layout of links on a page and the URL of a document. These "tree-shaped features" are placed in a Bayesian mutation model and learning is accomplished with a fast, online learning algorithm for the model. These new methods are applied to a personalized news recommendation tool, "the Daily You." The results of a 176 person user-study of news preferences indicate that the new Web-centric techniques out-perform classifiers that use traditional text algorithms and features. We also show that our methods produce an automated ad-blocker that performs as well as a hand-coded commercial ad-blocker. en_US
dc.description.statementofresponsibility by Lawrence Kai Shih. en_US
dc.format.extent 115 leaves en_US
dc.format.extent 11940599 bytes
dc.format.extent 11954614 bytes
dc.format.mimetype application/pdf
dc.format.mimetype application/pdf
dc.language.iso en_US
dc.publisher Massachusetts Institute of Technology en_US
dc.rights M.I.T. theses are protected by copyright. They may be viewed from this source for any purpose, but reproduction or distribution in any format is prohibited without written permission. See provided URL for inquiries about permission. en_US
dc.rights.uri http://dspace.mit.edu/handle/1721.1/7582
dc.subject Electrical Engineering and Computer Science. en_US
dc.title Machine learning on Web documents en_US
dc.type Thesis en_US
dc.description.degree Ph.D. en_US
dc.contributor.department Massachusetts Institute of Technology. Dept. of Electrical Engineering and Computer Science. en_US
dc.identifier.oclc 55673199 en_US


Files in this item

Name Size Format Description
55673199-MIT.pdf 11.30Mb PDF Full printable version

This item appears in the following Collection(s)

Show simple item record

MIT-Mirage