Automating website profiling for a deep web search engine
Name
525231644-MIT.pdf
Description
Full printable version
Size
24.91 MB
Format
Adobe PDF
Checksum (MD5)
d35a2afa0e16a2bbe6b6c2b73d204cbb
Author(s)
Yuan, Jeffrey W. (Jeffrey Weijie)
Advisor(s)
Michael Stonebraker.
Date Issued
2009
Publisher
Massachusetts Institute of Technology
Abstract
The deep web consists of information on the internet that resides in databases or is dynamically generated. It is believed that the deep web represents a large percentage of the total contents on the web, but is currently not indexed by traditional search engines. The Morpheus project is designed to solve this problem by making information in the deep web searchable. This requires a large repository of content sources to be built up, where each source is represented in Morpheus by a profile or wrapper. This research proposes an approach to automating the creation of wrappers by relying on the average internet user to identify relevant sites. A wrapper generation system was created based on this approach. It comprises two components: the clickstream recorder saves characteristic data for websites identified by users, and the wrapper constructor converts these data into wrappers for the Morpheus system. After completing the implementation of this system, user tests were conducted, which verified that the system is effective and has good usability.
Description
Thesis (M. Eng.)--Massachusetts Institute of Technology, Dept. of Electrical Engineering and Computer Science, 2009.
Cataloged from PDF version of thesis.
Includes bibliographical references (p. 53-55).
Subjects
Electrical Engineering and Computer Science.
MIT Department
Massachusetts Institute of Technology. Department of Electrical Engineering and Computer Science
Terms of Use
M.I.T. theses are protected by
copyright. They may be viewed from this source for any purpose, but
reproduction or distribution in any format is prohibited without written
permission. See provided URL for inquiries about permission.
copyright. They may be viewed from this source for any purpose, but
reproduction or distribution in any format is prohibited without written
permission. See provided URL for inquiries about permission.
Persistent DSpace Link
