Interacting with computers using images for search and automation
Name
549462441-MIT.pdf
Description
Full printable version
Size
33.52 MB
Format
Adobe PDF
Checksum (MD5)
911c2d8d0af8d9380e113b14a169451e
Author(s)
Yeh, Pei-Hsiu, 1978-
Advisor(s)
Trevor J. Darrell.
Alternative Title
Interactive image search for information retrieval and human computer interaction
Date Issued
2009
Publisher
Massachusetts Institute of Technology
Abstract
A picture is worth a thousand words. Images have been used extensively by us to interact with other human beings to solve certain problems, for example, showing an image of a bird to a bird expert to identify its species or giving an image of a cosmetic product to a husband to help purchase the right product. However, images have been rarely used to support similar interactions with computers. In this thesis, I present a series of useful applications for users to interact with computers using images and develop several computer vision algorithms necessary to support such interaction. On the application side, I examine two functional roles of images in human-computer interactions: search and automation. For search, I develop systems for users to obtain useful information about a location or a consumer product by taking its picture using a camera phone, to search online documentation about a GUI by taking its screenshot, and to ask general questions using pictures in a community based QA service. For automation, I design a visual scripting system to allow end-users insert screenshots of GUI elements directly into program statements.
(cont.) On the computer vision side, I describe the Adaptive Vocabulary Tree algorithm for indexing and searching a large and dynamic collection of images, the Dynamic Visual Category Learning algorithm for training and updating a set of dynamically changing object categories, the Vocabulary Tree SVM algorithm for fast object recognition by approximating the margins of a set of SVM classifiers efficiently, and the Multiclass Brand-and-Bound Window Search algorithm for simultaneously estimating the optimal location and label of an object in a large input image. Finally, I demonstrate the usability of each proposed application with user studies and the technical performance of each algorithm with series of experiments with large datasets.
Description
Thesis (Ph. D.)--Massachusetts Institute of Technology, Dept. of Electrical Engineering and Computer Science, 2009.
Cataloged from PDF version of thesis.
Includes bibliographical references (p. 156-166).
Subjects
Electrical Engineering and Computer Science.
MIT Department
Massachusetts Institute of Technology. Department of Electrical Engineering and Computer Science
Terms of Use
M.I.T. theses are protected by
copyright. They may be viewed from this source for any purpose, but
reproduction or distribution in any format is prohibited without written
permission. See provided URL for inquiries about permission.
copyright. They may be viewed from this source for any purpose, but
reproduction or distribution in any format is prohibited without written
permission. See provided URL for inquiries about permission.
Persistent DSpace Link
