2026-09-19T14:41:12Zhttps://dspace.mit.edu/server/oai/request

oai:dspace.mit.edu:1721.1/533082022-01-13T07:54:29Zcom_1721.1_7582com_1721.1_7581col_1721.1_131022

Trevor J. Darrell. Yeh, Pei-Hsiu, 1978- Massachusetts Institute of Technology. Dept. of Electrical Engineering and Computer Science. Massachusetts Institute of Technology. Department of Electrical Engineering and Computer Science 2010-03-25T15:29:18Z 2010-03-25T15:29:18Z 2009 2009 http://hdl.handle.net/1721.1/53308 549462441 Thesis (Ph. D.)--Massachusetts Institute of Technology, Dept. of Electrical Engineering and Computer Science, 2009. Cataloged from PDF version of thesis. Includes bibliographical references (p. 156-166). A picture is worth a thousand words. Images have been used extensively by us to interact with other human beings to solve certain problems, for example, showing an image of a bird to a bird expert to identify its species or giving an image of a cosmetic product to a husband to help purchase the right product. However, images have been rarely used to support similar interactions with computers. In this thesis, I present a series of useful applications for users to interact with computers using images and develop several computer vision algorithms necessary to support such interaction. On the application side, I examine two functional roles of images in human-computer interactions: search and automation. For search, I develop systems for users to obtain useful information about a location or a consumer product by taking its picture using a camera phone, to search online documentation about a GUI by taking its screenshot, and to ask general questions using pictures in a community based QA service. For automation, I design a visual scripting system to allow end-users insert screenshots of GUI elements directly into program statements. (cont.) On the computer vision side, I describe the Adaptive Vocabulary Tree algorithm for indexing and searching a large and dynamic collection of images, the Dynamic Visual Category Learning algorithm for training and updating a set of dynamically changing object categories, the Vocabulary Tree SVM algorithm for fast object recognition by approximating the margins of a set of SVM classifiers efficiently, and the Multiclass Brand-and-Bound Window Search algorithm for simultaneously estimating the optimal location and label of an object in a large input image. Finally, I demonstrate the usability of each proposed application with user studies and the technical performance of each algorithm with series of experiments with large datasets. by Tom Yeh. Ph.D. 166 p. eng Massachusetts Institute of Technology M.I.T. theses are protected by copyright. They may be viewed from this source for any purpose, but reproduction or distribution in any format is prohibited without written permission. See provided URL for inquiries about permission. http://dspace.mit.edu/handle/1721.1/7582 Electrical Engineering and Computer Science. Interacting with computers using images for search and automation Interactive image search for information retrieval and human computer interaction Thesis application/pdf Publication unknown unknown <Publication xmlns="https://www.openaire.eu/cerif-profile/1.1/" id="7b966b7f-89e9-4f79-95c4-64f133d48a34"> <Type xmlns="https://www.openaire.eu/cerif-profile/vocab/COAR_Publication_Types">http://purl.org/coar/resource_type/c_1843</Type> <Language>eng</Language> <Title>Interacting with computers using images for search and automation</Title> <Subtitle>Interactive image search for information retrieval and human computer interaction</Subtitle> <PublishedIn> <Publication> </Publication> </PublishedIn> <PublicationDate>2009</PublicationDate> <Authors> <Author> <DisplayName>Yeh, Pei-Hsiu, 1978-</DisplayName> <Affiliation> <OrgUnit> </OrgUnit> </Affiliation> </Author> </Authors> <Editors> </Editors> <Publishers> <Publisher> <DisplayName>Massachusetts Institute of Technology</DisplayName> <OrgUnit /> </Publisher> </Publishers> <License>http://dspace.mit.edu/handle/1721.1/7582</License> <Keyword>Electrical Engineering and Computer Science.</Keyword> <Abstract>A picture is worth a thousand words. Images have been used extensively by us to interact with other human beings to solve certain problems, for example, showing an image of a bird to a bird expert to identify its species or giving an image of a cosmetic product to a husband to help purchase the right product. However, images have been rarely used to support similar interactions with computers. In this thesis, I present a series of useful applications for users to interact with computers using images and develop several computer vision algorithms necessary to support such interaction. On the application side, I examine two functional roles of images in human-computer interactions: search and automation. For search, I develop systems for users to obtain useful information about a location or a consumer product by taking its picture using a camera phone, to search online documentation about a GUI by taking its screenshot, and to ask general questions using pictures in a community based QA service. For automation, I design a visual scripting system to allow end-users insert screenshots of GUI elements directly into program statements.</Abstract> <Abstract>(cont.) On the computer vision side, I describe the Adaptive Vocabulary Tree algorithm for indexing and searching a large and dynamic collection of images, the Dynamic Visual Category Learning algorithm for training and updating a set of dynamically changing object categories, the Vocabulary Tree SVM algorithm for fast object recognition by approximating the margins of a set of SVM classifiers efficiently, and the Multiclass Brand-and-Bound Window Search algorithm for simultaneously estimating the optimal location and label of an object in a large input image. Finally, I demonstrate the usability of each proposed application with user studies and the technical performance of each algorithm with series of experiments with large datasets.</Abstract> <Access xmlns="http://purl.org/coar/access_right" > </Access> </Publication>