MIT Libraries homeMIT Libraries logoDSpace@MIT

MIT
View Item 
  • DSpace@MIT Home
  • MIT Libraries
  • MIT Theses
  • Theses - Dept. of Electrical Engineering and Computer Sciences
  • Electrical Engineering and Computer Sciences - Ph.D. / Sc.D.
  • View Item
  • DSpace@MIT Home
  • MIT Libraries
  • MIT Theses
  • Theses - Dept. of Electrical Engineering and Computer Sciences
  • Electrical Engineering and Computer Sciences - Ph.D. / Sc.D.
  • View Item
JavaScript is disabled for your browser. Some features of this site may not work without it.

Using graphical representation of user interfaces as visual references

Author(s)
Zhang, Zongxiang
Thumbnail
DownloadFull printable version (17.66Mb)
Other Contributors
Massachusetts Institute of Technology. Dept. of Electrical Engineering and Computer Science.
Advisor
Rob Miller.
Terms of use
M.I.T. theses are protected by copyright. They may be viewed from this source for any purpose, but reproduction or distribution in any format is prohibited without written permission. See provided URL for inquiries about permission. http://dspace.mit.edu/handle/1721.1/7582
Metadata
Show full item record
Abstract
My thesis investigates using a graphical representation of user interfaces - screenshots - as a direct visual reference to support various kinds of applications. We have built several systems to demonstrate and validate this idea in domains like searching documentation, GUI automation and testing, and cross-device information migration. In particular, Sikuli Search enables users to search documentation using screenshots of GUI elements instead of keywords. Sikuli Script enables users to programmatically control GUIs without support from the underlying applications. Sikuli Test lets GUI developers and testers create test scripts without coding. Deep Shot introduces a framework and interaction techniques to migrate work states across heterogeneous devices in one action, taking a picture. We also discuss challenges inherent in screenshot-based interactions and propose potential solutions and directions of future research.
Description
Thesis (Ph. D.)--Massachusetts Institute of Technology, Dept. of Electrical Engineering and Computer Science, 2012.
 
Cataloged from PDF version of thesis.
 
Includes bibliographical references (p. 129-133).
 
Date issued
2012
URI
http://hdl.handle.net/1721.1/75630
Department
Massachusetts Institute of Technology. Dept. of Electrical Engineering and Computer Science.
Publisher
Massachusetts Institute of Technology
Keywords
Electrical Engineering and Computer Science.

Collections
  • Electrical Engineering and Computer Sciences - Ph.D. / Sc.D.
  • Electrical Engineering and Computer Sciences - Ph.D. / Sc.D.

Browse

All of DSpaceCommunities & CollectionsBy Issue DateAuthorsTitlesSubjectsThis CollectionBy Issue DateAuthorsTitlesSubjects

My Account

Login

Statistics

OA StatisticsStatistics by CountryStatistics by Department
MIT Libraries homeMIT Libraries logo

Find us on

Twitter Facebook Instagram YouTube RSS

MIT Libraries navigation

SearchHours & locationsBorrow & requestResearch supportAbout us
PrivacyPermissionsAccessibility
MIT
Massachusetts Institute of Technology
Content created by the MIT Libraries, CC BY-NC unless otherwise noted. Notify us about copyright concerns.