GUI Testing Using Computer Vision
Name
Miller_GUI testing.pdf
Size
3.64 MB
Format
Adobe PDF
Checksum (MD5)
3067894c67a47367f9f1fad99c759e93
Author(s)
Chang, Tsung-Hsiang
Yeh, Tom
Miller, Robert C.
Date Issued
April 2010
Journal
Proceedings of the 28th International Conference on Human Factors in Computing Systems (CHI '10)
Publisher
Association for Computing Machinery (ACM)
Citation
Tsung-Hsiang Chang, Tom Yeh, and Robert C. Miller. 2010. GUI testing using computer vision. In Proceedings of the 28th international conference on Human factors in computing systems (CHI '10). ACM, New York, NY, USA, 1535-1544.
Version
Author's final manuscript
Abstract
Testing a GUI's visual behavior typically requires human testers to interact with the GUI and to observe whether the expected results of interaction are presented. This paper presents a new approach to GUI testing using computer vision for testers to automate their tasks. Testers can write a visual test script that uses images to specify which GUI components to interact with and what visual feedback to be observed. Testers can also generate visual test scripts by demonstration. By recording both input events and screen images, it is possible to extract the images of components interacted with and the visual feedback seen by the demonstrator, and generate a visual test script automatically. We show that a variety of GUI behavior can be tested using this approach. Also, we show how this approach can facilitate good testing practices such as unit testing, regression testing, and test-driven development.
MIT Department
Massachusetts Institute of Technology. Computer Science and Artificial Intelligence Laboratory
Massachusetts Institute of Technology. Department of Electrical Engineering and Computer Science
Terms of Use
Creative Commons Attribution-Noncommercial-Share Alike 3.0
Persistent DSpace Link
DOI of Published Version
https://doi.org/10.1145/1753326.1753555
