2026-09-19T14:24:44Zhttps://dspace.mit.edu/server/oai/requestoai:dspace.mit.edu:1721.1/1032032026-06-16T18:51:51Zcom_1721.1_7582com_1721.1_7581col_1721.1_131022
Edward H. Adelson.
Isola, Phillip (Phillip John)
Massachusetts Institute of Technology. Department of Brain and Cognitive Sciences.
Massachusetts Institute of Technology. Department of Brain and Cognitive Sciences
2016-06-22T17:46:43Z
2016-06-22T17:46:43Z
2015
2015
http://hdl.handle.net/1721.1/103203
951465027
Thesis: Ph. D., Massachusetts Institute of Technology, Department of Brain and Cognitive Sciences, 2015.
Cataloged from PDF version of thesis.
Includes bibliographical references (pages 83-92).
Although impressionists assure us that the world is just dabs of light, we cannot help but see surfaces and contours, objects and events. How can a visual system learn to organize pixels into these higher-level structures? In this thesis I argue that perceptual organization reflects statistical regularities in the environment. When visual primitives occur together much more often than one would expect by chance, we may learn to associate those primitives and to form a perceptual group. The first half of the thesis deals with the identification of such groups at the pixel level. I show that low-level image statistics are surprisingly effective at higher-level segmentation. I present an algorithm that groups pixels by identifying meaningful co-occurrences in an image's color statistics. Consider a zebra. Black-next-to-white occurs suspiciously often, hinting that these colors have a common cause. I model these co-occurrences using pointwise mutual information (PMI). If the PMI between two colors is high, then the colors probably belong to the same object. Grouping pixels with high PMI reveals object segments. Separating pixels with low PMI marks perceived boundaries. If simple color co-occurrences can tell us about object segments, what might more complex statistics tell us? The second half of the thesis investigates high dimensional visual data, such as image patches and video frames. In high dimensions, it is intractable to directly model co-occurrences. Instead, I show that modeling PMI can be posed as a simpler binary classification problem in which the goal is to predict if two primitives occur in the same spatial or temporal context. This allows us to model PMI associations between complex inputs. I demonstrate the effectiveness of this approach on three domains: discovering objects by associating image patches, discovering movie scenes by associating frames, and discovering place categories by associating geotagged photos. Together, these results shed light on how a visual system can learn to organize raw sensory input into meaningful percepts.
by Phillip Isola.
Ph.D.
92 pages
eng
Massachusetts Institute of Technology
M.I.T. theses are protected by copyright. They may be viewed from this source for any purpose, but reproduction or distribution in any format is prohibited without written permission. See provided URL for inquiries about permission.
http://dspace.mit.edu/handle/1721.1/7582
Brain and Cognitive Sciences.
The Discovery of perceptual structure from visual co-occurrences in space and time
Thesis
application/pdf
false
Publication
unknown
unknown
<Publication xmlns="https://www.openaire.eu/cerif-profile/1.1/" id="06a84b3c-b8d9-4e72-9e17-fce0de60dd59">
<Type xmlns="https://www.openaire.eu/cerif-profile/vocab/COAR_Publication_Types">http://purl.org/coar/resource_type/c_1843</Type>
<Language>eng</Language>
<Title>The Discovery of perceptual structure from visual co-occurrences in space and time</Title>
<PublishedIn>
<Publication>
</Publication>
</PublishedIn>
<PublicationDate>2015</PublicationDate>
<Authors>
<Author>
<DisplayName>Isola, Phillip (Phillip John)</DisplayName>
<Affiliation>
<OrgUnit>
</OrgUnit>
</Affiliation>
</Author>
</Authors>
<Editors>
</Editors>
<Publishers>
<Publisher>
<DisplayName>Massachusetts Institute of Technology</DisplayName>
<OrgUnit />
</Publisher>
</Publishers>
<License>http://dspace.mit.edu/handle/1721.1/7582</License>
<Keyword>Brain and Cognitive Sciences.</Keyword>
<Abstract>Although impressionists assure us that the world is just dabs of light, we cannot help but see surfaces and contours, objects and events. How can a visual system learn to organize pixels into these higher-level structures? In this thesis I argue that perceptual organization reflects statistical regularities in the environment. When visual primitives occur together much more often than one would expect by chance, we may learn to associate those primitives and to form a perceptual group. The first half of the thesis deals with the identification of such groups at the pixel level. I show that low-level image statistics are surprisingly effective at higher-level segmentation. I present an algorithm that groups pixels by identifying meaningful co-occurrences in an image's color statistics. Consider a zebra. Black-next-to-white occurs suspiciously often, hinting that these colors have a common cause. I model these co-occurrences using pointwise mutual information (PMI). If the PMI between two colors is high, then the colors probably belong to the same object. Grouping pixels with high PMI reveals object segments. Separating pixels with low PMI marks perceived boundaries. If simple color co-occurrences can tell us about object segments, what might more complex statistics tell us? The second half of the thesis investigates high dimensional visual data, such as image patches and video frames. In high dimensions, it is intractable to directly model co-occurrences. Instead, I show that modeling PMI can be posed as a simpler binary classification problem in which the goal is to predict if two primitives occur in the same spatial or temporal context. This allows us to model PMI associations between complex inputs. I demonstrate the effectiveness of this approach on three domains: discovering objects by associating image patches, discovering movie scenes by associating frames, and discovering place categories by associating geotagged photos. Together, these results shed light on how a visual system can learn to organize raw sensory input into meaningful percepts.</Abstract>
<Access xmlns="http://purl.org/coar/access_right"
>
</Access>
</Publication>