Visual urban sensing : understanding cities through computer vision

Abstract

This thesis introduces computer vision algorithms that harness street-level imagery to conduct automated surveys of the built environment and populations at an unprecedented resolution and scale. We introduce new tools for computing quantitative measures of urban appearance and urban change. First, we describe Streetscore, an algorithm that quantifies how safe a street block looks to a human observer, using computer vision and crowdsourcing. We extend this work with an efficient convolutional neural network-based method that is capable of computing several perceptual attributes of the built environment from thousands of cities from all six inhabited continents. Second, we introduce a computer vision algorithm to compute Streetchange-a metric for change in the built environment-from time-series street-level imagery. A positive Streetchange is indicative of urban growth; while negative Streetchange is indicative of decay. We use these tools to introduce new datasets. We use the Streetscore algorithm to generate the largest dataset of urban appearance to date, which covers more than 1 million street blocks from 21 American cities. We use the Streetchange algorithm to also generate a dataset for urban change containing more than 1.5 million street blocks from five large American cities. These datasets have enabled research studies across fields such as economics, sociology, architecture, urban planning, and public health. We utilize these datasets to provide new insights on important research questions. With the dataset on urban appearance, we show that criminal activity has a robust positive correlation with the spatial variation in architecture within neighborhoods. With the dataset on urban change, we show that positive urban change occurs in geographically and physically attractive areas with dense, highly-educated populations. Taken together, the tools, datasets, and insights described in this thesis demonstrate that computer vision-driven surveys of people and places have the potential to massively scale up studies in social science, to change the way cities are built, and to improve the design, execution, and evaluation of policy and aid interventions.