The reality editor : an open and universal tool for understanding and controlling the physical world

Abstract

In a future where every physical object has the ability to compute and connect with other physical things, we have to rethink our present user interfaces and interaction metaphors. The desktop metaphor used in personal computers and smartphones was invented for data organization and is not well suited for interaction with things in the physical world. As a result, the growing number of interconnected things (or Internet of Things devices) surrounding us are becoming hard to operate. Each IoT device requires a different app to control it and forces the user to memorize a unique connection and interface. In addition, connected things made by different companies cannot easily be connected to one another. This thesis introduces a novel, directly mapped user interface for connected things built upon World Wide Web technology, a decentralized networking infrastructure for connected things to talk to each other, and a simple, visual user interface for understanding and controlling the connected things around us. The overall system is called the Reality Editor, an open-source, freely and publicly available tool for creating ecosystems of interconnected things. The thesis discusses the design of the Reality Editor, its core ideas and implementation details and a series of real world prototypes that were built to evaluate and improve the tool.