Ubiquitous Metadata: Design and Fabrication of Embedded Markers for Real-World Object Identification and Interaction

Abstract

The convergence of the physical and digital realms has ushered in a new era of immersive experiences and seamless interactions. As the boundaries between the real world and virtual environments blur and result in a "mixed reality," there arises a need for robust and efficient methods to connect physical objects with their virtual counterparts. In this thesis, we present a novel approach to bridging this gap through the design, fabrication, and detection of embedded machine-readable markers.

The vision of mixed reality relies on mobile and wearable devices being aware of the surroundings to enhance real-world experiences with contextual information. For individual object identification, machine-readable tags such as barcodes and RFID labels are typically used. Barcodes, though cost-effective, tend to be obtrusive, less durable, and less secure than RFID labels. Regardless of their type, such tags are usually added to objects after fabrication, rather than being integrated into the original design.

This thesis attempts to overcome the shortcomings of traditional post-hoc augmentation processes by proposing novel tagging approaches that extract hidden, integrated features of objects and employ them as machine-detectable markers. Our research focuses on the design, implementation, and evaluation of comprehensive systems for embedding and interacting with embedded markers. We categorize the proposed tagging approaches into three distinct categories: natural markers, structural markers, and internal markers. Natural markers, such as those used in SensiCut, are inherent fingerprints of objects repurposed as machine-readable identifiers, while structural markers, such as StructCode and G-ID, leverage the structural artifacts in objects that emerge during the fabrication process itself. Internal markers, such as InfraredTag and BrightMarker, are embedded inside fabricated objects using specialized materials. Leveraging a combination of methods from computer vision, machine learning, computational imaging, and material science, the presented approaches offer versatile solutions for object identification, tracking, and interaction.

These markers, seamlessly integrated into real-world objects, effectively communicate an object’s identity, origin, function, and interaction, functioning as gateways to "ubiquitous metadata" – a concept where metadata is embedded into physical objects, similar to metadata in digital files. Across the different chapters, we demonstrate the applications of the presented methods in diverse domains, including product design, manufacturing, retail, logistics, education, entertainment, security, and sustainability. Furthermore, we discuss the challenges and opportunities in deploying embedded machine-readable markers at scale and in consumer products, enabling novel interactions in AR, and addressing privacy issues.

In conclusion, this thesis presents a comprehensive exploration of embedded machine-readable markers as a means to connect real-world objects to virtual worlds to achieve ubiquitous metadata. The thesis forges ahead with a future where objects come alive, environments become interactive, and virtual worlds seamlessly merge with our everyday lives.