thesis in the field of Mechanical Engineering: Relevance for Human-Robot Collaboration: Definitions, Systems, Algorithms, and Applications

Abstract

Human-Robot Collaboration (HRC) combines the strengths of human and robotic capabilities to accomplish complex tasks, yielding significant impacts in various domains. To enable seamless interaction in dynamic and unpredictable environments, robots are required to efficiently and accurately perceive their surroundings, align reasoning with human cognition, anticipate key attributes, and generate safe, effective actions to support humans proactively. This thesis introduces relevance, a novel concept inspired by human cognition, to improve the efficiency, safety, and intelligence of HRC. Relevance enables robots to prioritize objects based on their importance to human goals, allowing them to concentrate computational resources on key elements. This focused approach reduces input space for essential algorithms, minimizes processing delays, and enhances safety and adaptability in dynamic environments, facilitating more natural and intuitive collaboration with humans. This thesis systematically explores the concept of relevance, introducing a hierarchical model for relevance quantification that combines scene understanding in cluttered environments with an event-based, multi-modality framework, enabling real-time relevance determination based on human objectives, preferences, spatial-temporal relationships, and constraints. A relevance-based perception strategy further directs models to prioritize key areas, reducing computational and inference times, while two new safety metrics—Critical Collision Probability (CCP) and Average Collision Probability (ACP)—quantify reduced collision risks in Human-Robot Collaboration (HRC). Additionally, a relevance-driven framework integrates relevance quantification with dynamic scene understanding and decision-making, achieving high human objective and relevance prediction accuracy. An advanced human intention prediction framework using head orientation, object affordance, and hand movement also enhances precision, accuracy, and F1 scores over baseline models. Results demonstrate that relevance quantification significantly reduces task planning time by 79.56% and inquiries by 80.84%, with a real-world coffee-serving demonstration highlighting its potential for proactive, autonomous assistance. Furthermore, the safe motion generation algorithm reduces collision incidents by 63.76% and collision frames by 44.74%, supporting accurate, safe robotic assistance in dynamic environments. The concept of relevance enhances the efficiency, safety, and intelligence of human-robot collaboration (HRC) within dynamic and unpredictable environments, supporting a deeper integration of robotics into diverse real-world applications. Its potential extends beyond HRC, with promising applicability in autonomous driving and other complex domains where adaptive, context-aware decision-making is essential.