Designing Learner-centric Tools for Physical Skill-learning

Abstract

Learning physical or psychomotor skills such as throwing a ball or operating a drill is integral to our lives. Whether for excelling in sports, mastering DIY tools, or gaining expertise in operating machinery, we are constantly learning to perfect physical skills, which are often taught by instructors and sometimes self-learned. The landscape of how we learn such skills, however, is undergoing a profound transformation with watershed advances in technology. For example, ubiquitous wearables that allow precision sensing, integrated machine learning models that enable personalized predictions, and augmented and virtual reality environments that promise immersive experiences are all revolutionizing how researchers design tools for personalized skill learning. Furthermore, these enabling technologies which offer self-learning opportunities outside the traditional classroom settings, when combined with generative artificial intelligence tools can pave the way for a future where every learner can self-learn through personalized learning experiences.

Amidst this excitement of innovation in enabling technologies, it is crucial, however, to design learning tools that are grounded in learner’s experiences and cater to the diverse nature of human learning. Each learner is unique in their skill levels, learning speeds, and preferences. Moreover, every learning experience is multifaceted, exceeding beyond mere skill acquisition, and encompassing broader facets like cultivating self-motivation, self-efficacy, and creativity. Given this multifaceted nature of human learning, my vision is to leverage technology to design learnercentric tools for comprehensive skill learning.

I fulfill this vision through my doctoral research which lies at the intersection of HumanComputer Interaction (HCI) and Learning Science (LS). In this work, I have combined HCI principles of tool design with LS frameworks to personalize the learning of physical skills by enhancing motivation, creativity, and self-refection. I have designed, built, and studied interventions grounded in learners’ and educators’ experiences thereby challenging prevailing techno-centric approaches that focus solely on skill enhancement. This work particularly investigates three frameworks for skill learning: adaptive, game-based, and refection-based, targeting motor skills, maker skills, and fabrication skills, respectively. These frameworks are informed by existing LS theories and insights into educators’ teaching strategies. The tools I built using these frameworks also leverage technological advancements in sensing, shape-changing interfaces, computer vision, and generative artificial intelligence.

Specifically, the adaptive learning framework focuses on dynamically adjusting the difficulty of training tasks based on individual skill levels. This personalized approach aims to optimize the learning experience by maintaining an optimal challenge point. The game-based learning approach explores the integration of fabrication with existing digital games, fostering engagement and motivation. And finally, the refection-based learning approach emphasizes self-directed learning through critical dialogue and prompts aligned with learning goals. The effectiveness of the learning tools built using these frameworks was evaluated through user studies. By examining the deployment of these tools and studying their impact on learners, my research contributes to expanding the design space for learning tools and deepening our understanding of how humans learn with technology. Through these contributions to the field of HCI and LS, this thesis opens up new ways of designing a learner-centric future of comprehensive skill development.