ColloGraphy: Designing Augmented Visual-Haptic Feedback Systems to Support Fine Motor Skill Learning

Abstract

Learning motor skills is an essential part of our daily lives. However, certain activities can be challenging to comprehend through observation alone, especially for beginners. Specifically, this thesis focuses on Chinese calligraphy writing as an example of a complex, fine motor task, in which learning it can be augmented by additional feedback. The traditional method of learning calligraphy writing involves learners comparing the visual differences in their writing with static expert manuscripts. In this process, the opportunity for novice learners to internalize the bodily sensations required for mastery is absent. To address this issue, this thesis presents the design of a series of prototypes that capture, recreate, and re-enact bodily movement during calligraphy writing. These prototypes support novice learners with augmented visual and haptic feedback. The thesis presents a comprehensive comparison of the three different approaches explored in the prototypes, highlighting the importance and challenge of finding the right amount of system intervention to achieve effective motor skill learning; in short, inconsistent or excessive intervention may lead to confusion or over-reliance, while insufficient intervention may fail to assist learning. Accordingly, design recommendations for successful multimodal feedback systems for supporting motor skill learning are recommended.