Learning from other perspectives : design and analysis of an in-place annotation system

Abstract

I designed and studied NB, an in-place collaborative document annotation system targeting students reading lecture notes and draft textbooks. Serving as a discussion forum in the document margins, NB lets users ask and answer questions about their reading material as they are reading. Questions, replies and comments from students and faculty members are displayed in place and provide new perspectives on the content. NB also provides comment browsing interfaces that help the staff cope with reading assignments in large classes. I describe the NB system and its evaluation in real class environments, where students used it to submit their reading assignments, ask questions and get or provide feedback. I show that this tool has been successfully incorporated into numerous courses worldwide, and that students prefer to use NB to read their notes, rather than printing out copies that are missing these annotations. The data I collected indicates that NB encourages students to comment on the class material, even students who are not verbally active in class. To understand how and why, I focused on a particularly successful class deployment where the instructor adapted his teaching style to take students' comments into account. I analyzed the annotation practices that were observed - including the way spatial locality was exploited in ways unavailable in traditional forums. I then surveyed 30 faculty members from classes where NB was substantially used and set up an A/B experiment in an edX course, where only half of the students had access to NB. Contrary to previous literature results, in-class participation, in-place annotations and forum annotations do not necessarily compete with each other. From those observations, I derive general design implications for online annotation tools in academia.