Using Student Annotated Hashtags and Emojis to Collect Nuanced Affective States

Abstract

Determining affective states such as confusion from students' participation in online discussion forums can be useful for instructors of a large classroom. However, manual annotation of forum posts by instructors or paid crowd workers is both time-consuming and expensive. In this work, we harness affordances prevalent in social media to allow students to self-Annotate their discussion posts with a set of hashtags and emojis, a process that is fast and cheap. For students, selfannotation with hashtags and emojis provides another channel for self-expression, as well as a way to signal to instructors and other students on the lookout for certain types of messages. This method also provides an easy way to acquire a labeled dataset of affective states, allowing us distinguish between more nuanced emotions such as confusion and curiosity. From a dataset of over 25,000 discussion posts from two courses containing self-Annotated posts by students, we demonstrate how we can identify linguistic differences between posts expressing confusion versus curiosity, achieving 83% accuracy at distinguishing between the two affective states.