Feedback loops in educational environments using web-based survey tools : new technology development and three implementation case studies

Abstract

This thesis presents lessons from the development of an on-line, web-based feedback system and preliminary analysis of the socio-technical interactions associated with the specification, design and use of this system. The system has been designed to facilitate many forms of feedback in educational environments. Novel technical components of the system include: (1) a highly detailed and flexible data model representing the social-system feedback loops, (2) reusable feedback objects and sub-objects which minimize content entry requirements while improving data aggregation capabilities, and (3) feedback objects which respond dynamically based on the context in which they are used. Novel social system learning associated with this thesis centers on the mechanisms to facilitate feedback in academic and industry learning environments. This thesis features the design, implementation, and preliminary use of the Online Feedback Organization, Requesting and Monitoring for Educators (OnFORME) system. OnFORME was created to enable ubiquitous feedback collection in a semi-transparent manner for educators in both academic and corporate training environments. OnFORME provides a way for educators to quickly author feedback objects (e.g. surveys) and publish them via the World Wide Web so that students can easily respond. The author used agile development methodology to design and implement this application directly from customer use cases in a bottom-up approach. Inductive research methods included observations of thirteen potential adoption trials, of which nine progressed to full usage trials. In depth interviews of potential users, actual users, and experts in this field led to insights on the adoption and use of new IT systems in education.

(cont.) Findings based on usage trials include (1) adoption is more likely when a parallel set of supporting institutional arrangements already exists, as well as specific feedback tools already in use, (2) face-time and hands-on demonstration are required for new system adoption, (3) including feedback as part of a predefined student role improves response rates, and (4) performing a training survey prior to the students' departure from a classroom setting increases the response rate.