Google BigQuery for Education
Name
LS2017_paper_59.pdf
Description
Accepted version
Size
3.28 MB
Format
Unknown
Checksum (MD5)
e1081e2fbffb92b5f82e36a9d71be250
Author(s)
Lopez, Glenn
Seaton, Daniel T.
Ang, Andrew
Tingley, Dustin
Chuang, Isaac L.
Alternative Title
Framework for Parsing and Analyzing edX MOOC Data
Date Issued
April 2017
Publisher
ACM
Citation
Lopez, Glenn, Seaton, Daniel T., Ang, Andrew, Tingley, Dustin and Chuang, Isaac. 2017. "Google BigQuery for Education."
Version
Author's final manuscript
Abstract
© 2017 ACM. The size and complexity of MOOC data present overwhelming challenges to many institutions. This paper details the functionality of edx2bigquery -An open source Python package developed by Harvard and MIT to ingest and report on hundreds of MITx and HarvardX course datasets from edX, making use of Google BigQuery to handle multiple terabytes of learner data. For this application, we find that Google BigQuery provides ease of use in loading the multi-faceted MOOC datasets and near real-Time interactive querying of data, including large clickstream datasets; moreover, we are able to provide flexible research and reporting dashboards, visualizing and aggregating data, by interfacing services associated with BigQuery. This framework makes it feasible for edx2bigquery to be open source, following standards which emphasize the importance of data products that transcend a particular data science platform and allow teams with diverse backgrounds to interact with data. edx2bigquery is being adopted by other institutions with an aim toward future collaboration.
MIT Department
MIT-IBM Watson AI Lab
Terms of Use
Creative Commons Attribution-Noncommercial-Share Alike
Persistent DSpace Link
DOI of Published Version
https://doi.org/10.1145/3051457.3053980
