Show simple item record

dc.contributor.authorLeong, Michael
dc.contributor.authorAbdelhalim, Awad
dc.contributor.authorHa, Jude
dc.contributor.authorPatterson, Diane
dc.contributor.authorPincus, Gabriel L
dc.contributor.authorHarris, Anthony B
dc.contributor.authorEichler, Michael
dc.contributor.authorZhao, Jinhua
dc.date.accessioned2024-08-28T18:45:34Z
dc.date.available2024-08-28T18:45:34Z
dc.date.issued2024
dc.identifier.urihttps://hdl.handle.net/1721.1/156437
dc.description.abstractTransit riders' feedback provided in ridership surveys, customer relationship management (CRM) channels, and in more recent times, through social media is key for transit agencies to better gauge the efficacy of their services and initiatives. Getting a holistic understanding of riders' experience through the feedback shared in those instruments is often challenging, mostly due to the open-ended, unstructured nature of text feedback. In this paper, we propose leveraging traditional transit CRM feedback to develop and deploy a transit-topic-aware large language model (LLM) capable of classifying open-ended text feedback to relevant transit-specific topics. First, we utilize semi-supervised learning to engineer a training dataset of 11 broad transit topics detected in a corpus of 6 years of customer feedback provided to the Washington Metropolitan Area Transit Authority (WMATA). We then use this dataset to train and thoroughly evaluate a language model based on the RoBERTa architecture. We compare our LLM, MetRoBERTa, to classical machine learning approaches utilizing keyword-based and lexicon representations. Our model outperforms those methods across all evaluation metrics, providing an average topic classification accuracy of 90%. Finally, we provide a value proposition of this work demonstrating how the language model, alongside additional text processing tools, can be applied to add structure to open-ended text sources of feedback like Twitter. The framework and results we present provide a pathway for an automated, generalizable approach for ingesting, visualizing, and reporting transit riders' feedback at scale, enabling agencies to better understand and improve customer experience.en_US
dc.language.isoen
dc.publisherSAGE Publicationsen_US
dc.relation.isversionof10.1177/03611981231225655en_US
dc.rightsCreative Commons Attribution-Noncommercial-ShareAlikeen_US
dc.rights.urihttp://creativecommons.org/licenses/by-nc-sa/4.0/en_US
dc.sourcearxiven_US
dc.titleMetRoBERTa: Leveraging Traditional Customer Relationship Management Data to Develop a Transit-Topic-Aware Language Modelen_US
dc.typeArticleen_US
dc.identifier.citationLeong, M., Abdelhalim, A., Ha, J., Patterson, D., Pincus, G. L., Harris, A. B., Eichler, M., & Zhao, J. (2024). MetRoBERTa: Leveraging Traditional Customer Relationship Management Data to Develop a Transit-Topic-Aware Language Model. Transportation Research Record.en_US
dc.contributor.departmentMassachusetts Institute of Technology. Department of Urban Studies and Planning
dc.relation.journalTransportation Research Record: Journal of the Transportation Research Boarden_US
dc.eprint.versionAuthor's final manuscripten_US
dc.type.urihttp://purl.org/eprint/type/JournalArticleen_US
eprint.statushttp://purl.org/eprint/status/PeerRevieweden_US
dc.date.updated2024-08-28T18:38:56Z
dspace.orderedauthorsLeong, M; Abdelhalim, A; Ha, J; Patterson, D; Pincus, GL; Harris, AB; Eichler, M; Zhao, Jen_US
dspace.date.submission2024-08-28T18:38:58Z
mit.licenseOPEN_ACCESS_POLICY
mit.metadata.statusAuthority Work and Publication Information Neededen_US


Files in this item

Thumbnail

This item appears in the following Collection(s)

Show simple item record