Identifying customer needs from user-generated content

Abstract

Understanding customer needs is an important part of marketing strategy, product development, and marketing research. The explosive growth of user-generated content (UGC) creates an opportunity to enhance industry-standard interview-based approaches for identifying customer needs. However, the traditional manual review approach is neither efficient nor effective when applied to a large UGC corpus because non-informative and repetitive content crowd out information about customer needs. We identify customer needs from UGC by combining machine learning methods to select content for review with human judgement to formulate customer needs. In particular, we use a convolutional neural network to filter out non-informative content and dense sentence representations to identify sufficiently different sentences for manual review. An empirical proof-of-concept compares customer needs for oral care products identified from online reviews (UGC) with customer needs identified by a third-party professional consulting firm using industry-standard methods. In this application, UGC identifies additional customer needs, unreachable by the interview-based approach. Our approach improves efficiency of manual review in terms of a number of unique customer needs per unit effort.