Identifying Customer Needs from User-Generated Content

Artem Timoshenko and John R. Hauser , 2018, 18-124-08

Deep understanding of customer needs is extremely important to marketing strategy and product development. In marketing strategy, customer needs help to segment the market, identify strategic dimensions for differentiation, and make efficient channel management decisions. In product development, customer needs are used to identify new product opportunities, improve design of new products, and help manage product portfolios.

To identify customer needs, firms traditionally rely on interviews and focus groups, which are expensive and time-consuming. User-generated content (UGC), such as online reviews, social media, and blogs, offers an alternative source of information from which to identify customer needs.

In this study, Artem Timoshenko and John Hauser compare user-generated content (Amazon reviews) to experiential interviews as a source of customer needs. In the oral care product category they investigate, UGC identifies a comparable set of customer needs and at a lower cost than the experiential interviews conducted by a professional marketing consulting firm. Moreover, UGC contains customer needs not identified in experiential interviews, which suggests new opportunities for product development and/or new strategic positioning.

The authors further propose and validate a machine-learning hybrid approach to improve the efficiency of identifying customer needs from UGC. The approach uses machine learning methods to identify relevant content and remove redundancy from a large UGC corpus. The selected content is then provided to human analysts to formulate customer needs. Machine learning amplifies efficiency of the qualitative review by 15-20% in terms of professional services cost. The overall gains of analyzing UGC with the proposed approach over the traditional interview-based analysis are 46-52%.

Artem Timoshenko is a Ph.D. candidate in marketing and John R. Hauser is Kirin Professor of Marketing, both at MIT Sloan School of Management.

The authors thank John Mitchell, Steven Gaskin, Carmel Dibner, Andrea Ruttenberg, Patti Yanes, Kristyn Corrigan, and Meaghan Foley for their help and support. They thank Regina Barzilay, Clarence Lee, Daria Dzyabura, Dean Eckles, Duncan Simester, Evgeny Pavlov, Guilherme Liberali, Theodoros Evgeniou, and Hema Yoganarasimhan for helpful comments and discussions, and Ken Deal and Ewa Nowakowska for suggestions on earlier versions of this paper. This paper has benefited from presentations at the 2016 Sawtooth Software Conference in Park City Utah, the MIT Marketing Group Seminar, the 39th ISMS Marketing Science Conference, and presentations at Applied Marketing Science, Inc. and Cornerstone Research, Inc. The applications in §6 were completed by Applied Marketing Science, Inc. Finally, they thank the anonymous reviewers and Associate Editor for constructive comments that enabled them to improve our research.

Note: This report is an accepted manuscript for Marketing Science


  • Corporate: FREE
  • Academic: FREE
  • Subscribers: FREE
  • Public: $0.00



Employees of MSI Member Companies enjoy the benefits of complete online access to content, member conferences and networking with the MSI community.



Qualified academics benefit from a relationship with MSI through access to, conferences and research opportunities.



The public is invited to enjoy partial access to content, a free e-newsletter, selected reports and more.




Learn more about becoming a member of the institute

Read More

Stay Informed

The MSI Mailing List

Subscribe to our email list to stay informed about upcoming events, news, etc.

Social Media