Journal Selections from MSI

Text Mining Creates “Map” of Competitive Landscape

Not all consumers write about products and services on online forums, but the many who do create a body of opinions and reflections that has the potential to give marketers insights that market research cannot generate.

In Marketing Science, Oded Netzer, Ronen Feldman, Jacob Goldenberg, and Moshe Fresko describe a procedure for mining the text of online forums and translating the data into perceptual maps that depict the relationships among sets of competing brands and their attributes, and that show how perceptions evolve over time and in response to changing market conditions. Whether the commenters on online forums are representative of consumers generally remains to be established.

The text mining procedure identified pairs and trios of brands (or other entities of interest) in the messages that consumers post to online forums, together with the phrases used to link them (such as “better than”). This step was performed by an automated machine learning tool combined with manually created rules. Grammatical and typing errors in the messages were corrected by a method that identifies likely errors by their infrequency and replaces them with statistically more likely terms. The result was a matrix of co-occurrences of brands and attributes. More popular brands co-occur with all attributes than do less popular brands, so the data were normalized.

The paper applied the method to two types of forums. The first,, contained over 800,000 messages posted by 76,000 consumers in which 135 sedan car models were mentioned. Using the principle that the more frequently two car brands co-occur the more similar they are, a visualization of similarities and differences in the market for sedan cars was constructed. It revealed 16 clusters of car models. Next a list of more than 1,000 car descriptors used in the messages was factor-analyzed and the resulting three factors, along with car characteristics, were used to construct a semantic network to explain judgments of similarity.

The second study examined a set of five diabetes drug forums. Here the focus went beyond drug pair co-occurrence, and attended to the co-occurrence of particular drugs and adverse drug reactions. The visualization showed similarities among drugs based on patterns of adverse reaction. Additionally this analysis tracked how frequently one drug was described as high cost over time, along with a monthly index of consumer confidence in the U.S. economy. As consumer confidence declined with the onset of the recession in 2008, mentions of high cost increased.

“Mine Your Own Business: Market-Structure Surveillance Through Text Mining” by Oded Netzer, Ronen Feldman, Jacob Goldenberg, and Moshe Fresko, Marketing Science (May/June 2012), 521-543 To purchase article, go to

Related links

Social Media Intelligence: Measuring Brand Sentiment from Online Conversations
David A. Schweidel, Wendy W. Moe, and Chris Boudreaux, 2012 [12-100]


You must be logged in to leave comments. Please login.



Employees of MSI Member Companies enjoy the benefits of complete online access to content, member conferences and networking with the MSI community.



Qualified academics benefit from a relationship with MSI through access to, conferences and research opportunities.



The public is invited to enjoy partial access to content, a free e-newsletter, selected reports and more.



Become a Subscriber

MSI's Online Library of 400+ reports, authored by marketing academics, offers new research and evidence-based insights

Read More

Priority Topics

Stay Informed

The MSI Mailing List

Subscribe to our email list to stay informed about upcoming events, news, etc.

Social Media