Conferences
MSI 2023 Immersion: How to Use ChatGPT for Research and Marketing
For marketers and market researchers, the question is not whether AI will replace humans but rather where and how to do this. Social media, search and other online consumer activities have been described as “the world’s largest focus group.” Trained on this data, large language models (LLM) such as ChatGPT now allow users to synthesize and query this vast store of information for research insights and to develop and test marketing content.
At MSI’s in-person Immersion 2023 Conference, “hands on” applications of the latest academic theory will demonstrate the potential of generative AI to “replace” humans constructively in ways that can enhance both research and marketing.
- Introduction to Large Language Models and Their Applications in Marketing Practice
Zhou Yu, Columbia University
This session first explains the internal mechanism of large language models, such as GPT-4 and ChatGPT. Then we will discuss some strengths and shortcomings of current large language models. Finally, we discuss how to apply these models in marketing practices. We will do some exercises together for content creation and show case chatbots powered by LLMs that handle inbound prospect questions and perform prospect qualifications.
- Using GPT for Market Research
Ayelet Israeli, Harvard Business School
In this session, we will discuss the potential uses and benefits of LLMs for researchers and practitioners who aim to understand consumer preferences. We will focus on OpenAI’s GPT and use it to generate hundreds of survey responses to our prompts and evaluate whether these responses are useful and meaningful (based on the research available here). Then, we will dive into whether and how to apply this approach: When could we / should we use GPT for market research questions? What are the limitations and challenges that exist? What are the opportunities it presents? And what should we consider when designing prompts (and evaluating GPT’s responses)? We will also present a short demo using session participants’ use cases.
- LLM-Enabled Search: Potential for Consumer Search, Advertising, and Sales
Dan Goldstein, Microsoft Research
LLM-enabled search engines have arrived. How might they transform consumer behavior and marketing practice? This session considers the potential of LLM-enabled search in streamlining consumer product search and the ability to craft instantaneous product comparison matrices. Beyond these innovations, we face critical questions: How do we navigate the challenges of LLM accuracy and address the emerging concerns of putting too much trust in AI generated results? We’ll share empirical data from studies on individuals using LLM-enabled search for product research and explore effective strategies to handle LLM output errors. We conclude by considering the past and future of search advertising and ask another question: Are LLMs setting the stage for a renaissance of the conversational sales model?
- Language Models for Automated Market Research: A New Way to Generate Perceptual Maps
Zsolt Katona, UC Berkeley
This session demonstrates a method to generate perceptual maps using the latest large language models (e.g. ChatGPT). Comparing the AI generated results with human surveys of brands perceptions shows high agreement rates if the right prompts are used. The method works for maps built from both for raw similarity measures and also from brand attributes with appropriate prompting. A key advantage of the the AI-based market research method is that it can elicit brand perceptions and preferences for certain demographic groups, given time periods and even certain contexts, which may be difficult to accomplish with surveys.
The MSI 2023 Immersion is included in MSI Corporate Membership.
Please note, we are able to accommodate a maximum of 5 attendees per member company. Registrants over this limit will be placed on a wait list.
The conference will be held at the Advertising Research Foundation at: 432 Park Ave S., 4th Floor, NYC 10016
Tuesday, September 12, 2023
8:30 – 9:00 a.m. Registration & Continental Breakfast
9:00 – 9:15 a.m. Welcome and Opening Remarks
John Lynch, University of Colorado and Executive Director, Marketing Science Institute
9:15 – 10:30 a.m. Introduction to Large Language Models and Their Applications in Marketing Practice
Zhou Yu, Columbia University
This session first explains the internal mechanism of large language models, such as GPT-4 and ChatGPT. Then we will discuss some strengths and shortcomings of current large language models. Finally, we discuss how to apply these models in marketing practices. We will do some exercises together for content creation and show case chatbots powered by LLMs that handle inbound prospect questions and perform prospect qualifications.
10:30 a.m. – 10:45 a.m. Coffee Break with Speakers
10:45 a.m. – 12:00 p.m. Using GPT for Market Research
Ayelet Israeli, Harvard Business School
In this session, we will discuss the potential uses and benefits of LLMs for researchers and practitioners who aim to understand consumer preferences. We will focus on OpenAI’s GPT and use it to generate hundreds of survey responses to our prompts and evaluate whether these responses are useful and meaningful (based on the research available here). Then, we will dive into whether and how to apply this approach: When could we / should we use GPT for market research questions? What are the limitations and challenges that exist? What are the opportunities it presents? And what should we consider when designing prompts (and evaluating GPT’s responses)? We will also present a short demo using session participants’ use cases.
12:00 – 1:15 p.m. Lunch with Speakers & Town Hall
The success of generative AI and LLMs is due in large part to the vast amounts of public data available online for training. Just as there are questions about fair use of copyrighted content, there are concerns about privacy and regulations that may restrict access to this data.
To explore these issues, you are invited to join us for this ARF Town Hall: Striking a Balance: Privacy, Personalization & Profit
Personalization means an enhanced online experience and more efficient transactions for consumers. However, in recent years, regulators and web browsers have limited online tracking to reduce privacy concerns such as unauthorized use of collected data, behavioral tracking, and filter bubbles. This has led to consumer data fragmentation for many organizations.
What is the tradeoff between personalization and privacy? How are the benefits of personalization altered, which lead to lower quality recommendations, increased search costs, decreased customer engagement and loyalty as well as overall lower conversion rates? Addressing the privacy issues is a concerted effort that will involve policymakers, and technology organizations.
1:15 – 1:30 p.m. Coffee Break with Speakers
1:30 – 2:45 p.m. LLM-Enabled Search: Potential for Consumer Search, Advertising, and Sales
Dan Goldstein, Microsoft Research
LLM-enabled search engines have arrived. How might they transform consumer behavior and marketing practice? This session considers the potential of LLM-enabled search in streamlining consumer product search and the ability to craft instantaneous product comparison matrices. Beyond these innovations, we face critical questions: How do we navigate the challenges of LLM accuracy and address the emerging concerns of putting too much trust in AI generated results? We’ll share empirical data from studies on individuals using LLM-enabled search for product research and explore effective strategies to handle LLM output errors. We conclude by considering the past and future of search advertising and ask another question: Are LLMs setting the stage for a renaissance of the conversational sales model?
2:45 – 3:00 p.m. Break & Dessert
3:00 – 4:15 p.m. Language Models for Automated Market Research: A New Way to Generate Perceptual Maps
Zsolt Katona, UC Berkeley
This session demonstrates a method to generate perceptual maps using the latest large language models (e.g. ChatGPT). Comparing the AI generated results with human surveys of brands perceptions shows high agreement rates if the right prompts are used. The method works for maps built from both for raw similarity measures and also from brand attributes with appropriate prompting. A key advantage of the the AI-based market research method is that it can elicit brand perceptions and preferences for certain demographic groups, given time periods and even certain contexts, which may be difficult to accomplish with surveys.
4:15 p.m. Conclude
4:15 – 5:30 p.m. Networking Reception
Park South Hotel - JDV by Hyatt
MSI has secured a block of rooms at the Park South Hotel at a discounted rate of $439 per night for a Queen room. In order to secure a room at the group rate please use this link to book.
Availability and the discount for rooms can only be guaranteed until August 18, 2023, so please make sure to book your accommodations early.