Webinars
Can AI Stand in for Your Consumers? What Marketers Need to Know Before Using LLMs for Research
As large language models (LLMs) become faster and cheaper, many marketing and insights teams are testing them as stand-ins for real consumers, promising quicker insights at lower cost. But do these “synthetic consumers” actually behave like humans?
In this webinar, researchers evaluate leading LLMs using a well-established decision task and find a critical disconnect between human-like language and human-like decision making. Even with advanced techniques like prompt engineering and fine-tuning, LLMs consistently fail to reproduce key patterns of human behavior, and often in unpredictable ways driven by small changes in prompts or framing.
Join this session to learn when LLM-based research can add value, where it introduces hidden risk, and how to use AI responsibly without replacing the human insight your decisions depend on.
speaker
Dokyun (DK) Lee is a Kelli Questrom Chair Associate Professor of Information Systems and Computing & Data Sciences at Boston University. His research examines the development, deployment, and impact of artificial intelligence in business and society, with particular emphasis on generative AI, large language models, and unstructured data. His work studies how AI systems affect firm behavior, consumer behavior, market outcomes, and broader societal consequences, including regulation and governance. This includes empirical and causal analysis of AI reliability, human–AI interaction, and the economic implications of algorithmic systems, with attention to the limitations, failure modes, and unintended consequences that arise when AI technologies are deployed in real-world organizational and legal contexts. He is the Principal Investigator of the Business Insights through Text (BIT) Lab (www.dkBITLAB.com) and the lead of the Boston University Digital Business Institute Generative AI Lab, where he conducts interdisciplinary research integrating AI, economics, and information systems. Area of Expertise and Research - Generative AI in Business and Society: Economic and societal impact, organizational use, evaluation, governance, and regulatory implications - Economics of Unstructured Data: Content extraction, value measurement, monetization, and engineering - AI Reliability and Validity: Behavioral consistency, robustness, and limits of AI systems - Unintended Consequences of AI: Market impacts, societal and regulatory risk - Customized and Enterprise Human-AI Systems: Design, assessment, and improvements His research is applied across digital consumer management, AI regulation, platform and market design, competition, advertising, human–AI collaboration, innovation, and creativity. Dokyun has published in leading peer-reviewed journals, including Management Science, Information Systems Research, MIS Quarterly, Proceedings of the National Academy of Sciences (PNAS), Science Advances, Nature Scientific Reports, and Journal of Marketing Research, as well as top artificial intelligence venues such as AAAI, AIES, WWW, and NeuRips Workshop. Dokyun’s work has received numerous scholarly distinctions, including ISR Best Paper Award (2020), AAAI Award (2021), AMA Don Lehmann Award (2024), Management Science Best Paper Award (2025), 6 finalist distinctions for the Management Science ISR and Marketing Division Best Paper Award, and 13 best-paper awards from prominent conferences (WISE, CIST, ICIS, INFORMS). Dokyun's research has been supported by organizations including Adobe, Google, NVIDIA, McKinsey & Company, Bosch Institute, Marketing Science Institute, Net Institute, Prudential Foundation, and MassMutual. His work is frequently consulted in contexts involving AI system impact evaluation, economic impact assessment, regulatory analysis, and disputes concerning the design, deployment, or effects of generative AI technologies, with implications for firms, consumers, and society at large.