May 9, 2023 | 12:00 pm - 12:30 pm ET 


MSI Webinar: Ensembling Experiments to Optimize Interventions Along Customer Journey: A Reinforcement Learning Approach

Firms adopt randomized experiments to evaluate various interventions (e.g., website design, creative content, and pricing). However, most randomized experiments are designed to identify the impact of one specific intervention. The literature on randomized experiments lack a holistic approach to optimize a sequence of interventions along the customer journey. Specifically, locally optimal interventions unveiled by randomized experiments might be globally sub-optimal when considering their interdependence as well as the long-term rewards. Fortunately, the accumulation of a large number of historical experiments creates exogenous interventions at different stages along the customer journey and provides a new opportunity. This study integrates multiple experiments within the Reinforcement Learning (RL) framework to tackle the questions that cannot be answered by standalone randomized experiments: How can we learn optimal policy with a sequence of interventions along the customer journey based on an ensemble of historical experiments? And how can we learn from multiple historical experiments to guide future intervention trials? We propose a Bayesian Recurrent Q Network (BRQN) model that leverages the exogenous interventions from multiple experiments to learn their effectiveness at different stages of the customer journey and optimize them for long-term rewards. Beyond optimization within the existing interventions, the Bayesian model also estimates the distribution of rewards, which can guide subject allocation in the design of future experiments to optimally balance exploration and exploitation. In summary, the proposed model creates a two-way complementarity between RL and randomized experiments, and thus provides a holistic approach to learning and optimizing interventions along the customer journey.


Download the Presentation

Watch the Recording

Read the Summary


Yicheng Song

University of Minnesota

Yicheng Song is an assistant professor at the Carlson School of Management, University of Minnesota. His research focuses on investigating digital user’s decision journeys and the corresponding firm strategies using inter-discipline approaches: Machine Learning, Bayesian Modeling, Economic Structural Modeling and Reinforcement Learning. He holds PhD in information systems from Boston University and PhD in Computer Science from Chinese Academy of Sciences.

By using you agree to our use of cookies as identifiers and for other features of the site as described in our Privacy Policy.