Working Papers

Measuring Simple Preferences: An Approach to Blind, Forced-Choice Product Testing

Jan 1, 1985

Subject
Consumer product testing via repeated trials in one session.

Type of Report
Analysis of methodological issues.

Objective
To help researchers choose among alternative repeat-trial, forced-choice product test formats in terms of efficiency in estimating subjects’ discrimination ability and true preferences.

Method
Develops analytical techniques for comparing formats; illustrates techniques with a comparison between a format consisting of four paired comparisons (FPC) and one of two triangle comparisons and a single paired comparison (DTSP).

Conclusions

  • When subjects cannot discriminate between the products being tested, their choices are a result of chance rather than true preference and will bias the preference results toward a 50/50 tie. The analytical techniques allow a researcher to estimate subjects’ true preference and discrimination ability for any observed sample result and to calculate approximate confidence intervals, relative efficiencies, and sample sizes before any subjects are tested.
  • The FPC format in most cases is more efficient than the DTSP format for estimating preference; at best it is four times more efficient for a given sample size. However, the DTSP format is always more efficient for estimating the average discrimination ability of the population as well as the heterogeneity of discrimination.
  • When estimating preferences with either format, the minimum sample size required for a 95% confidence interval to a precision of ±.05 is about 400. The required sample size rapidly increases as average discrimination ability decreases.
  • The results have implications for evaluating preference test data reported in comparative advertisements; it appears that many claims in such ads are based on inadequate sample sizes.
  • The approach depends upon some key assumptions: each subject has a discrimination probability which is constant across trials; nondiscriminating subjects choose randomly; and each subject has a preference, however weak, which is consistent across trials.

Target Audience/Applicability
Marketing researchers in consumer packaged goods companies

This resource is not available online.

By using MSI.org you agree to our use of cookies as identifiers and for other features of the site as described in our Privacy Policy.