Subject
Consumer product testing via repeated trials in one session.

Type of Report
Analysis of methodological issues.

Objective
To help researchers choose among alternative repeat-trial, forced-choice product test formats in terms of efficiency in estimating subjects’ discrimination ability and true preferences.

Method
Develops analytical techniques for comparing formats; illustrates techniques with a comparison between a format consisting of four paired comparisons (FPC) and one of two triangle comparisons and a single paired comparison (DTSP).

Conclusions

• When subjects cannot discriminate between the products being tested, their choices are a result of chance rather than true preference and will bias the preference results toward a 50/50 tie. The analytical techniques allow a researcher to estimate subjects’ true preference and discrimination ability for any observed sample result and to calculate approximate confidence intervals, relative efficiencies, and sample sizes before any subjects are tested.
• The FPC format in most cases is more efficient than the DTSP format for estimating preference; at best it is four times more efficient for a given sample size. However, the DTSP format is always more efficient for estimating the average discrimination ability of the population as well as the heterogeneity of discrimination.
• When estimating preferences with either format, the minimum sample size required for a 95% confidence interval to a precision of ±.05 is about 400. The required sample size rapidly increases as average discrimination ability decreases.
• The results have implications for evaluating preference test data reported in comparative advertisements; it appears that many claims in such ads are based on inadequate sample sizes.
• The approach depends upon some key assumptions: each subject has a discrimination probability which is constant across trials; nondiscriminating subjects choose randomly; and each subject has a preference, however weak, which is consistent across trials.

Target Audience/Applicability
Marketing researchers in consumer packaged goods companies