The MaxDiff is a long-established academic mathematical theory with very specific assumptions about how people make choices: it assumes that respondents evaluate all possible pairs of items within the displayed set and choose the pair that reflects the maximum difference in preference or importance. It may be thought of as a variation of the method of Paired Comparisons. Consider a set in which a respondent evaluates four items: A, B, C and D. If the respondent says that A is best and D is worst, these two responses inform us on five of six possible implied paired comparisons: The only paired comparison that cannot be inferred is B vs. C. In a choice among five items, MaxDiff questioning informs on seven of ten implied paired comparisons.
Overview
In 1938 Richardson introduced a choice method in which subjects reported the most alike pair of a triad and the most different pair. The component of this method involving the most different pair may be properly called "MaxDiff" in contrast to a "most-least" or "best-worst" method where both the most different pair and the direction of difference are obtained. Ennis, Mullen and Frijters derived a unidimensional Thurstonian scaling model for Richardson's method of triads so that the results could be scaled under normality assumptions about the item percepts. MaxDiff may involve multidimensional percepts, unlike most-least models that assume a unidimensional representation. MaxDiff and most-least methods belong to a class of methods that do not require the estimation of a cognitive parameter as occurs in the analysis of ratings data. This is one of the reasons for their popularity in applications. Other methods in this class include the 2- and 3-alternative forced choice methods, the triangular method which is a special case of Richardson's method, the duo-trio method and the specified and unspecified methods of tetrads. All of these methods have well developed Thurstonian scaling models as discussed recently in Ennis which also includes a Thurstonian model for first-last or most-least choice and ranks with rank-induced dependencies. There are a number of possible processes through which subjects may make a most-least decision, including paired comparisons and ranking, but it is typically not known how the decision is reached.
Maxdiff and best–worst scaling have erroneously been considered synonyms. Respondents can produce best-worst data in any of a number of ways, with a maxdiff process being but one. Instead of evaluating all possible pairs, they might choose the best from n items, the worst from the remaining n-1, or vice versa. Or indeed they may use another method entirely. Thus it should be clear that maxdiff is a subset of BWS; maxdiff is BWS, but BWS is not necessarily maxdiff. Indeed, maxdiff might not be considered an attractive model on psychological and intuitive grounds: as the number of items increases, the number of possible pairs increases in a multiplicative fashion: n items produces n pairs. Assuming respondents do evaluate all possible pairs is a strong assumption. Early work did use the term maxdiff to refer to BWS, but with Marley's return to the field, correct academic terminology has been disseminated throughout Europe and Asia-Pacific.
In North America the term maxdiff continues to be used for studies that are in fact BWS studies. This probably reflects the fact that Sawtooth Software in the United States implements a procedure it has named maxdiff. Indeed, it is unclear whether this procedure implements maxdiff procedures in estimating parameters of their models, or whether the simpler, sequential BWS model is used. Datagame has implemented an application called MaxDiff Rankifier which uses gamification techniques and interactive visuals to streamline the data collection process.
Conduct and analysis of a maxdiff survey
A maxdiff survey may be designed and analysed in two ways:
Through manual design and researcher-led analysis using standard statistical software procedures or
By utilising a commercial survey company which supports this type of discrete choice model.
Analysis of a MaxDiff survey will usually contain data that shows how many times each attribute was displayed, in addition to the numbers of times the attribute was chosen as best and worst. These three data points quantify the relative importance of each item.
