MaxDiff Analysis: Survey Design and a Worked Example
Learn how MaxDiff compares priorities within a list. Follow a balanced task design, check count scores and use the findings to choose focused prototype work.

MaxDiff asks respondents to choose the most and least important item from small sets, then estimates each item's relative priority. It can help when a simple ranking is difficult. If your decision depends on trade-offs between complete offers and prices, use conjoint instead.
MaxDiff analysis at a glance
What it answers | Relative priorities among comparable items |
What you need | A defined audience, item list and balanced most/least choice tasks |
What you get | Relative item priorities, with the response base and design retained |
Use something else when | You need trade-offs between complete offers and prices; use conjoint. |
The result helps you focus development or messaging work within a defined list. Keep the wording comparable, connect the items through the task design and explain how you calculate the scores. Net best-minus-worst counts are descriptive proportions, not fitted utilities or buying probabilities. Price and product performance need separate evidence.
How to set up a MaxDiff survey
About this example: Imagine you are choosing which meal-planner functions to develop for households arranging dinner. The illustrative task design and counts below show how to prioritize within a fixed list.
Code | Item |
|---|---|
A | Combine ingredients into one shopping list |
B | Save a weekly dinner plan |
C | Filter recipes by preparation time |
D | Share the plan with another household member |
Your meal-planner team has more useful functions than it can develop. The question asks about value in dinner planning, so keep that meaning consistent instead of mixing importance, attractiveness and purchase intent. Interpret the result using the question you actually asked.
Each statement describes one function at a similar level. “Stress-free family life” would be a whole product promise beside the narrow preparation-time filter and could dominate without explaining what to build. Split compound items and make their mechanism clear before testing the list.
Pilot whether people understand “least valuable.” It means low value among the shown alternatives, rather than harmful or unwanted in all circumstances. A household can find sharing useful and still pick it as least valuable when a combined shopping list matters more.
MaxDiff vs ranking questions
Use ranking when buyers can meaningfully order a short, clear list. Consider MaxDiff when ordering the whole list becomes difficult and designed subsets can make the comparisons manageable. Pilot both around the same decision, audience and wording before choosing the format.
Comparison | What each task gives you |
|---|---|
Task | MaxDiff: Choose the most and least valuable item in each of several smaller sets. Ranking: Put the complete shown list in order. |
Output | MaxDiff: Best/worst counts or choice-model scores, interpreted under the declared design and scoring method. Ranking: An order for each person and a distribution of ranks. Adjacent positions don’t reveal equal preference distances. |
Respondent effort | MaxDiff: Each set is smaller, but repeated sets add work and still force distinctions. Ranking: One full-list task can be straightforward for a short list and demanding for a longer one. |
List length it suits | MaxDiff: A longer list when a designed sequence provides enough coverage without exhausting respondents. Ranking: A short, clearly differentiated list that people can meaningfully order. Neither method has a universal list-length cutoff. |
Best for | MaxDiff: Relative priorities when a pilot shows that smaller repeated comparisons solve a full-list task problem. Ranking: A simple ordering when the audience can compare the whole list. Pilot ties and difficult distinctions before using a narrow difference to decide. |
For the meal-planner list above, a ranking question could ask: “Place these benefits in order from most to least valuable when planning your dinners. If two seem equally valuable, tell us that before completing the forced order.” A person might put shopping lists first, saved plans second, preparation-time filters third and household sharing fourth. Those positions show order, without telling you the distance between adjacent benefits.
A simple arithmetic example shows why the full pattern matters: ranks 1 and 4 average 2.5; ranks 2 and 3 also average 2.5. The first pair is polarized and the second is concentrated in the middle. These are illustrative positions, not collected participant records. Retain rank distributions and difficult distinctions rather than letting a small mean-rank gap dictate a large investment.
MaxDiff also forces distinctions, using the repeated subsets shown below. You still need to explain whether the score comes from counts or a fitted model. The method doesn't turn a close call into a strong preference or purchasing effect.
Keep ranking if the short list is easy to order and report near ties honestly. Test MaxDiff when full ordering overwhelms buyers, while checking the burden of the complete sequence. If statements overlap or mix a whole product promise with a narrow feature, rewrite them first. Neither format can repair that problem. Both describe priority within the included alternatives. Use a separate complete-offer comparison when the decision concerns paid configurations, willingness to pay or market behavior.
The whole instrument determines the workload
Pilot observation | Interpretation and response |
|---|---|
Difficulty separating two items | Interpretation to investigate: Near tie or overlapping wording Design response: Preserve the ambiguity or clarify the items |
“Worst” understood as harmful | Interpretation to investigate: Question meaning differs from low value Design response: Explain least valuable neutrally and repilot |
Repeated tasks feel redundant | Interpretation to investigate: Burden exceeds useful differentiation Design response: Review list size and task plan |
Full ordering is overwhelming | Interpretation to investigate: List or task is too demanding Design response: Test a designed smaller-subset approach |
Look at hesitation, explanation and completion as well as time. Fast responses can reflect ease or rushing. Repeated best-worst tasks can be manageable individually while becoming tiring collectively.
Before shortening the task plan, check which comparisons remain. MaxDiff scoring needs sufficient item coverage and connected comparisons. Hiding part of a ranking list changes the task. Ask the researcher to explain how reducing the work affects the evidence.
Choose the format at a workload the audience and team can sustain. A rank report with honest ambiguity can be more useful than a complicated model whose distinctions people could not meaningfully make.
Check what every item gets compared with
Use these four subsets: ABC, ABD, ACD and BCD. Each item appears three times; every pair appears together twice. Randomize position within tasks and manage task order. This four-item illustration is small enough to inspect; a real design should consider the full item set, exposure, connectivity and respondent burden.
One best-worst task from the balanced design
Which is most and least valuable when planning your dinners?
For an ABC task, show:
- A: combine ingredients into one shopping list;
- B: save a weekly dinner plan;
- C: filter recipes by preparation time.
Choose one most valuable and a different least valuable item from that displayed set.
The item counts and analysis script let you reconstruct the arithmetic. Sawtooth's official introduction describes best-worst item prioritization.
Copy the task design separately from the counts:
task_id | Details |
|---|---|
T1 | shown_item_codes: A, B, C question: Most and least valuable for dinner planning |
T2 | shown_item_codes: A, B, D question: Same question |
T3 | shown_item_codes: A, C, D question: Same question |
T4 | shown_item_codes: B, C, D question: Same question |
The table specifies sets; randomize presentation positions and manage task order in the instrument.
Four tasks with three items create twelve item appearances per person. Across 100 complete respondents, there are 1,200 item exposures: 300 for each of A, B, C and D. There are 400 best selections and 400 worst selections, one of each per completed task.
The exposure balance makes count comparison easy in this small example. A longer design should also maintain meaningful connected comparisons and manageable burden. Merely showing each item the same number of times does not guarantee that all relevant items are well compared.
Do not always put the same item first within its subset. Position and task order should be managed separately from which items appear together. The saved design specifies sets; the actual instrument and records should preserve their presentation.
Equal exposure can hide two disconnected lists
Before approving a longer MaxDiff questionnaire, ask what connects its items. Showing everything equally often is useful, but it doesn't guarantee that the responses support one priority order. A list split into isolated blocks can make two local winners look comparable when the tasks never compare their parts of the list.
Consider a separate hypothetical six-function design for the meal planner. Retain A–D as defined above, and add E, reuse a previous week's plan, and F, track ingredients already at home. This is a structural teaching example with no response data, separate from the four-item design and counts above.
The proposed schedule shows every person six three-item tasks: ABC three times and DEF three times. Every item appears three times per person. With 100 complete respondents, each item would have 300 exposures and the design would have 1,800 item exposures overall. Those totals look balanced.
But no item in ABC ever appears with an item in DEF. Repeating the schedule for more people leaves that separation intact. The tasks compare A, B and C with one another and D, E and F with one another, without connecting the blocks. Under the multinomial-logit rule, shifting all three utilities in one isolated block by the same amount leaves its within-task choice probabilities unchanged. Sawtooth's technical paper gives that choice rule. A common ranking across the blocks would therefore need information or assumptions beyond those tasks. Normalizing both sets to a common-looking score does not create the missing comparisons.
A simple structural audit treats each item as a point and connects two points whenever they occur in the same task. Starting at A reaches B and C, then stops. Starting at D reaches E and F. There are two connected components. Sawtooth's MaxDiff design guidance calls for items to be compared directly or indirectly, and distinguishes individual-level connectivity from connectivity obtained by pooling participants for aggregate methods.
A repair can connect the list without adding tasks
For illustration, replace the repeated blocks with the following six sets. Every person sees all six, with task order and item position managed separately.
Task | Shown item codes |
|---|---|
1 | A, B, C |
2 | D, E, F |
3 | A, B, D |
4 | C, E, F |
5 | A, C, E |
6 | B, D, F |
Every item still appears three times. The task count and planned item-exposure totals are unchanged, while the comparison graph now has one component. For example, A connects to D through task 3 and D connects to F through task 2. The two blocks are no longer isolated.
This repair demonstrates connectivity, not a finished recommended design. Pair frequencies remain unequal, and A–F, B–E and C–D never appear together directly. Indirect paths connect them. A provider still needs to assess pair and position balance, task burden, the intended estimator and how much information the planned design supplies for the required comparisons. One connecting task is not proof of adequate precision. Neither these six tasks nor 100 respondents is a universal requirement.
Run that audit at the level the report promises. A connected pooled schedule can contain disconnected schedules for individual respondents. An individual-level analysis may then depend more heavily on pooling or model assumptions. A segment's available tasks can also lose connections when the full sample remains connected. Ask how that affects the specific individual or segment claims, rather than accepting a single whole-study connectivity badge.
For the meal-planner brief, return the original split schedule for revision before fielding. Ask for the actual item sets and assignment versions, their connected components, item and pair exposures, and a design assessment for the planned analysis. After collection, check the valid tasks too: skipped or excluded bridge tasks can remove a connection that existed on paper. Preserve the original records and report any resulting limitation. Connectivity makes the missing comparison visible; it does not turn a relative priority into demand or a purchase prediction.
MaxDiff analysis: calculate best-worst scores
Every item has 300 exposures under the complete four-task design.
Item | Details |
|---|---|
A | Best / worst choices: 140 / 40 Net score: |
B | Best / worst choices: 130 / 60 Net score: |
C | Best / worst choices: 80 / 140 Net score: |
D | Best / worst choices: 50 / 160 Net score: |
These aggregate counts support the declared net score. For A, (140−40)/300=0.333; B gives 0.233; C −0.200; D −0.367 after rounding. Positive means more best than worst selections within the available exposures. Negative means the reverse. It does not mean negative utility in an independently estimated choice model.
The best counts sum to 400 and the worst counts sum to 400, matching 100 respondents and four tasks. That reconciliation checks arithmetic; it cannot reconstruct individual consistency, task validity or subgroup priorities. An aggregate file does not contain those records.
Sawtooth’s score documentation distinguishes count summaries from modeled scores. A fitted model can account for the task structure and investigate individual differences. Its output needs its own scale interpretation; do not compare a count percentage directly with a model score normalized to 100.
The list defines the competition. A and B lead here among four supplied functions. Adding a crucial missing benefit can change priorities. A high relative score does not establish absolute need, a purchase probability or the price premium for including the function.
Related guides: Kano analysis.
Preserve the task-level evidence
An ABC task needs one best and a different worst, both among A, B and C. An unshown code, blank or identical pair fails that complete-task rule. Store participant ID, task ID, shown items, positions and both selections so those checks are possible.
Use actual valid item exposures under the scoring rule. If a participant stops after two tasks, the planned balance no longer guarantees equal observed bases. Do not divide every item by 300 merely because that was the intended schedule.
Declare the treatment of partial completion and invalid tasks before analysis. A complete-case rule and a valid-task rule answer different coverage questions. Show the number of affected people and tasks, especially if exclusions change the roadmap priority.
Retain explanations from a follow-up as a separate layer. They can diagnose why an item mattered, while the best-worst task supplies the comparative choice. A persuasive quote cannot replace the count base, and a count alone cannot tell the team which part of a function to implement.
Turn the top two into product work
A’s lead points to shopping-list quality: duplicate ingredients, units, recipe portions and substitutions must work in an actual shopping journey. B asks whether saving the weekly plan lets people return to it when dinner changes. Those are focused prototype tasks with observable failure points.
Put implementation effort beside the priority. A difficult high-priority function and an inexpensive moderate-priority function can deserve a different release order. Explain the decision in the roadmap without pretending that cost has changed the measured MaxDiff score.
D’s low aggregate result calls for a household check before deletion. Sharing could be important when two adults coordinate meals and unimportant for a solo planner. The aggregates contain no subgroup records, so they cannot establish that distinction. Design a targeted question or inspect appropriate collected records.
If the team needs a subscription tier, translate selected benefits into clear inclusion levels and displayed prices for a separate offer-choice study. A relative rank cannot be converted directly into dollars. A high-priority function may already be expected in every acceptable plan.
MaxDiff vs conjoint
MaxDiff prioritizes items within a list; conjoint analysis estimates tradeoffs among complete offers, including price when the design varies it. Use the four meal-planning benefits above to distinguish a priority task from a feasible paid-package choice.
Example: prioritizing a meal planner or pricing its tiers
A hypothetical meal planner is considering a shopping list, saved weekly plan, preparation-time filters and household sharing. MaxDiff repeats small subsets and asks which benefit is most and least valuable. Conjoint varies complete offers and asks what the buyer would choose.
MaxDiff and conjoint ask buyers different questions
Priority task
From shopping list, weekly plan and time filters, select most and least valuable for dinner planning.
Configuration task
Attribute | Complete offer comparison |
|---|---|
Planning | Offer A: Weekly plan Offer B: Weekly plan Offer C: Weekly plan |
Shopping list | Offer A: Included Offer B: Included Offer C: Not included |
Household sharing | Offer A: Not included Offer B: Included Offer C: Included |
Monthly price | Offer A: $5 Offer B: $9 Offer C: $5 |
Choose a complete plan, or none. Weekly planning is present in every plan.
The two tasks answer different questions. The configuration card compares complete plans; it does not show customer outcomes or a measured price premium.
The illustrated $5/$9 task holds weekly planning constant. It can compare the displayed plans, but cannot estimate the separate contribution of weekly planning. A liked feature is not automatically worth the price of the bundle that contains it. This single illustrative card does not independently identify all feature and price effects; that requires a designed set of varied tasks.
The result should change a specific product choice
Question | Method comparison |
|---|---|
Which benefits should we investigate? | MaxDiff can contribute: Relative priority of tested items Conjoint can contribute: Tradeoffs within designed offers |
What configuration might appeal? | MaxDiff can contribute: Input to deciding candidate features Conjoint can contribute: Simulated preference for feasible configurations |
How does price affect the offer? | MaxDiff can contribute: Not answered by feature priority alone Conjoint can contribute: Estimated sensitivity within tested price levels |
What will our actual sales be? | MaxDiff can contribute: Not established Conjoint can contribute: Not established by preference shares alone |
A priority result narrows prototype investment. A configuration result helps select a viable tier or product. Both are relative to their supplied alternatives and context. Neither preference output alone forecasts actual sales.
The MaxDiff counts in this guide are illustrative, while our conjoint simulator uses assumed coefficients. Their scales describe different calculations; do not compare them as customer results.
Feasible paid tiers, uncertain configuration: study complete offers at the intended prices. Include rejection when realistic and the alternatives buyers actually consider. Keep cost and operations beside estimated preference.
A benefit must become a feature the team can supply
Prioritization input | Configuration and evidence |
|---|---|
Shopping-list function is worth investigating | Configuration decision: Included or absent in a realistic plan Evidence still needed: Choices among feasible offers and actual list performance |
Sharing may serve a household subgroup | Configuration decision: Sharing inclusion in a defined tier Evidence still needed: Audience fit and tradeoff with the displayed price |
Saved weekly plan appears useful | Configuration decision: Present across all candidates or varied deliberately Evidence still needed: A study cannot estimate a level held constant in every task |
“Shopping list included” needs a clear mechanism that the team can implement. “Sharing included” needs a defined household/account arrangement. Price needs the same commitment, billing unit and channel across candidates.
Check the design before moving to priced packages
For MaxDiff, check comparable wording, connected comparisons, actual item exposure and what scores mean. For conjoint, ask about feature levels, impossible combinations, answer coding, model estimation and predictions for tasks held out of estimation. In both studies, the audience and response source must fit the decision.
Move from priorities to priced packages only when the next study adds evidence the business needs. If observing the prototype already identifies a repair, finish that first. For a simulated study, agree the audience data, design and validation relevant to the task with the provider.
Make a bounded roadmap decision
For this decision, concentrate the next prototype work on shopping-list consolidation and weekly-plan usability. Investigate sharing among relevant households before removing it. Defer equal investment in all four until feasibility and use are understood.
Keep the exact list, task plan, score definition and response bases with the recommendation. Explain how the chosen work addresses the leading needs and what remains uncertain. If cost or audience differences would change the roadmap, include that trade-off.
Use ranking when a short list can be meaningfully ordered, conjoint when feasible paid configurations are the question, and observed tasks when delivery of the benefit is uncertain. Confirm design and scoring with the research provider; a modeled study needs task-relevant grounding and validation in addition to a clear instrument.
When the decision is a combination of benefits
TURF stands for total unduplicated reach and frequency. It asks which combination of items reaches the most respondents under a stated rule and can use MaxDiff data. Choosing a set requires knowing which preferences overlap within people. The aggregate counts here cannot show whether A and B serve the same households or whether D adds coverage. Declare what counts as reached; that calculation does not establish observed use or sales.
Our TURF analysis guide works through the complete person-by-item records, overlap and launch constraints behind a feasible two-product range. It shows why the highest individual scores and the best feasible set can lead to different choices.
MaxDiff template: copy the counts and analysis
Save the complete CSV as maxdiff.csv and the Python code as maxdiff.py in the same folder, then run python3 maxdiff.py. The standard-library script checks the four-task design, reconciles 400 best and 400 worst selections and prints each net score. The aggregate records do not add respondent-level evidence.
Complete item-count records
item,description,exposures,best_count,worst_count,evidence_type
A,Combined shopping list,300,140,40,fictional_teaching_data
B,Saved weekly dinner plan,300,130,60,fictional_teaching_data
C,Preparation-time filters,300,80,140,fictional_teaching_data
D,Household sharing,300,50,160,fictional_teaching_data
Complete design check and net-score calculation
"""Verify small balanced design and compute fictional net count scores."""
import csv,itertools,collections
from pathlib import Path
tasks=['ABC','ABD','ACD','BCD']
items=collections.Counter(i for t in tasks for i in t)
pairs=collections.Counter(p for t in tasks for p in itertools.combinations(t,2))
assert set(items.values())=={3} and set(pairs.values())=={2}
print('Teaching design:',tasks,'item exposures per respondent:',dict(items),'pair co-occurrences:',dict(pairs))
rows=list(csv.DictReader(Path(__file__).with_name('maxdiff.csv').open()))
assert sum(int(r['best_count']) for r in rows)==400
assert sum(int(r['worst_count']) for r in rows)==400
print('Fictional teaching counts; scores are not estimated utility or probability.')
for r in rows:
n=int(r['exposures']);b=int(r['best_count']);w=int(r['worst_count'])
assert b+w<=n
print(r['item'],round((b-w)/n,6))
For related method choices, consult our MaxDiff-versus-conjoint comparison and our ranking-versus-MaxDiff guide.
Your next prototype depends on which functions solve the buyer's problem. Discuss your concept test to agree the item comparison, audience and evidence the roadmap decision needs.
Related guides: concept testing; why rating scales flatten priorities.
