Monadic concept testing: set up one clean reaction per buyer

A monadic test shows one complete concept to each buyer so you can collect independent first reactions. A sequential test shows several concepts to each person; preserve their order and linked answers. If you need to choose what to develop, a position effect can change the next step even when pooled concept scores look similar. The illustrative A/B comparison below shows how to spot that problem.

Set up a study the team can actually reproduce

Start with two complete product offers, A and B, and a comparable eligible buying audience. Choose whether the decision needs independent appeal or a sequential screen before you assign buyers.

For independent appeal, randomize eligible people to A or B and show one card. Keep the buyer rule, presentation and question block comparable. For a sequence, assign AB or BA and preserve position for both. Plan any critical audience balance within the allocation rather than adjusting mid-fieldwork.

Save planned assignment separately from actual display. A person assigned A who receives B because of a routing error has not supplied the intended exposure. A record of both lets the analyst diagnose the issue instead of relying on the group label.

The unit is the person for independent cells. In a sequential study, two concept evaluations remain linked to one person. Preserving that link is necessary to distinguish independent first-position comparisons from paired concept comparisons.

Fix the block and product version

Use unaided comprehension, relevance, concerns, intent at the displayed price and alternatives. Keep outcome wording, timeframe and labels identical after each card. Define the positive summary before reading results and retain the full distribution.

Pilot the cards and block as one experience. A missing fact should be clarified consistently in the study version. Giving extra explanation only to a confused second card changes the comparison unless that presentation is deliberately the intervention.

Qualtrics’ concept program guidance documents single and sequential setups. Agree which information history your primary result is intended to preserve.

Read the AB/BA comparison before naming a winner

Our four aggregate records and copyable CSV contain counts for 100 people, 50 in each sequence. The same predeclared positive-intent outcome applies throughout.

All four aggregate records

Each row has evidence_type=fictional_teaching_data. The n is the concept-position base; positive is the positive-intent count; positive_share is that count divided by the row’s base.

Copy the complete concept-order CSV

sequence,concept,position,n,positive,positive_share,evidence_type
AB,A,1,50,30,0.60,fictional_teaching_data
AB,B,2,50,21,0.42,fictional_teaching_data
BA,B,1,50,29,0.58,fictional_teaching_data
BA,A,2,50,22,0.44,fictional_teaching_data

The position pattern is larger than the pooled lead

Positive intent by concept and exposure position

Pooled: A 52% · B 50% · 100 people total

Aggregate counts: 50 AB and 50 BA, 100 people total. First/second rates compare sequence groups, not reconstructed person-level transitions.

A has 30 positive first-position answers and 22 second-position answers, pooling to 52 of 100. B has 29 and 21, pooling to 50. Yet A’s rates are 60% and 44%; B’s 58% and 42%. Both lose 16 percentage points when shown second.

That position pattern is more consequential than a pooled two-point lead. Fatigue, anchoring and comparison context are possible explanations; the aggregate records cannot identify which occurred. It also cannot reconstruct how an individual changed their answer.

Choose the follow-up that resolves progression

For this comparison, do not develop A solely because 52% exceeds 50%. Inspect burden, instructions and first-exposure understanding. If progression still depends on independent appeal, run a focused independent-cell follow-up on the remaining viable concepts.

A declared first-exposure analysis can be useful. It compares A-first AB people with B-first BA people, rather than paired answers from one person. Choose it for the research purpose and design, not retrospectively because it favors the team’s preferred concept.

If only one card is unclear second, inspect both its wording and what the earlier card taught. A concept issue and an exposure issue can coexist. The next study should resolve the material uncertainty, rather than repeat the entire screen without a new question.

Preserve a stopped block as missing evidence

If someone completes A and stops before B, keep A and record the missing B exposure or outcome. Do not code it as negative intent. Uneven loss by sequence can create a later-position pattern as well as change the achieved comparison.

Report assigned, exposed and valid-answer counts by sequence and position. Separate a display failure from a skipped optional item and an actual nonpositive response. The reason determines what evidence is available.

Specify the primary analysis and any complete-pair sensitivity in advance. A complete-pair base can differ from the first-exposure base. Show that change instead of suggesting the same group of people supplied every summary.

Keep the joint outcomes needed for paired inference

Save linked outcomes with the identifiers needed to retrieve each answer and display.

These four example rows show the record structure; they do not reconstruct the 100-person aggregate records. Participant ID links outcomes, response ID identifies the answer, exposure ID identifies the display and position records order. Here 1 means definitely/probably would buy, while 0 is another valid intent response. Missing and not applicable stay separate.

For a paired comparison, retain positive to both, A only, B only and neither. The A-only and B-only states determine the within-person disagreement. The two pooled margins do not reveal those joint states.

A mathematical bound makes that missing information concrete. With 52 A-positive and 50 B-positive among 100, the number positive to both could range from 2 to 50. At the first endpoint, 50 are A-only and 48 B-only; at the second, 2 are A-only and none B-only. Both joint arrangements match the same margins. These are possible tables, not reconstructed study outcomes.

The total discordant count therefore could be 98 or 2 at those extremes, despite the identical two-point marginal difference. A paired uncertainty calculation depends on that information. The aggregate records support descriptive totals and position patterns, while leaving that inference unavailable.

Do not treat this as an invitation to select a convenient joint table. Collect and preserve the real links. A model-run identifier should likewise retain its construction and relation to other generated outputs, while remaining distinct from a human participant.

Deliver a recommendation with its exposure record

A useful report states which concept to progress or revise, the supporting independent or sequential outcome, and the exposure issue that affects the decision. It should let another reviewer retrieve the card, block, base and follow-up.

For human studies, sample structure and related responses affect uncertainty. For modeled responses, assignment helps inspect sensitivity but does not create independent human sampling. Task-relevant grounding and validation remain necessary.

Your next decision is a bounded independent follow-up, not an artificial rescue of A’s pooled lead. Our method chooser explains when that cost is warranted; our concept examples guide shows what the resulting evidence should change.

Further reading

• our concept-testing methods guide
• our concept-testing guide

Bring the decision, audience and evidence needed to a scoped Sapien study discussion.