Market research surveys: turn a delivery question into a usable decision
Build the survey around a decision and primary measure. Define audience, stimulus, answer rules and valid bases, then pretest meaning and pilot routing. In the delivery example, misunderstanding the cutoff directs a specific copy revision before the next test.
A buyer rates the delivery offer useful while believing changes remain possible during the delivery slot. The real cutoff is the evening before. That favorable answer would approve a promise the business does not supply. The six fictional records, shared with the quantitative guide, show how a short survey can catch the mistake and direct the next sentence.
Define what the survey can authorize
Design choice | What to record | Example for a fictional delivery study |
|---|---|---|
Target population | Geography, experience and exclusions | Adults using grocery delivery in the past month in the defined service area |
Sampling frame | Actual contactable units | Eligible accounts in a stated period, with known coverage gaps |
Recruitment | Invitations, channels and incentives | Document invitation dates and reminders |
Stimulus | Exact text, image, price and conditions | Same two-hour-window proposition for every respondent |
Primary measure | Question, coding and denominator | Correct unaided deadline interpretation among exposed eligible records |
Comparisons | Predefined relevant groups | Recent first-time and repeat users |
Quality rules | Exclusions and review evidence | Duplicates, invalid routing and incomplete exposure reviewed consistently |
Limits | Likely missing experiences | Nonusers and unreachable accounts outside the claim |
The decision is whether the current wording can progress, not whether customers want flexibility in the abstract. The displayed proposition permits a two-hour window chosen at checkout and changes until the evening before delivery. Define correct comprehension before asking: the respondent must identify that evening-before cutoff unaided.
In this six-record demonstration, all six must correctly identify the cutoff for the wording to progress. This is an illustrative gate, not a validated benchmark for a real study.
The question must reveal the deadline unaided
Select a two-hour delivery window at checkout. Changes are allowed until the evening before delivery.
When can you last change the delivery window?
Open response collected before answer choices.
Q3’s fictional answer: “I can change it during the two-hour slot.”
Comprehension: incorrect. Usefulness: 4/5.
The exact offer and one teaching record. A favorable usefulness rating does not answer the comprehension question.
The sampling brief names recent users in a defined service area. Existing accounts can fit an existing-customer question, but miss people who left or never used the service. Decide whether the recommendation concerns that reachable audience or a wider market, then design recruitment accordingly.
Keep the finished questionnaire short for a reason
Place eligibility first, then unaided reactions, then diagnostic choices. Ask one thing at a time. Keep ordinal response scales ordered; vary unordered item presentation where it serves the design. A neutral “Cannot judge” option is different from the middle of an opinion scale.
For this delivery study, freeze the exact fictional card: “Select a two-hour delivery window at checkout. Changes are allowed until the evening before delivery.” This is the same teaching case as the quantitative worked example; its shared data demonstrate the survey workflow here, rather than an independent study.
Use the four-question instrument below only after adult and service-area eligibility have been established in recruitment. Its first question checks recent grocery-delivery use, not age or location.
Order | Exact question or action | Answers and routing |
|---|---|---|
1 | Have you received a grocery delivery in the past 30 days? | Yes; No; Prefer not to answer. Continue Yes only. |
Exposure | Show the fixed card above | Save card version and exposure completion. |
2 | When can you last change the delivery window? | Open text; ask before supplying options or explanation. |
3 | How useful would this arrangement be for your household? | 1 Not at all; 2 Slightly; 3 Moderately; 4 Very; 5 Extremely; Cannot judge. |
4 | What information is missing from this description? | Open text; None is valid. |
This is the complete four-question instrument for this narrow decision. Predeclare correct comprehension as an explicit evening-before deadline; other substantive answers are incorrect, and missing/unanswerable responses remain separate. Ratings 4 or 5 count as top-two usefulness among substantive ratings; Cannot judge is reported separately.
Pretest question meaning with people similar to the intended users. Ask them to restate “window,” “changes” and “evening before,” then explain the information used for their answer. This checks whether the instrument measures the intended cutoff before the software begins collecting more versions of the same misunderstanding.
Keep the response options from supplying the primary answer. The open deadline item comes before any cutoff choices or explanation. Usefulness and missing information have separate jobs: one concerns whether the arrangement fits, the other can reveal what prevents a judgment. Together they tell the team whether to revise words, complete the offer or reconsider the terms.
Test the saved response path, not only the questionnaire
The process pilot checks a different failure: whether the saved record matches the planned encounter. Run eligible, ineligible and cannot-judge paths on the intended device. Inspect the stored card version, open answer and rating code. The cases below should be repaired before fieldwork if they appear.
Pilot problem | Why it matters | Repair before fieldwork |
|---|---|---|
A skipped card still produces a rating | The answer has no verified exposure | Save exposure completion and prevent that route |
“Cannot judge” is stored as 3 | Unanswerability becomes moderate usefulness | Preserve a separate code and exclude it from substantive ratings |
A participant changes the eligibility answer after exposure | The screening basis becomes ambiguous | Preserve route history and apply a stated inclusion rule |
The mobile card hides the deadline | Device presentation changes the proposition | Repair display and repeat the exposure rehearsal |
Keep the four-question core intact unless a planned decision requires more context. A first-time/repeat split, for example, needs a defined period and reliable history; adding the label alone supplies no interpretable comparison.
Keep a fieldwork version log and a record-level reason for excluded or unusable answers. A material change to the deadline during collection is a new exposure. Analyze it accordingly instead of presenting a single percentage whose proposition changed.
Follow a favorable answer to the wrong offer
The six fictional records are Q1: correct/5; Q2: correct/4; Q3: incorrect/4; Q4: correct/3; Q5: incorrect/2; Q6: correct/Cannot judge. Correct comprehension is 4/6 (66.7%). Top-two usefulness is 3/5 (60%); one person cannot judge usefulness.
Q3 says, “I can change it during the two-hour slot.” Its favorable rating describes an incorrectly understood offer. Under the illustrative all-six comprehension gate, the wording fails.
Put the deadline first in the next candidate: “Change your delivery window until the evening before delivery. At checkout, choose a two-hour delivery window.” Retest unaided comprehension under comparable exposure, then use the clearer candidate in a suitable behavioral evaluation. The rewrite has no measured result yet.
Send the writer the exact error and card version, not the 60% usefulness headline. Q3’s answer locates the risk in the cutoff, so the candidate puts it before the checkout instruction. The favorable rating is still useful: it shows why appeal must be read beside the offer the person imagined.
Repeat unaided deadline interpretation under comparable exposure. If the cutoff is now understood but the arrangement fits poorly, the next choice belongs to service terms or household timing. If it is still misunderstood, inspect the phrase and presentation.
Write a survey report the wording team can use
For each chart provide the question, response base and evidence type. Distinguish invited, started, eligible and analyzed records. Preserve partial responses separately from explicit “None” or “Cannot judge.” Report weighting and exclusions if used.
The AAPOR disclosure standards set out information needed to evaluate a survey. A conventional margin of sampling error should not be attached to opt-in or synthetic data without an appropriate, explained inferential basis.
Use the analysis guide to inspect coding, exclusions and denominators. Discuss a managed synthetic survey with Sapien, including the audience grounding and validation required for your decision.
Lead the report with a revision the owner can make: put the cutoff first and retest it. Keep comprehension 4/6 and high usefulness 3/5 beside their original answers and bases. Record invitation and recruitment separately in real fieldwork; the teaching rows contain no invitation funnel.
Once the offer is understood, a commercial comparison can ask whether it changes ordering. If buyers understand but reject the cutoff, a clearer sentence has completed its job and exposed a service decision. The survey saves effort when it prevents that problem from being mistaken for weak marketing.
Continue with open and closed questions.
