Open vs closed research questions: discover the obstacle, then measure it
Use open questions to discover experiences and obstacles in respondents’ own terms. Use closed questions to compare defined answers. Decide whether you need one main reason or every obstacle, then carry unexpected accounts into the next tested questionnaire.
The delivery question reveals an unlisted obstacle: the recipient could not reach the collection desk before it shut. A closed list that offers only “could not find the collection point” would turn an hours problem into a directions problem. The questions and responses below are fictional examples of that design choice.
Ask the same event both ways
A fictional delivery company wants to investigate missed deliveries. The following questions and responses are invented examples.
The open answer becomes a different closed choice
“I couldn’t get to the collection desk before it shut.”
Open code: collection access, not unclear location.
Which one best describes the main reason for the missed collection?
- Could not locate the collection point
- Could not reach it during opening hours
- Another reason
- Cannot recall
Worked next-question candidate. Pretest the option meanings; no previous respondent is claimed to have seen this new list.
Format | Question | What it captures | Limitation |
|---|---|---|---|
Open | “Tell me what happened during your most recent missed delivery.” | An episode and the respondent’s explanation | Requires interpretation and coding |
Closed | “Which one best describes the main reason for your most recent missed delivery?” | Selection among defined categories | The provided list shapes the answer |
For the closed question, use options discovered and tested in the intended audience: not home during the window; did not hear the arrival; could not find the collection point; delivery went to the wrong address; another reason; cannot recall. These are illustrative options, not a validated taxonomy.
Define “missed delivery” and the recall period before fielding. If the question concerns delivery attempts that the recipient could not collect, do not combine those with a courier’s failure to arrive.
The open account can preserve a sequence or several obstacles. The closed main-reason item requests one selected category. Neither is universally better; their different outputs should serve different decisions.
Let the unlisted answer improve the next instrument
Fictional response | Code | Reason |
|---|---|---|
“I was on a work call and didn’t hear the buzzer.” | Did not hear arrival | The account reports not hearing the buzzer during a work call |
“The driver left it at number 18; I live at 16.” | Wrong address | Explicit location mismatch |
“I couldn’t get to the collection desk before it shut.” | Collection access | Not the same as an unclear collection location |
“I don’t remember.” | Cannot recall | Keep separate from missing response |
The third response reveals a gap in the proposed closed list. Add and test a collection-hours option if the study needs it. Do not force the response into “could not find the collection point” simply to keep the first codebook unchanged.
Write code definitions, inclusion/exclusion rules and handling of multiple reasons. Retain original text. If a question asks for the main reason, a multi-code analysis should not quietly become a single-choice percentage.
Collection access is the correct code for the closing-time account in this exercise. Add a distinct hours option to a new instrument only after testing its meaning. Do not force the response into a location code to preserve a tidy original taxonomy.
Choose discovery, main reason or multiple obstacles
Ask the unaided open question before showing the reason list so discovery is not merely a repetition of supplied choices. Then decide whether the next comparison needs one main obstacle or every obstacle. A single-main-reason result has one choice per substantive response; a multiple-response design can have overlapping option shares.
Pretest location and hours as separate meanings. Keep another reason and cannot recall where appropriate. Distinguish delivery attempts the recipient missed from a courier’s failure to arrive, and specify the recall period.
Save a new list version. The revised question can measure collection hours going forward; it cannot create a past selected response that the earlier instrument never offered. Preserve open text when the evolution of the categories matters.
Repair a leading question in either format
Original: “Why is our delivery service convenient?”
Open rewrite: “What worked well or poorly in your most recent delivery experience?” This permits an unfavorable account.
Closed rewrite: “How easy or difficult was it to receive your most recent delivery?” Very difficult; Somewhat difficult; Neither; Somewhat easy; Very easy; Cannot judge. This measures perceived ease, rather than the reason for it.
Follow-up: “What made it easy or difficult?” The open explanation helps interpret the rating without assuming a positive result.
For synthetic answers, preserve model provenance and remember that open fluency does not establish actual lived experience. Use the market-research survey guide and analysis guide. Discuss a managed synthetic questionnaire with Sapien when its grounding supports the decision.
The open rewrite permits an unfavorable episode; the closed rewrite measures perceived ease, while the follow-up explains it. The business can use both without treating an explanation, a rating and a main-reason selection as the same metric.
Continue with Pew Research Center’s question-writing guidance.
