Likert scales: write a clear item, then read the distribution

A Likert scale combines related agreement items to measure a defined construct, meaning the quality you intend to measure, under a stated scoring rule. A single agreement question is a Likert-type item. Write each item with neutral wording and ordered labels that fit the question. Keep cannot judge, not applicable and missing outside the midpoint. Report the full distribution and valid base before a mean or combined agreement percentage.

Repair the item before interpreting its score

If you are checking whether app instructions help people start a task, the wording must distinguish clarity from trust, price fairness or general liking. The illustrative question repairs and ten-record instructions example below show how to write an item and read its answers.

“The excellent app is simple and trustworthy.”

Remove “excellent” and split the constructs. Ask “How easy or difficult was it to complete the task?” and separately “How much do you trust the app to handle the information described?” Use labels tailored to each question. Ease and trust can move in different directions.

“I don’t disagree that the price is unreasonable.”

Replace the double negative with “How reasonable or unreasonable is the price shown?” Use very unreasonable / somewhat unreasonable / neither / somewhat reasonable / very reasonable, with cannot judge when appropriate. The revision reduces the work required to interpret the direction.

“Rate the product: good, very good, excellent.”

The options supply no neutral or negative answer. Ask a defined question, such as overall impression, and provide very unfavorable / somewhat unfavorable / neither / somewhat favorable / very favorable. Add cannot judge separately if the stimulus is insufficient.

The first item bundles ease with trust and supplies praise before the answer. Someone can complete the task easily while distrusting the described information handling. One five-point response cannot tell which construct they rated. Split the questions, remove the judgment and choose labels suited to each.

The other repairs address direction and range. Double negatives make the respondent decode a linguistic puzzle before rating the price. A scale containing only good, very good and excellent prevents a neutral or unfavorable answer. These changes improve what the response can mean; they are not cosmetic adjustments made after seeing a disappointing score.

Choose labels that answer the actual question

An agreement item asks how strongly someone endorses a specified proposition. Ease asks about difficulty of a task; usefulness asks about benefit in a context. The latter are ordered rating items, rather than agreement items. A single agreement item is also distinct from a multi-item Likert scale. There is no need to turn every question into “I agree that…”.

Use an ordered, understandable response range and save the exact labels beside numeric codes. A balanced bipolar question should permit both directions and a midpoint where relevant. A unipolar usefulness question runs from absence to a high degree of usefulness; forcing a negative counterpart would change the construct.

The midpoint represents a substantive middle answer. Cannot judge can mean insufficient exposure; not applicable can mean the situation does not apply; missing means no usable response was recorded. None is automatically neutrality. Place special states separately from the ordered scale and define how each enters analysis.

Keep categories in their meaningful order. Pew’s question-writing guidance explains how wording and response ordering shape what an instrument collects. Pilot the question with the actual stimulus and audience: a clear scale cannot repair an answerable-looking question about an experience the participant never had.

Reconstruct the instructions result from ten records

The ten response records, copyable CSV and completed worksheet below use “The instructions explain how to start.” There are eight valid agreement ratings, one cannot judge and one missing record. The distribution table shows the entire valid distribution rather than hiding it behind one favorable number.

The same ten records support two different base statements

“The instructions explain how to start.”

Outside the ordered scale: 1 cannot judge; 1 missing

Agreement: 4 / 8 valid ratings = 50%; 4 / 10 records = 40%

The eight valid ratings are distributed as follows:

The table shows shares of the eight valid ratings. Cannot judge and missing remain outside the ordered scale.

Agreement is prespecified here as categories 4 and 5. Three agree and one strongly agrees: four of eight valid ratings, or 50%. Four of all ten records supplied agreement, or 40%. Both calculations are correct for their stated bases; they answer different questions. Always name the base next to the percentage.

Two valid answers disagree and two are neutral. The result is not a universal pass or fail for the instructions. It identifies what the next investigation must explain. The ten-record example illustrates analysis, rather than supplying a generalizable product finding or enough evidence for detailed segment claims.

Use a mean deliberately and keep the distribution

What the mean and median leave out

The coded mean is 26/8 = 3.25, a descriptive summary under equal numeric spacing. It does not establish equal psychological distances between labels. The two middle codes are 3 and 4; their numerical midpoint is 3.5, which is not an answer category. The distribution preserves the actual categories.

The ordered codes are 1, 2, 3, 3, 4, 4, 4 and 5. The convention in the note above is recoverable from those answers. Retain the categories beside a summary so the reader can see the actual responses rather than infer that someone selected an intermediate numeric midpoint.

A combined agreement share discards intensity within the combined categories. A mean compresses both intensity and distribution. Either can aid comparison when wording, coding and bases stay consistent, but preserve the full categories for a decision that depends on resistance or uncertainty.

For the instructions decision, inspect the task and reasons behind disagreement and neutral answers. Verify what the cannot-judge record saw and why the missing record occurred. They may reveal routing or exposure problems, but the category alone does not diagnose the cause. Use the explanation to propose a specific repair, then compare it under the same task.

Align direction without erasing the meaning

Create an item dictionary with exact wording, labels, code direction, special states and the scoring rule. Agreement with “the instructions are confusing” points in the opposite substantive direction from agreement with “the instructions are clear.” A dashboard that treats both high values as favorable will combine contradictory meanings.

For a declared 1–5 reverse-code rule, use 6−original value: 1→5, 2→4, 3→3, 4→2 and 5→1. Apply that mapping only to valid ordered values. Missing, cannot judge and not applicable retain their states; they are not numbers to reverse or zeros to average.

Recoding aligns the direction of an item for a stated analysis. It does not make trust, ease and price fairness the same construct. It also cannot repair a confusing double negative or reconstruct the answer to a different question. Fix problematic wording prospectively and document the break if comparing with earlier waves.

Before combining or comparing, check a few records by hand against the dictionary. A total can look plausible while an item is reversed incorrectly. Save the raw answer, transformed value and rule so the result can be recovered.

Build a score around a defined construct

If several items will form a scale, define the construct and intended use first. Decide what each item contributes, how direction is aligned and how a score is calculated. Specify the minimum usable item set and missing-data handling before collection. Otherwise the analyst can produce favorable averages from different combinations of items.

For example, “understanding how to start” and “understanding what happens next” might contribute to a carefully developed clarity measure. “The price is fair” and “I trust the app” ask different questions. Similar answer formats do not justify putting all four into one clarity score.

For formal measurement, use or develop an appropriate validated instrument and examine relevant reliability and validity in the intended population and context. Changing wording, audience or scoring can change the interpretation. High internal consistency alone does not establish validity: similarly worded items can agree without covering the construct needed for the decision.

The instructions item is a single-rating analysis, not a validated multi-item instrument. A team needing an operational clarity check may reasonably use a direct item plus task evidence. A team making formal measurement claims needs corresponding instrument evidence, rather than adding more agreement statements and renaming the average.

Inspect and copy the source records

All ten response records

The status field is fictional teaching data in every record. Blank codes for Cannot judge and Missing stay blank in the CSV.

Copy the complete teaching CSV

status,record_id,answer,code
fictional teaching data,1,Strongly disagree,1
fictional teaching data,2,Disagree,2
fictional teaching data,3,Neither,3
fictional teaching data,4,Neither,3
fictional teaching data,5,Agree,4
fictional teaching data,6,Agree,4
fictional teaching data,7,Agree,4
fictional teaching data,8,Strongly agree,5
fictional teaching data,9,Cannot judge,
fictional teaching data,10,Missing,

The worksheet keeps the measure, coding, denominator calculations and decision record together. Copy its complete Markdown text into your analysis record if you need to reuse the structure.

Complete analysis worksheet

# Completed Likert-item analysis

This completed example uses N089-likert-teaching-data.csv, wholly fictional teaching data. It illustrates the effect of denominator and coding choices; these results estimate no real audience.

## Measure and coding

Question: agreement that “The instructions explain how to start.”
Ordered codes: 1 Strongly disagree; 2 Disagree; 3 Neither; 4 Agree; 5 Strongly agree.
Cannot judge and missing: retained as separate records, with blank substantive codes.
Evidence status: fictional; no human sampling precision or population claim.

| Analysis step | Completed teaching working | Result |
|---|---|---|
| All records | IDs 1–10 | 10 |
| Valid agreement base | IDs 1–8 with codes 1–5 | 8 |
| Counts for 1 / 2 / 3 / 4 / 5 | 1 / 1 / 2 / 3 / 1 | Total 8 |
| Cannot judge / missing | 1 / 1 | Total 2 |
| Agreement count | Codes 4 and 5: 3 + 1 | 4 |
| Agreement among valid answers | 4 / 8 × 100 | 50% |
| Agreement among all records | 4 / 10 × 100 | 40% |
| Coded sum | 1 + 2 + 3 + 3 + 4 + 4 + 4 + 5 | 26 |
| Coded mean | 26 / 8 | 3.25 under equal-spacing convention |
| Middle ordered observations | Positions 4 and 5 | Codes 3 and 4 |

## Decision record

Interpretation: four agree, two disagree and two are neutral among eight substantive answers; two further records cannot judge or are missing. This does not establish that the instructions are clear.
Missing evidence: coded reasons, exposure and routing; no original response reasons are supplied in this teaching CSV.
Next step: inspect or collect reasons and verify the shown instruction version before selecting a wording change.
The fictional dataset supplies no real owner, recruitment design or population finding.

Finish with an interpretable change

For the instructions case, investigate the task contexts behind disagreement and uncertainty, then repair a specific instruction or exposure problem. Retain exact wording, labels and primary outcome in the follow-up. If they change, disclose the new instrument instead of attributing every score movement to a better product.

For concept or wave comparisons, keep the distribution, valid base and decision-relevant reasons attached to the recommendation. A modeled response distribution also retains its grounding and validation scope; more generated answers do not create a human survey margin of error. The point of the rating is to help choose a defensible next action, not merely to produce a neat score.

For related decisions, consult our guide to leading and double-barreled questions and our synthetic-versus-human-respondents guide.

Bring the decision, audience and evidence needed to a scoped Sapien study discussion.