Data Quality in Market Research: Which Responses Should You Trust?

Data quality in market research means being able to trace a conclusion to records that are relevant, interpretable and fit for the decision. A large completed sample isn't enough if the wrong people answered, the exposure failed or the cleaning removed genuine disagreement.

Suppose your team is testing a subscription offer. Several responses arrive quickly. One person gives the same rating throughout. Another writes an unclear explanation but supplies a valid unfavorable rating. You need a rule for each case before a supplier's “clean sample” becomes the basis for a launch decision.

Each case needs a supported treatment: keep it, review it, exclude one answer or replace the whole response.

A flag is a reason to investigate

Common quality signals include duplicate submissions, implausible completion times, repeated answer patterns, contradictory eligibility and irrelevant open text. They answer different questions. An exact duplicate can distort a count; a failed product exposure can invalidate a rating; unclear prose may affect only the interpretation of an open answer.

Treating all three as “bad respondents” makes it hard to see what the cleaning actually repaired. Our data analysis guide shows how raw and prepared records should remain separate. Response-quality review adds the evidence behind each treatment.

Pew Research Center's 2026 study compared attention checks, automated prescreening and voter-file matching in an online opt-in survey. Screening improved some measures, but matching also removed valid cases and could increase error. That finding concerns the studied opt-in setting; it doesn't establish a universal cleaning rule for every research source.

The practical implication is to inspect both sides of a rule: the problematic responses it catches and the valid responses it might remove.

Four treatments, with a reason for each

Write the rule before inspecting which treatment produces the preferred result. State what evidence is sufficient, who resolves uncertain cases and what happens when reviewers disagree. A speed flag can trigger a review without becoming an automatic deletion threshold.

Qualtrics' response-quality documentation illustrates why timing needs context: some speed flags depend on the accumulating response distribution. A flag seen midway through fieldwork isn't a fixed fact about a person's legitimacy.

The questionnaire itself may be responsible. If careful participants consistently misread an attention check, rewriting the check is different from deleting everyone who failed it. If a required image didn't load, the resulting uncertainty is an exposure failure, not evidence that the participant was dishonest.

How cleaning can move a decision

Consider an illustrative set of 12 returned records about a meal-plan subscription. Eleven are unique; one record repeats an earlier submission exactly. Six of the 12 raw rows contain a favorable consideration answer. The example is a small accounting exercise, not a market estimate or a completed study.

The review finds these cases:

The remaining records are usable under the same rules. After deduplication, six of 11 unique records are favorable. Removing the ineligible case and the case that fails the declared review rule leaves five favorable answers among nine eligible retained records.

The last line looks more encouraging because an unfavorable answer disappeared. The ambiguous explanation doesn't, by itself, justify removing that person's valid rating. The changed denominator explains the stronger-looking result.

This is why a quality report needs more than the final percentage. It should let you reconstruct what was removed, what remained and whether a different defensible treatment would alter the recommendation.

A supplier return you can inspect

Ask for a compact review ledger rather than a collection of disconnected quality scores. One row per returned record should connect the original identifier, relevant flags, supporting evidence, rule applied, reviewer decision and affected measures. Keep the original values intact and record changes in a prepared dataset.

The summary should reconcile:

• Returned records, duplicate copies and unique records.
• Eligibility failures, whole-case exclusions and item-level missing answers.
• The retained base for each reported measure and audience cut.
• Results under the declared rules and any material alternative treatment.
• Replacements, their recruitment source and whether the original acceptance rules still apply.

A supplier's replacement completes the agreed sample only if it meets the same requirements. Filling the total with a different source or eligibility definition changes the study. Our panel-provider guide explains how to agree what sample you will accept before buying it.

Quality flags can overlap, so don't add their counts to obtain the number excluded. One response might be fast, repetitive and irrelevant; it still represents one case. Likewise, a record retained for consideration may be unusable for a particular open-text analysis. Each result needs its own valid base.

When the right answer is another collection

Cleaning can't recover an offer nobody saw, verify a participant identity from insufficient evidence or make an inaccessible audience appear in the sample. If the failure concerns the research design or recruitment source, a smaller “clean” file may leave the business question unanswered.

For the meal-plan decision, the team should pause if it can't verify that eligible respondents saw the actual service area, price and subscription terms. Recollecting the affected comparison may be more useful than a precise percentage from an uncertain encounter.

It may also be possible to keep a narrower conclusion. A verified subset could explain a specific misunderstanding without estimating its prevalence across all target buyers. Say what that subset can support rather than presenting it as the originally intended market result.

When using a modeled audience in Sapien, identify it as modeled evidence. A synthetic response isn't a human completion that has passed a fraud screen; the source and its validation answer a different question. Our synthetic-versus-human guide helps match the evidence to the decision.

A report that survives its exclusions

The useful endpoint is a decision you can defend after inspecting the removed records. You should know which issues changed the result, which uncertainty remains and whether the supplier repaired the original failure.

If a conclusion depends on deleting a few inconvenient but usable answers, reconsider the conclusion. If it remains useful across justified treatments, the review has done its job: it has made the limits visible without selecting the answer the team hoped to hear.