Overly tidy findings are a signal, not a success metric
Investigate when participants repeat the team's product language, every session confirms the same priority, hypothetical willingness is high but recent examples are sparse, or observers remember praise and not failed workarounds. These patterns do not prove manipulation. Preserve the discussion guide version, recruitment source, session order, interviewer, notes, and available recording before making changes. State the anomaly precisely—for example, 'six participants endorsed the proposed calculator, but only one described calculating anything in a recent purchase.'
Evidence: Pew Research Center; Government Digital Service
Branch causes across sample, guide, moderator, and synthesis
Recruitment bias may select advocates, employees, or unusually experienced users. Guide bias can name the problem, imply the preferred answer, or place concept questions before recall. Moderator bias appears through praise, interruption, selective probing, or inconsistent follow-ups. Analysis bias promotes vivid quotes, merges opinions with actions, or ignores deviant cases. A fifth branch is a genuine repeated problem. Give each branch predicted evidence so 'bias' does not become an all-purpose explanation for results the team dislikes.
Evidence: Government Digital Service; Oxford Academic
Audit the prompt-response sequence
Sample a session and mark who introduced each important word. Flag yes-no prompts, assumptions, forced alternatives, and questions about an imagined future. Note whether the moderator gives more time to supportive answers. Pew's methods work shows that wording and order can affect responses; the Oxford review provides independent historical evidence that form and placement matter. Compare early open recall with answers given after the concept appears. A finding that exists only after prompting belongs in concept feedback, not problem discovery.
Evidence: Pew Research Center; Oxford Academic
Check recent episodes against general claims
For every claimed priority, locate a recent situation, action, workaround, and consequence. If someone says price is always decisive but recounts choosing faster delivery at a higher cost, preserve both statements. The contradiction may reflect context rather than dishonesty. GOV.UK recommends real examples and follow-up questions; use them to identify which circumstance changed the choice. A problem-evidence table should show unsupported generalities and behavior-anchored episodes in different columns rather than averaging them into one confidence score.
Evidence: Government Digital Service
Work a diagnostic example with competing explanations
Imagine most participants say they want a comprehensive product database after the interviewer demonstrates one. Possible causes include genuine unmet need, novelty politeness, a recruited group of power users, or a guide that made the database seem expected. Recode only the pre-demonstration sections for recent comparison behavior, inspect rejected alternatives, and recruit a small contrasting group. If recurring manual spreadsheets appear before the concept, evidence strengthens. If enthusiasm disappears without the demonstration, narrow the claim to reaction rather than discovered need.
Evidence: Government Digital Service; Pew Research Center
Repair one mechanism and rerun a small round
For recruitment bias, add missing experience groups and unsuccessful users. For guide bias, move concepts later and replace implied answers with timeline prompts. For moderator effects, rehearse silence and standardized neutral probes. For synthesis bias, have a second analyst extract observations independently and require a deviant-case row. Change one major mechanism at a time so the result is interpretable. Do not discard the original sessions; retain them as evidence of how the earlier method shaped the conclusion.
Evidence: Government Digital Service; Government Digital Service
Escalate privacy and acknowledge sample limits
Remove or restrict unnecessary personal data, and direct ethical or legal questions to the responsible qualified reviewer. Qualitative interviews can reveal mechanisms and language, but a small purposive sample cannot estimate market prevalence. Close the diagnosis with the confirmed bias sources, evidence against them, remaining uncertainty, the revised guide, and the next decision. If no discriminating check can separate genuine recurrence from elicitation effects, report the ambiguity instead of claiming validated demand.
Evidence: Government Digital Service; Oxford Academic
Measure whether the revised method changes evidence quality
Track the share of important claims anchored to a recent episode, the number of participant terms introduced without prompting, the presence of contradictory cases, and the proportion of findings another analyst can trace to notes. These are process indicators, not proof that the problem is commercially important. If the revised round produces less enthusiastic but more concrete evidence, treat that as improved research quality rather than a failed interview campaign.
Evidence: Government Digital Service; Pew Research Center
Sources and further reading
These references informed this article. A source supports a claim; it does not imply endorsement of TenMultigure or any future product reference.
- Using in-depth interviewsGovernment Digital Service · Accessed August 10, 2026
Provides the neutral-question and recent-example baseline against which moderator and guide deviations are diagnosed.
- Analyse a research sessionGovernment Digital Service · Accessed August 10, 2026
Supports the second-analyst extraction and deviant-case checks used to detect selective synthesis after sessions.
- Writing Survey QuestionsPew Research Center · Accessed August 10, 2026
Documents wording and order effects that can make post-concept enthusiasm look like an independently recalled problem.
- The Effect of the Question on Survey Responses: A ReviewOxford Academic · Accessed August 10, 2026
Adds independent methodological evidence for testing response-shaping causes rather than blaming participant honesty.
Reviewed by TenMultigure Editorial Review. See an error or a source that has changed? Tell the editorial team.
Review method: AI-assisted desk research with editorial checks. Reviewed ; next scheduled review . Built a five-branch interview-bias diagnosis, added prompt-origin coding and a database concept example, and defined single-mechanism repairs with preserved baselines.