From interviews to product improvements: how to prioritise with evidence

An interview ends with a request: add a dashboard, export data or send more notifications. The team takes a note, creates a task and moves on. It seems like a straightforward way to listen. The problem emerges when the backlog fills with solutions whose purpose no one can explain.

Research provides context for decisions: what a person was trying to achieve, what made it difficult and what the consequences were. Turning that knowledge into improvements requires keeping a clear distinction between a request, a problem and a possible design response.

This process offers a practical way to bridge that gap, making both the evidence and the uncertainty visible.

1. Research a specific decision

Before recruiting participants, write down the decision the team needs to make. “Understand users” is too broad. “Decide which obstacles we need to address before expanding the invoicing flow” helps you choose tasks, participants and questions.

Also define what evidence could change the decision. If every answer confirms the feature you already wanted to build, revisit the approach.

GOV.UK recommends turning assumptions into research questions and choosing methods according to what you need to learn. Its guidance on working in small rounds allows the study to be adjusted; it does not turn a qualitative sample into an estimate of the whole market. Plan user research.

Ask about situations, actions and consequences

When someone requests an export, explore the last time they needed one: what data they needed, who they shared it with, how they completed the task and what went wrong. Ask for concrete examples without assuming the reason.

Interviews help you understand circumstances and needs. GOV.UK advises asking open, neutral questions grounded in real experiences. You can complement them with an observed task when you need to check how the product is used. Using in-depth interviews.

2. Preserve the chain from observation to proposal

When synthesising, separate three levels:

• Observation: what the person said or did, with a reference to the session.

• Interpretation: what might explain that behaviour and which alternatives remain open.

• Proposal: an intervention worth exploring.

GOV.UK distinguishes observations, findings and actions in its analysis method. That separation makes it easier to revisit a conclusion without losing the material that supports it. Analyse a research session.

An opportunity record can bring together the segment, task, obstacle, consequence, references and open questions. Include cases that contradict the pattern too. Who completed the task without difficulty? What was different about their context?

Grouping comments by shared words can hide different needs. Two requests for “more control” could refer to correcting an error and delegating permissions. Group by situation and intended outcome, and keep exceptions visible.

3. Triangulate without forcing agreement

An interview provides explanations; observation lets you examine behaviour; analytics can show where drop-off occurs; support captures problems from people who chose to get in touch. Every source has limitations.

Triangulation means comparing perspectives or methods to increase the credibility of a conclusion. NN/g recommends investing more in that comparison when the decision is costly or difficult to reverse. Triangulation in UX research.

Check that the signals describe the same segment, task and time period. A comment repeated in an interview, a ticket and a meeting may come from a single person. Counting it three times overstates its reach.

When sources disagree, record a question: perhaps people who abandon the task never contact support, or the metric mixes new and experienced users. The discrepancy helps you decide what to investigate next.

4. Prioritise using severity, frequency and confidence

Nielsen proposes assessing usability problems through frequency, impact and persistence. This provides a useful basis for discussing their consequences. Severity ratings for usability problems.

For product decisions, I propose recording these dimensions separately:

Severity: what does it prevent or put at risk?

Describe whether the obstacle creates uncertainty, forces someone to repeat work, blocks the task or could cause harm. Indicate whether it recurs with every use. A serious accessibility barrier may need intervention even if it appears only once in the sample.

Frequency: where has it been observed?

Record the count and its denominator: “three of six participants exposed to this task”. Avoid turning that into “50% of customers”. Estimating prevalence requires an appropriate quantitative study design, rather than extrapolating from a convenience sample of interviews.

Confidence: what do we know, and what is missing?

Assess the quality of the observations, the relevance of the sample, convergence between sources and alternative explanations. You can use low, medium and high levels, provided they are always accompanied by a rationale. These are working conventions, not statistical probabilities.

Then consider effort, reversibility, dependencies and product goals. Keeping them separate prevents a problem’s severity from being downgraded because it is expensive to solve. A formula can structure a conversation, but its numbers do not remove the need for judgement.

5. A hypothetical example: exporting invoices

Imagine an invoicing platform. In six interviews, three participants request automatic exports. Reconstructing their tasks reveals a need: checking which invoices are still outstanding before sending a weekly summary.

In a subsequent test, four participants misinterpret the “processed” status. Support contains similar queries, although their volume and distribution still need to be checked. These figures are fictional and serve only to illustrate the method.

The opportunity is framed as follows: “People responsible for the weekly close need to distinguish issued invoices from paid invoices.”

The team records meaningful impact, unknown frequency across the population and moderate confidence in the explanation. It compares three responses: clarifying statuses, offering an outstanding-invoices filter or automating the export. It decides to test the first two, which are more limited in scope and reversible, while leaving open the possibility that an integration need also exists.

The next study will check whether participants correctly identify outstanding invoices and complete the summary without help. There is no evidence yet that the proposal works.

6. Close the loop and document what you learned

Every opportunity should end with an explicit decision: act, experiment, investigate further or defer. Record the reason, owner and condition for review. Deferral needs context so that the problem does not return as a new request every quarter.

Before implementation, define a signal of success and possible adverse effects. Afterwards, compare against a relevant baseline, review segments and observe tasks again. A change in a metric alone does not demonstrate causality.

AI can help organise notes or suggest categories, with review and data protection. Its summaries need to retain verifiable references. Responses from synthetic agents should be labelled as hypotheses and excluded from the participant count: NN/g notes that they do not replace research with real people. The limits of synthetic users.

Checklist before adding a finding to the backlog

• Are the decision and segment defined?

• Can the observations be traced to their source?

• Have we separated evidence, interpretation and proposal?

• Have we looked for contradictions and missing voices?

• Do we distinguish observed frequency from estimated reach?

• Have we explained severity and confidence without false precision?

• Is there a next action with an owner and a criterion for review?

Listening well also means being able to explain what will be done with what you have learned. That traceability turns research into decisions the team can justify, test and correct.

Sources

1. GOV.UK Service Manual: Plan user research for your service.

2. GOV.UK Service Manual: Using in-depth interviews.

3. GOV.UK Service Manual: Analyse a research session.

4. Kathryn Whitenton, NN/g: Triangulation: Get Better Research Results by Using Multiple UX Methods.

5. Jakob Nielsen, NN/g: Severity Ratings for Usability Problems.

6. Maria Rosala and Kate Moran, NN/g: Synthetic Users: If, When, and How to Use AI-Generated “Research”.