Designing AI products: control, trust and recovery

A tool summarises a meeting, identifies decisions and suggests tasks. Everything looks right until it assigns a commitment to the wrong person. Can the user spot the error, correct it and stop it from being sent? That sequence reveals as much about the product’s quality as the model’s accuracy.

Designing with AI means deciding what can be delegated, what needs review and how to continue when the result goes wrong. Control, trust and recovery need to be part of the main flow.

The following recommendations combine research and interaction guidelines with practical proposals for product teams. The examples are hypothetical; they do not describe real projects or results.

1. Define the task before choosing the interface

Before designing a chat interface, it helps to write one sentence: “This feature helps this person complete this task, under these conditions.” Then identify which part calls for interpretation and which requires accuracy.

In the summary example, drafting a synthesis and assigning owners are different operations. One allows for variations in style; the other can create incorrect commitments.

Microsoft’s Guidelines for Human-AI Interaction, published in 2019, recommend communicating the system’s capabilities and level of performance. To put this into practice, the team can document:

• What information the feature uses.

• What it produces and what is out of scope.

• Which situations require review.

• What happens after the result is accepted.

A description such as “Creates a draft from the transcript; review names and agreements before sharing” sets concrete expectations. It also makes it possible to check whether the flow delivers what it promises.

2. Adjust the level of control to the consequences

Separate suggesting, preparing and executing

A title suggestion requires little supervision. Sending a message or changing permissions calls for a different level of control. As a design principle, assess each action by its impact, reversibility and scope.

In a hypothetical customer support tool, the system could prepare a reply, show the references it used and let the user edit it. Sending the message would be a separate action, with the recipient and final content visible.

For bulk changes, the review should show how many items will change, which ones and on what basis. A generic confirmation provides less information than a view of the changes.

Offer understandable controls

Google PAIR’s Feedback + Control guidance proposes balancing automation and control, and explaining what effect feedback will have and when.

In practice, “Make it better” leaves too many decisions open. Controls such as “Keep the figures”, “Make it shorter” or “Use only these documents” let people express constraints that can be checked. They should be presented as actual product capabilities, with their limitations, and validated technically.

It also helps to separate three actions: editing this result, saving a preference and reporting a problem. The user should understand which affects only the current document and which changes future experiences.

3. Build trust with verifiable evidence

Useful trust depends on knowing when to rely on the system and when to review its work. PAIR’s Explainability + Trust chapter explores that calibration and the choice of explanations suited to the context.

For a hypothetical document assistant, an answer could include links to the passages consulted and distinguish information found in the sources from generated suggestions. The team needs to check that the references support the claims: displaying links alone does not guarantee support.

A figure such as “92% confidence” needs a definition and validation that the user can interpret. If the product lacks that foundation, it is more useful to communicate a specific limitation: “Only documents from this project were consulted” or “No date was found in the sources.”

Design the review as well as the explanation

Buçinca, Malaya and Gajos studied interventions to encourage reflection in AI-assisted decisions. In their 2021 experiment with 199 participants, these interventions reduced reliance on incorrect recommendations compared with simple explanations, but the most effective interventions received less favourable subjective ratings. The finding belongs to that experimental context; it does not establish a universal formula.

The practical implication is to test where adding effort is worthwhile. For an important decision, checking data before accepting may be useful. For correcting a sentence, the same interruption could be excessive.

4. Design recovery from the first prototype

PAIR’s Errors + Graceful Failure guidance recommends anticipating errors and returning control to the person. An apology is of little use if previous work has disappeared or the only way out is to start again.

A practical approach is to design three paths:

• Correct one part: change a name or figure without regenerating everything.

• Go back: restore the previous version and compare differences.

• Continue manually: complete the task with the usual tools.

In our hypothetical summary, each agreement could be edited separately. If a section of the transcript is missing, the product would identify it, preserve what has already been reviewed and allow information to be added.

Recovery also requires an honest account of the current state. Stopping text generation is not the same as cancelling an external operation. If a message has already been sent, the interface needs to show this clearly and offer the options that are actually available. Promising “Undo” only makes sense when the system can reverse the action.

5. Evaluate the whole journey

Microsoft’s HAX Playbook proposes exploring interaction failures and simulating behaviours before building a complete system. This makes it possible to investigate recovery in early prototypes.

For a test session, prepare results that are correct, incomplete, and plausible but wrong. Observe whether people detect the problem, find the evidence and manage to finish the task.

As an evaluation approach, record:

• Tasks completed to an acceptable standard, including review.

• Total time, from the request to a usable result.

• Errors detected and errors accepted without being noticed.

• Time and steps needed to recover.

• Work lost during correction.

Compare these results with the existing flow. Faster generation does not demonstrate time savings if extensive review is needed afterwards. Break results down by experience and context too: an average can hide significant difficulties.

Checklist before launching an AI feature

• Does the person understand what the feature does and what they need to review?

• Can they anticipate which data will be used and which actions will take place?

• Do higher-impact actions have a proportionate level of review?

• Do the references allow relevant claims to be checked?

• Can one part be corrected without losing the rest of the work?

• Do cancelling, reversing and continuing manually have clear behaviours?

• Does the feedback mechanism explain its scope and when it takes effect?

• Have plausible errors been tested alongside successful results?

• Does the evaluation include the cost of supervision and recovery?

An AI product needs to let people understand what is happening, decide how much to delegate and resolve problems as they arise. Those capabilities are designed into permissions, wording, states and ways out of the flow. They should be treated as product requirements from the start.

Sources

1. Amershi et al. (2019). Guidelines for Human-AI Interaction. CHI 2019, Microsoft Research.

2. Google PAIR. Feedback + Control. People + AI Guidebook; no individual publication date visible.

3. Google PAIR. Explainability + Trust. People + AI Guidebook; no individual publication date visible.

4. Buçinca, Malaya and Gajos (2021). To Trust or to Think: Cognitive Forcing Functions Can Reduce Overreliance on AI in AI-assisted Decision-making. Experimental research.

5. Google PAIR. Errors + Graceful Failure. People + AI Guidebook; no individual publication date visible.

6. Microsoft. HAX Playbook. Design tool; no individual publication date visible.