Why your users don’t trust your AI feature

In recent months, we redesigned the screens of two products we had delivered, a neobank for SMEs and a platform connecting doctors and pharmaceutical companies, imagining what an AI assistant could bring to them. Cash-flow forecasting, suspicious payment detection, automatic invoice reading, verification of healthcare professionals’ registrations. Areas where a mistake is costly and nobody wants to trust blindly.

On every screen, we came back to the same questions. Not “which model should we use”, but: what does the user need to see to decide whether to follow the suggestion? Here are the seven decisions we now make systematically.

1. Say what the AI did, not what it is

“Powered by AI” says nothing useful. Users don’t wonder what technology is running behind the screen; they wonder what happened and whether they need to check something.

Compare “Invoice analysed by AI” with “Read by the copilot in 4 seconds · 5 of 6 fields confident”. The second sentence says what was done, how long it took, and above all where to look. It turns a decorative badge into working information.

It’s also an elegant way to settle the regulatory question: clearly stating that an AI was involved is what is required byArticle 50 of the AI Act. You might as well do it with a sentence that’s actually useful.

2. Show doubt where it exists, and only there

Showing a confidence percentage next to every item looks transparent. In practice, after three screens where everything shows 97%, 98%, 99%, nobody reads the numbers anymore. They become wallpaper, and the day a 62% shows up, it goes unnoticed.

We do it differently. The confidence level remains available, discreetly, for those who look for it. But when an item drops below the threshold where checking is needed, its visual status changes: alert colour, a “To check” label, and a sentence explaining why. In invoice reading, five fields are simply ticked, and the sixth, the IBAN, is outlined in orange with the reason for the doubt.

The right threshold isn’t decided in a meeting. It’s tuned with users, by observing the point at which they start double-checking on their own.

3. Explain in the user’s own professional terms

“Anomaly detected, score 0.87” is a data scientist’s sentence. A finance manager doesn’t know what to do with it. “3.1 times the average of the last 12 payments to this supplier” and “New IBAN registered on 2 October, different bank” are sentences they understand immediately, because those are the signals they would check themselves.

A good explanation doesn’t describe how the model works. It gives users the elements they would have looked for if they had done the job by hand. That’s why writing these explanations can’t be left to the technical team alone: you need someone who knows the user’s line of work, and tests to check that the sentence is understood.

4. Link every suggestion to its source

When the AI extracts an amount from an invoice, we highlight the exact spot on the invoice the amount comes from. When it summarises a document for a doctor before a meeting with a pharmaceutical company, we list the documents used below the summary: summary of product characteristics, French National Authority for Health (HAS) opinion, data sheet provided by the company.

The source does two things. It lets people verify at a glance instead of rereading everything. And it changes the nature of the relationship: the AI no longer asks to be believed, it shows its work. For sensitive topics, health, finance, law, this is often what makes the difference between a feature that gets used and one that gets worked around.

5. Prefer a range to a single figure

A forecast displayed to the cent, “€1,362,418.27 at end of December”, gives an impression of precision nobody can guarantee. The day the actual figure differs by €40,000, the user concludes the tool doesn’t work.

“About €1.36M, between €1.22M and €1.41M” says the same thing, honestly. On a chart, an uncertainty band that widens over time makes it clear in a second that we’re more confident about next week than about December. Paradoxically, showing uncertainty strengthens trust: the forecast lands within the stated range, and the tool told the truth.

6. Give humans the final say, for real

Many interfaces say “you stay in control” and then offer a huge “Accept” button and a tiny grey link to decline. That’s not control, it’s a nudge.

Three rules we apply:

• Declining must be as easy as accepting. On a flagged payment, “Block and verify” and “Approve anyway” are two real buttons, side by side.
• Correcting must be possible without starting over. A misread field is edited in place, and the correction helps improve the next ones.
• Nothing irreversible without approval. The AI prepares payments; it doesn’t send them. It prepares the registration file; the compliance team approves it.

This last rule also protects the product team. An AI error approved by a person is an incident. An error executed automatically is a crisis.

7. Measure trust, not just usage

The number of clicks on the assistant button tells you almost nothing. What matters is what people do with its suggestions. We track three indicators:

• The acceptance rate without edits, which tells you whether the suggestions are good.
• The correction rate, and above all on which fields, which tells you where the model or the interface falls short.
• Re-verification : does the user reopen the source even though everything was marked as confident? If so, your confidence signal isn’t believed.

Even before measuring in production, five interviews with a prototype connected to a real model, with real data, are enough to spot most of these problems. Static mockups lie: an AI that always answers perfectly in Figma prepares nobody for the moment it gets things wrong.

What we no longer do

Finally, a few habits we’ve dropped along the way:

• Putting a sparkle on every button that has anything remotely to do with a model.
• Making chat the default interface. More often than not, a well-placed suggestion in the existing screen beats a conversation.
• Hiding the tool’s limitations on a help page. If the AI can’t read handwritten invoices, we say so right where the invoice is uploaded.
• Animating the response word by word to look “smart” when it’s already ready. The typewriter effect reassures in a demo and irritates in daily use.

None of these decisions requires a better model. They require design time, testing with real users, and a team where design and development work on the same screen at the same time. That, ultimately, is all we try to do at Kapti.

You can see these principles applied on the screens for Titane and Moofize, and in the demos of our internal projects.

Frequently asked questions

Should you show a confidence score for every AI suggestion?

It’s better to keep it discreet and only highlight items that fall below a threshold where checking is needed. A score shown everywhere ends up no longer being read.

How do you test an AI feature before building it?

With a prototype connected to a real model and real data, tested with five users. Static mockups show an AI that is never wrong and don’t prepare you for the moment it is.

Which metrics should you track for an AI feature?

The acceptance rate without edits, the correction rate per field, and re-verification, meaning cases where the user reopens the source even though the suggestion was marked as confident.