LLMs can't save bad UX

A while back I wrote about the AI features users actually want. The short version: stop building chatbots and start building smart defaults. People agreed with it, and then most of them went back to building chatbots.

This post is about what happens next. The AI feature ships and nobody uses it. The LLM works correctly and the product still fails, because the problem was never the model.

The pattern

I've seen this with products I've worked on and with clients who come to me after their AI features underperform:

1. The product has a UX problem. Search is bad, onboarding is confusing, or reporting is manual and tedious.
2. The team adds an AI layer on top. "Ask our assistant to find what you need." "Let AI guide you through setup." "Generate reports with natural language."
3. The UX problem is still there, now with extra latency and less predictability.
4. Users try the AI feature once, get a mediocre result, and go back to working around the original problem by hand.
5. The team concludes that users don't want AI. The real conclusion is that users don't want a slower version of the same problem.

The model did what it was told. It didn't matter, because the problem was upstream in the product thinking.

Three examples

The search that should have just worked

A client had an internal knowledge base with terrible search: keyword matching, no ranking, no typo tolerance. Users hated it. The fix they chose was a "semantic search" chatbot.

The chatbot was good at understanding queries. Ask "how do we handle refunds for enterprise clients?" and it found the right document. But:

• Each answer took 3-4 seconds: embedding, then vector search, then LLM synthesis.
• It answered with a conversational paragraph when users wanted a list of documents.
• Users couldn't scan results. They had to read a generated summary.
• When the summary was wrong, there was no way to browse alternatives.

What they needed was Typesense or Meilisearch: fuzzy search under 50ms with typo tolerance and relevance ranking. A search bar that works, with no generated text.

The AI version was technically impressive and practically worse.

The onboarding wizard nobody asked for

Another client had a complex SaaS product with a 12-step setup flow and a 34% completion rate. Their fix was an AI assistant that walked users through setup in a conversation.

It was well built. It understood context, remembered where you left off, and answered questions about each step.

Completion dropped to 31%.

The problem was never that users didn't understand the steps. The problem was that there were 12 of them. Users didn't need step 7 explained; they needed step 7 gone.

We removed the assistant and collapsed the flow from 12 steps to 4, with smart defaults filling in the rest. Completion went to 78%.

The report generator that generated distrust

This one was in a product of our own. We added a natural language report builder: "Show me revenue by product category for Q3, excluding returns." It demoed well.

In production, users would generate a report, cross-check it against the data tables, find a 2-3% discrepancy from rounding or filter edge cases, and stop trusting the feature. Once a user catches an AI-generated report being slightly wrong, they stop trusting it even when it's right.

Financial users need exact numbers. "Approximately correct" is not a valid state for revenue reporting. We replaced the builder with a guided form that uses AI for one thing: pre-selecting likely filters based on the user's recent activity.

The AI went from doing the whole job to suggesting which buttons to click. Usage went from 8% to 41%.

The latency tax

Every AI feature costs latency. An API call takes 500ms to 5s depending on the model and the prompt. Users feel it.

Google's research found that a 200ms delay in search results reduces engagement. Amazon found that every 100ms of latency costs 1% of sales. Those numbers are in milliseconds, and we're adding seconds and calling it an improvement.

Treat AI latency as UX debt. Every second of LLM time has to be paid back by a proportional improvement in what the user gets. If the AI saves 10 seconds of manual work and adds 4 seconds of waiting, the net gain is 6 seconds. If it saves 2 seconds and adds 4, you've made the product worse.

Most AI features I evaluate fail this test. The latency hits every user on every interaction. The benefit reaches some users some of the time.

When latency is acceptable

When the work happens in the background. If the AI finishes before the user asks for the result, there is no perceived latency:

• A document is uploaded, the AI extracts entities in the background, and the entities are already tagged when the user opens it.
• Data changes, the AI writes a summary overnight, and the insights are waiting on the dashboard in the morning.
• A form opens and the AI pre-fills fields from history, so the user sees a populated form instantly.

Move AI processing out of the user's critical path. If they have to wait for it, it had better be worth the wait.

The accuracy cliff

Traditional software is either correct or broken. A button works or it doesn't. A calculation is right or wrong.

AI features live in a probabilistic middle. Say a feature is right 85% of the time, wrong 10% of the time, and confidently wrong 5% of the time. Users can live with the 10%. The 5% is what kills trust.

When a feature confidently presents wrong information, such as a hallucinated number in a report, the wrong customer name in a generated email, or a wrong date in a summary, the user has to decide whether to trust it or verify everything it produces. If they verify everything, you haven't saved them time. You've added a step.

The escape hatch principle

Every AI feature needs an escape hatch: a way for the user to fall back to manual control without friction. Design it in from the start.

• AI pre-fills a form; the user can edit every field.
• AI suggests a response; the user can rewrite it.
• AI ranks search results; the user can switch to chronological or alphabetical order.

The AI suggests and the user decides. Once the AI decides on its own (auto-send, auto-file, auto-approve) you need 99%+ accuracy, and you probably don't have it.

I killed an AI feature after 4 months because it auto-categorized incoming documents. 92% accuracy sounds good until you realize 8% of documents are misfiled, and some of them matter. Users spent more time checking the AI's work than they would have spent categorizing by hand.

When AI does fix UX

I build AI systems for a living. The features that work share one trait: they reduce cognitive load without adding interaction complexity.

Smart defaults

The highest-return AI feature is almost always pre-populated fields. No new UI, no chatbot, no "AI-powered" badge. The form is mostly filled in when you open it.

On one product, AI-driven field pre-population cut form completion time by 40%. Users didn't know AI was involved. They thought the product had gotten smarter, which is the right experience for AI.

Anomaly surfacing

Don't make users ask for insights. Surface them. "Revenue dropped 18% in mobile checkout after deploy #1234" is useful because nobody had to ask. The AI found the pattern, matched it to a likely cause, and reported it before anyone noticed.

This works because the AI does work the user couldn't easily do: correlating metrics across systems.

Content transformation

Summarization, translation and format conversion are tasks where the input and output are clearly defined and the user can check the result right away. "Summarize this 50-page document" is a legitimate use case because:

• The user can scan the summary and check it against what they know.
• A 90% accurate summary of 50 pages is still more useful than reading 50 pages.
• A few seconds of latency is proportional to saving 30+ minutes of reading.

The latency-to-value ratio passes.

Five questions before adding an AI feature

1. What does the user do today without AI? If the answer is "nothing, this is a new capability," AI might be the right tool. If the answer is "click three buttons," you need to prove AI is faster than three buttons.
2. What happens when the AI is wrong? If the user corrects it in 2 seconds, that's acceptable. If nobody notices until the data is downstream, it's dangerous.
3. Is the latency proportional to the value? 4 seconds to save 30 minutes of reading, yes. 4 seconds for a result keyword search returns in 50ms, no.
4. Does it remove a step or add one? If the user now has to review AI output before acting, you've added a step. The feature has to save more time than the review costs.
5. Would better traditional UX solve the same problem? The question is not whether AI could solve it but whether AI is the best way to solve it. A date picker beats a natural language date parser. Type-ahead search beats a semantic search chatbot. Boring solutions that work beat impressive ones that don't.

The uncomfortable part

Most failed AI features aren't failed AI. They're failed product thinking in an AI costume.

The search was bad before the chatbot, and the chatbot put a natural language layer on top of bad search. The onboarding was too long before the assistant, and the assistant narrated the same 12 steps with more words. The reports were confusing before the generator, and the generator made them probabilistically wrong.

If the UX is broken, fix the UX first. Once that foundation is solid, look for the cognitive work users still have to do and see where AI can take it off their hands.

The order matters. AI on top of good UX is powerful. AI on top of bad UX is an expensive way to make the problem worse.