A common way to design an AI solution is to start with the technology.
“We need an AI agent.”
“Let’s use RAG.”
“Which model should we use?”
The problem is that these questions begin with capability, rather than the problem.
My preferred approach is almost the opposite:
Start with the user's actual input, walk backward through how the problem is solved today, trace the information and decisions involved, and only then determine where AI belongs.
Then, once the problem has been understood, build the solution forward.
In simple terms:
Think backward. Build forward.
1. Start with the raw user input
Begin with what the user actually gives the system—not the cleaned-up version of it that appears in a requirements document.
Consider this message from a parent to a school's front desk:
“Hi, I sent my daughter's transcript request last week. Has it been processed? She needs it for a UK visa appointment on Friday.”
There is more here than “transcript status inquiry.”
The message contains:
- an implicit request for status;
- a student who must be identified;
- a missing request reference;
- a deadline;
- urgency;
- potentially incomplete information.
That messiness matters.
If we reduce it to:
“Parent requests transcript status.”
we have already thrown away information the eventual system may need.
So the first question is:
What does the user actually provide, and what are they trying to accomplish with it?
2. Define the desired outcome
The user's words are not necessarily the desired outcome.
“Can I get a refund?” might appear to be a question requiring an answer.
But the real outcome could be:
Determine whether the customer qualifies, calculate the applicable refund, process it if permitted, and tell the customer what happens next.
That distinction changes the design entirely.
A useful AI system is therefore not necessarily one that answers questions well.
It is one that helps produce the intended outcome reliably.
So before thinking about models, ask:
What does success actually look like for the user?
3. Reverse-trace how a competent human solves it
Now forget AI.
Ask:
If the AI did not exist, how would a competent person solve this request today?
And ideally, observe the actual process, not just the documented SOP.
Our school example might look like this:
Parent's message
↓
Front-desk staff identifies the student
↓
Checks transcript request log
↓
If unclear, contacts the registrar
↓
Registrar checks records
↓
Registrar estimates current processing status
↓
Front desk relays the answer
The interesting part is what happens between the boxes.
Perhaps the request log is usually a day behind.
Perhaps staff know that the registrar keeps the real status in another place.
Perhaps the registrar relies partly on personal knowledge of the current backlog.
That tacit knowledge is part of the real workflow even though it may not appear in the SOP.
If you design from the documented process alone, you may build a beautifully automated version of a process that nobody actually follows.
4. Break the workflow into decisions
“Resolve the request” is too large a unit for system design.
Break the workflow into smaller decisions and actions.
For example:
- Identify the student.
- Identify the request.
- Determine whether the request exists.
- Determine its current status.
- Determine whether the stated deadline requires escalation.
- Decide what information can be given.
- Respond or escalate.
Now each step can be examined separately.
Is it primarily:
- retrieval?
- classification?
- reasoning?
- generation?
- prediction?
- an external action?
This is where the problem starts becoming an engineering problem rather than an abstract “AI use case.”
5. Trace every decision back to its information source
For each decision, ask:
What does the human need to know to make this decision, and where do they get it?
You might discover:
Student identity → Student database
Request status → Transcript tracking system
Processing policy → Internal policy document
Current backlog → Registrar's operational data
Escalation rules → School procedure
This exposes something extremely important:
The model is not the source of truth.
The database may be authoritative for student identity.
The tracking system may be authoritative for request status.
A policy document may define what is allowed.
An API may confirm whether an action actually succeeded.
The AI should reason over these sources rather than inventing facts from its own parameters.
6. Find the data gaps
At this point, compare:
What the human needs
versus
What the system actually has
This often reveals that the biggest problem is not the model.
Perhaps the human knows a request is urgent because they can see a visa appointment date in an email, but the system has no structured deadline.
Or the human knows the registrar's current backlog because someone tells them verbally, but that information exists nowhere the software can access.
You now have a data gap.
There are only a few honest choices:
Capture the missing information.
Integrate the missing system.
Ask the user for the missing information.
Or accept that the system cannot make that decision reliably.
This is much better than quietly asking an AI model to guess.
7. Find the leverage points—and delete bad steps
This is where I would be especially careful.
The goal is not to automate every human step.
Sometimes the right question is:
Should this step exist at all?
Suppose a support agent walks to another office every time they need a status update.
Building an AI agent that automatically walks them through the same dependency is not intelligent automation.
It is automating a symptom.
Perhaps the real solution is to make the operational system authoritative and keep it updated.
Then the unnecessary step disappears.
This leads to a principle I strongly believe in:
Do not automate a broken process before asking whether the process itself should be changed.
Sometimes the best AI design removes work rather than reproducing it.
8. Decide where AI belongs
Only now should AI enter the conversation.
For every step, ask:
Does this actually benefit from AI?
Consider a customer-support workflow:
Interpret "my dashboard is broken"
→ AI can help classify intent
Determine customer account tier
→ Database query
Check whether the service is currently down
→ Status API
Find relevant troubleshooting guidance
→ Retrieval
Decide how to explain the issue
→ AI can help reason and compose
Create a support ticket
→ Deterministic API call
The AI is important—but it is not doing everything.
This gives us a useful rule:
Use AI where ambiguity, language, or reasoning creates leverage. Use conventional software where deterministic logic is better.
The best AI systems are often hybrids.
9. Establish the AI decision boundary
One question deserves to be explicit:
Where does the AI stop?
An AI system may be perfectly capable of producing an answer but still not be authorized to make a particular decision.
For example:
- It may classify a support issue.
- It may retrieve relevant policy.
- It may draft a response.
But a legal complaint, high-value refund, security incident, or uncertain identity match may require human review.
The system should know not only:
“What should I do?”
but also:
“When am I not allowed to decide?”
That boundary is part of the architecture.
10. Choose the technology from the problem
Once you understand the work, the appropriate architecture becomes easier to choose.
A task that transforms known information into language may simply need an LLM.
A task that requires finding organizational knowledge may need retrieval.
A task that requires interacting with business systems may need tool calling.
A multi-step process with dynamic decisions may benefit from an agentic workflow.
A forecasting or fraud-detection problem may be better served by traditional machine learning.
The principle is simple:
The problem should determine the architecture. The architecture should determine the technology.
Not the other way around.
11. Build forward from the reverse trace
We started at the user and walked backward:
User Input
↓
Desired Outcome
↓
Human Workflow
↓
Decisions
↓
Information
↓
Sources
↓
Data Gaps
↓
AI Opportunities
Now we build in the opposite direction:
Data Sources
↓
APIs / Retrieval / Tools
↓
Business Rules
↓
AI Components
↓
Orchestration
↓
Application
↓
User
This is why I think of architecture as a compression of the workflow.
The system should represent the important parts of the process we discovered—not blindly reproduce every human step.
12. Design the failure path as carefully as the happy path
A production AI solution must answer questions such as:
What happens when information is missing?
What happens when sources disagree?
What happens when retrieval fails?
What happens when the model is uncertain?
What happens when the user is asking for something outside the system's authority?
When does a human take over?
Can we audit what happened?
Can we determine what information the system used?
These are not additions to the design.
They are part of the design.
A system that handles the happy path beautifully but fails unpredictably on real-world variance is not production-ready.
13. Test against real inputs, not perfect examples
Once the system is built, go back to where you started: the user's actual input.
Do not test only:
“I want to check the status of my transcript request.”
Also test:
“Hi, I sent it last week. Any news?”
“My daughter needs it for Friday.”
“I sent two requests. I'm asking about the second one.”
“I don't remember the reference number.”
Real users do not consistently speak in the structured language of your requirements document.
The system should be evaluated against the variance that existed in the original problem.
And when it fails, ask:
Which part of our reverse trace did we misunderstand?
Sometimes the answer is the prompt.
Sometimes it is the data.
Sometimes it is the workflow.
Sometimes it is the assumption that the process was well defined in the first place.
14. Measure the outcome, not just the model
Finally, define success in terms of the original problem.
A support AI should not be celebrated simply because it produces fluent answers.
The better question is:
Did the customer's problem get resolved correctly?
Useful measures might include:
Task completion
Did the system actually accomplish what the user needed?
Accuracy
Did it reach the correct conclusion?
Resolution time
Did it reduce the time required?
Escalation quality
Did it recognize cases that required human intervention?
Cost and latency
Did it do all of this efficiently?
The model's benchmark score may matter, but it is ultimately a component of a larger system.
The framework
The entire approach can be reduced to one sequence:
THINK BACKWARD
USER INPUT
↓
DESIRED OUTCOME
↓
HUMAN WORKFLOW
↓
ATOMIC DECISIONS
↓
INFORMATION REQUIRED
↓
AUTHORITATIVE SOURCES
↓
DATA GAPS
↓
AI LEVERAGE POINTS
↓
AI DECISION BOUNDARY
↓
SYSTEM ARCHITECTURE
BUILD FORWARD
This is the mindset I find most useful when designing AI-powered solutions.
Don't begin with:
“What can this model do?”
Begin with:
“What is the user trying to accomplish?”
Then walk backward.
Understand how the work is actually done. Expose the decisions. Trace the information. Find the gaps. Remove unnecessary steps. Decide where AI creates genuine leverage and where conventional software is better.
Then build forward.
Think backward from the outcome. Build forward into the system.
The AI is not the solution by itself.
The solution is the system you design around the problem.