Fast Answers, Slow Checks
Sambo Chea’s practical notes on System 1, System 2, and using AI without giving up engineering judgment.
Thinking, Fast and Slow by Daniel Kahneman offers a useful warning for engineers: our first answer often arrives before our careful reasoning does.
The book describes two modes of thought. System 1 is fast and automatic. It spots patterns, recognizes familiar things, and produces an immediate answer. System 2 is slower and deliberate. It checks evidence, compares options, and works through uncertainty.
Neither is bad. Fast thinking helps us work. Slow thinking stops us from shipping a convincing mistake.
In the AI era, this matters more. AI makes drafts, code, plans, and explanations cheap to generate. It does not make them true. The engineer remains responsible for deciding what matters and verifying what is real.
When a good answer is not enough
This code can look complete:
System 1 says: “It is short, readable, and probably fine.”
System 2 asks: what happens when the request is retried? Can two workers charge the same order? What happens if the payment succeeds but saving the order fails?
Clear code is not proof of correct behavior. The same is true when AI explains a bug with confidence. Treat the explanation as a hypothesis. Ask what log, test, metric, or small experiment would prove it.
Five traps worth noticing
Familiar feels true
We trust information more easily when it is clear, repeated, or familiar. AI output often has this quality: tidy code and a confident explanation.
Pause before approval. Read the diff, run the relevant test, and check the source when a claim affects security, money, data, or users.
The first proposal becomes the plan
The first option can quietly control the rest of a conversation.
Instead of asking, “Design a Kubernetes architecture,” ask: “Given our traffic, team size, budget, and reliability needs, compare a single service, managed services, containers, and Kubernetes. What is the simplest option that meets the need?”
The question should not choose the architecture before the evidence does.
The dashboard tells only part of the story
If revenue and orders are increasing, it is easy to conclude that the business is healthy. But returns, acquisition cost, support load, margin, or cash flow may be moving in the wrong direction.
What we see is not all that exists. Before deciding, ask what data is missing and who is affected but not represented in the dashboard.
A recent incident feels more likely than it is
After a database outage, every slow request can look like another database outage. Memory is useful, but it is not measurement.
Start with the usual causes in your own data. If most slow requests come from SQL queries, inspect them before adding a new cache or changing the network. Evidence should lead; resemblance should only suggest where to look.
A confident estimate hides uncertainty
“This will take two days” is often an inside view: a picture of the happy path. Check similar work your team has already completed. That outside view usually includes integration, permissions, rollout, review, and unexpected edge cases.
Estimate a range. Name the uncertainty. Reduce it with a small spike or experiment before committing to a date.
Use AI for speed; keep humans for judgment
AI is not literally System 1. The comparison is still useful: it can quickly generate options, spot patterns, draft code, and help explain unfamiliar systems. Humans must own goals, trade-offs, risk, and accountability.
For important work, use three passes:
- Create: ask for a solution and its assumptions.
- Challenge: ask for failure cases, missing context, and simpler alternatives.
- Judge: compare both against requirements, evidence, and the cost of being wrong.
Do not ask AI only to agree with your plan. Ask it to explain why the plan may be unnecessary or unsafe.
A decision loop for engineering teams
Use this before building, changing architecture, or responding to a surprising metric:
Start with: What evidence says this is the problem? Then ask what is missing, what else could explain it, and what is the smallest safe action that teaches us something.
AI can make implementation faster. That is valuable only if we spend the saved time thinking better: define the problem, expose assumptions, keep the design simple, verify the result, and learn from reality.
Keep this close
- Your first answer is a hypothesis.
- A polished explanation can still be wrong.
- Look for missing information and base rates.
- Do not let the first solution become the default.
- Use AI to generate options; use engineering judgment to choose and verify them.
The best teams will not be the ones that generate the most code. They will be the ones that move quickly and know when to slow down.
Reference
- Daniel Kahneman, Thinking, Fast and Slow (2011)