Your Brain Lies to You — Cognitive Biases Everyone Should Know

Department of Cognitive Science | Level: Beginner | Duration: 25 minutes

Objectives

After this lesson, you will be able to:

Name and explain seven major cognitive biases
Recognize each bias in a real-world example
Apply at least one counteracting strategy per bias
Explain why cognitive biases matter for AI governance

1. Why Your Brain Lies

Your brain is not a logic machine. It is a survival machine. Over hundreds of thousands of years, evolution optimized it for speed, not accuracy. The result: a set of mental shortcuts (heuristics) that work well enough most of the time but systematically fail in predictable ways.

These predictable failures are called cognitive biases. They are not signs of stupidity — they affect everyone, including experts. The difference between a naive thinker and a careful thinker is not the absence of bias. It is the awareness of it.

This course covers seven biases that cause the most damage in decision-making, especially in technology and governance contexts.

2. Confirmation Bias

What It Is

The tendency to search for, interpret, and remember information that confirms what you already believe — while ignoring or dismissing information that contradicts it.

Vivid Example

A developer is convinced that Framework X is the best choice. They read five articles praising it and one article criticizing it. Later, they remember the five positive articles vividly but have only a vague memory of the criticism. When asked, they say: “Everything I’ve read says it’s great.” This is not lying. The brain genuinely filtered the information asymmetrically.

How to Counteract It

Actively seek disconfirming evidence. Before making a decision, ask: “What would change my mind?” Then go look for exactly that.
Red team your own ideas. Assign someone (or yourself) the role of finding every reason the idea is wrong.
Pre-mortem analysis. Imagine the decision failed. What went wrong? This forces you to consider the negative case.

Governance Relevance

Confirmation bias is why Demerzel’s tetravalent logic includes the C (Contradictory) state. When evidence conflicts, the system does not silently resolve it in favor of the existing belief — it flags the contradiction for investigation.

3. Anchoring

What It Is

The tendency to rely too heavily on the first piece of information you encounter (the “anchor”) when making decisions, even if that information is irrelevant.

Vivid Example

In a classic experiment, researchers spun a roulette wheel in front of participants. The wheel “randomly” landed on either 10 or 65. Participants were then asked: “What percentage of African countries are in the United Nations?” People who saw 65 on the wheel guessed significantly higher than those who saw 10 — even though a roulette wheel has absolutely nothing to do with the question.

In practice: if someone says “this project will take six months” early in a meeting, every subsequent estimate will orbit around six months, regardless of the evidence.

How to Counteract It

Generate your own estimate before hearing others. Write it down privately, then compare.
Consider the range, not just the point. Ask: “What’s the best case? Worst case? Most likely?” This breaks the single-anchor pattern.
Be suspicious of round numbers. “About a million users” or “six months” are almost certainly anchors, not analyses.

Governance Relevance

Confidence thresholds in Demerzel’s framework (0.9 / 0.7 / 0.5 / 0.3) force explicit calibration rather than allowing a single anchor to dominate. You cannot just say “I’m pretty confident” — you must assign a number that maps to a specific action.

4. Availability Heuristic

What It Is

The tendency to judge the likelihood of something based on how easily examples come to mind, rather than on actual frequency.

Vivid Example

After seeing news coverage of a plane crash, people dramatically overestimate the risk of flying — even though flying is statistically far safer than driving. The plane crash is vivid, emotional, and recent, so it comes to mind easily. The thousands of uneventful flights that day are invisible.

In tech: a team experiences one catastrophic deployment failure. For the next year, they over-engineer every deployment, adding weeks of process to prevent a recurrence — even though the actual failure rate is 0.1%.

How to Counteract It

Ask for the base rate. Before judging how likely something is, look up how often it actually happens. “How many deployments failed last year out of how many total?”
Be suspicious of vivid anecdotes. A compelling story is not data. One vivid example can overpower a hundred quiet successes.
Track actual frequency. Logs, metrics, and records beat memory every time.

Governance Relevance

This is why Demerzel’s policies require evidence-backed belief states (T/F/U/C with probability weights) rather than gut feelings. A governance decision based on “I remember something going wrong” is not acceptable — the belief must be grounded in evidence with explicit confidence levels.

5. Dunning-Kruger Effect

What It Is

People with low skill in a domain tend to overestimate their ability, while people with high skill tend to underestimate theirs. The less you know, the less you know about how much you don’t know.

Vivid Example

A junior developer who just completed an online tutorial announces they can “definitely build a production-ready distributed system.” A senior engineer with 20 years of experience says “I think we can probably build it, but there are several unknowns I’m worried about.” The junior is overconfident because they cannot see the complexity. The senior is cautious because they can.

How to Counteract It

Calibrate against experts. When you feel confident about something outside your expertise, ask someone who actually works in that area.
Track your predictions. Write down what you think will happen, then check later. If you are consistently wrong, your confidence is miscalibrated.
Embrace “I don’t know.” The most dangerous words in decision-making are not “I don’t know” — they are “I’m sure.”

Governance Relevance

The U (Unknown) state in tetravalent logic exists precisely for this. When an agent does not have enough evidence, the correct answer is not a guess — it is “Unknown.” This triggers investigation rather than false certainty.

6. Sunk Cost Fallacy

What It Is

The tendency to continue investing in something because of what you have already invested (time, money, effort), even when the evidence says you should stop.

Vivid Example

You have spent 8 months building a feature. User testing shows nobody wants it. The rational choice is to kill it. But the team says: “We’ve already invested so much — we can’t stop now.” The 8 months are gone regardless. They cannot be recovered. The only question is: “Given where we are now, is this the best use of our next month?” Past investment is irrelevant to that question.

How to Counteract It

Apply the fresh-start test. Ask: “If we were starting from scratch today, with no investment so far, would we choose to build this?” If the answer is no, the existing investment should not change that answer.
Separate the decision-maker from the investor. The person who approved the original investment often cannot objectively evaluate whether to continue. Get a fresh perspective.
Celebrate killing bad projects. Make stopping something a sign of good judgment, not failure.

Governance Relevance

Demerzel’s rollback policy explicitly supports reversal of decisions regardless of prior investment. Article 3 of the constitution (Reversibility) says: prefer reversible actions. The ability to stop and reverse is a feature, not a failure.

7. Status Quo Bias

What It Is

The tendency to prefer the current state of affairs simply because it is current, even when alternatives would be better.

Vivid Example

A team has been using a particular tool for three years. A clearly superior alternative exists — it is faster, cheaper, and better supported. But switching would require learning something new, so the team stays put. The default wins not because it is best, but because it is already there.

How to Counteract It

Reverse the question. Instead of “Should we switch?” ask “If we were using the alternative today, would we switch to what we currently have?” If the answer is no, you have status quo bias.
Quantify the cost of inaction. Doing nothing is not free. Calculate what the current choice costs you in time, money, or opportunity.
Set regular review points. Schedule quarterly reviews of major tool and process choices so that the default is reconsidered periodically.

Governance Relevance

The kaizen policy requires continuous improvement — actively looking for better approaches rather than accepting the status quo. PDCA cycles (Plan-Do-Check-Act) build re-evaluation into the process.

8. Survivorship Bias

What It Is

The tendency to focus on successes (the “survivors”) while ignoring the failures that are no longer visible, leading to false conclusions about what causes success.

Vivid Example

Startup advice articles feature founders who dropped out of college and became billionaires. Conclusion: dropping out of college leads to success! But for every college dropout billionaire, there are thousands of college dropouts working ordinary jobs. You never hear their stories. The successful dropouts are visible; the unsuccessful ones are invisible.

In music: “Just practice 8 hours a day like the greats!” But for every musician who practiced 8 hours and succeeded, many more did the same and didn’t. Practice is necessary but not sufficient — and survivorship bias makes it look like the only variable.

How to Counteract It

Ask: “Where are the ones who didn’t make it?” For every success story, look for the invisible failures that followed the same path.
Look at the full sample, not just the survivors. Studying only successful companies tells you what winners do, not what causes winning.
Be wary of “just do what they did” advice. The full picture includes everyone who did the same thing and failed.

Governance Relevance

Demerzel’s belief-currency policy requires tracking disconfirming evidence, not just confirming evidence. Governance decisions must account for what failed and disappeared, not just what succeeded and remained visible.

9. The Big Picture — Why This Matters for AI Governance

AI agents inherit human biases through their training data, their designers’ assumptions, and their optimization targets. An AI governance framework that ignores cognitive biases is building on a foundation of sand.

Demerzel’s architecture addresses biases systematically:

Bias	Governance Countermeasure
Confirmation bias	C (Contradictory) state forces attention to conflicting evidence
Anchoring	Explicit confidence thresholds prevent anchoring to a single estimate
Availability heuristic	Evidence-based belief states override vivid anecdotes
Dunning-Kruger	U (Unknown) state prevents false certainty
Sunk cost fallacy	Rollback policy + Reversibility article support stopping bad investments
Status quo bias	Kaizen policy mandates continuous improvement
Survivorship bias	Belief-currency tracks disconfirming evidence

Knowing your biases does not eliminate them. But it lets you build systems — human or AI — that compensate for them.

Key Terms

Term	Definition
Cognitive bias	A systematic pattern of deviation from rational judgment
Heuristic	A mental shortcut that enables quick decisions but can produce errors
Confirmation bias	Favoring information that confirms existing beliefs
Anchoring	Over-relying on the first piece of information encountered
Availability heuristic	Judging likelihood by ease of recall rather than actual frequency
Dunning-Kruger effect	Low-skill individuals overestimate ability; high-skill individuals underestimate it
Sunk cost fallacy	Continuing an investment because of past cost rather than future value
Status quo bias	Preferring the current state simply because it is current
Survivorship bias	Drawing conclusions from successes while ignoring invisible failures

Self-Check Assessment

1. A team says “We’ve invested too much to stop now.” What bias is at work, and what question should they ask instead?

Sunk cost fallacy. They should ask: “If we were starting from scratch today, would we choose this project?” Past investment is irrelevant to future decisions.

2. After a major security breach, the team wants to add five layers of security review to every deployment. What bias might be driving this?

Availability heuristic. The vivid recent breach makes the risk feel larger than it is. They should look at the base rate — how many deployments have actually had security issues? — and design controls proportional to the actual risk.

3. You feel very confident about a topic you just learned about last week. What should concern you?

Dunning-Kruger effect. Early in learning, you do not yet know what you do not know. Seek expert feedback, track your predictions, and be open to the possibility that your confidence exceeds your competence.

4. Why does Demerzel use U (Unknown) instead of forcing a True/False answer?

To counteract the Dunning-Kruger effect and prevent false certainty. When evidence is insufficient, “Unknown” triggers investigation rather than a guess. This is more honest and leads to better decisions.

Pass criteria: Can name and define all seven biases, provide a real-world example for at least five, and explain how at least three connect to AI governance concepts.

Research Basis

Cognitive bias taxonomy from Daniel Kahneman, Thinking, Fast and Slow (2011)
Dunning-Kruger effect from Kruger & Dunning, “Unskilled and Unaware of It” (1999)
Sunk cost research from Arkes & Blumer, “The Psychology of Sunk Cost” (1985)
Survivorship bias from Abraham Wald’s WWII aircraft armor analysis
Anchoring experiments from Tversky & Kahneman, “Judgment Under Uncertainty” (1974)
Governance countermeasures map to Demerzel’s tetravalent logic, rollback, kaizen, and belief-currency policies
Belief state: T(0.85) F(0.02) U(0.10) C(0.03)