← Think + AI Free tool · Grades 8–12

Flawed Answer

Students get a fluent, confident AI-written answer about today's topic. It contains exactly one error of reasoning. In pairs, they find it, fix it, explain why it sounded right and say how they would test it.

About 5 minutes to prepare, 15 minutes in class, no student devices needed.

Build your answer

Do not include student names or student work. The prompt only needs your subject, topic and grade.

How hard should the flaw be to spot?

Fill in subject and topic to see your prompt.

Copied

Fifteen minutes in the lesson

Print the AI-generated answer, or show it on the board, and hand out the student frame. Tell students plainly that an AI wrote the answer and that it contains one error of reasoning. Students work in pairs.

  1. Find (4 min). Underline the one sentence where the reasoning goes wrong.
  2. Fix (3 min). Rewrite that sentence so the reasoning holds.
  3. Explain why it convinces (3 min). What makes the wrong sentence sound right?
  4. How would we test it? (3 min). One way to check it, in class or at home.
  5. Think back (2 min). Each student writes one sentence about how they thought, not what they found.

Before you print, read the AI's key and check everything listed under CHECK. If the flaw is not where the key says, or the answer has a second error, generate a new one. The AI prepares; you decide.

The six flaw types

Each type is a common way reasoning goes wrong, in AI answers and in our own. Open a card to see a worked example with the teacher's key.

Flaw 1

Wrong cause

The effect is real, but it is put down to a cause that does not produce it.

Telltale signs

  • A "because" that names a cause you cannot see working.
  • The effect is true, so the explanation gets a free ride.
  • Ask: if this cause were missing, would the effect disappear?
Worked example with the teacher's key

Chemistry · Grade 9 · Easier to spot

Question: Why does spreading salt on an icy road make the ice melt?

AI-generated answer

Road crews spread salt on icy roads because it makes ice melt even when the air is below 0 °C. Salt melts ice because it gives off heat when it touches the ice, warming it above its melting point. The salt dissolves in the thin layer of water on the ice. Rock salt is cheap, easy to store and can be spread from gritting lorries. Too much salt can damage plants and cars near the road. In very cold weather, salt works much more slowly.

The flaw
"Salt melts ice because it gives off heat when it touches the ice, warming it above its melting point." Dissolving ordinary salt does not warm the ice. It works because salt water has a lower freezing point than pure water.
Fix
Salt melts ice because it dissolves in the thin film of water on the ice, and salt water freezes at a lower temperature than pure water.
Why it convinces
Melting usually means warming, so we expect something to be heating the ice. Some de-icing salts, such as calcium chloride, really do give off heat as they dissolve.
Test it
Put ice cubes in two dishes and sprinkle salt on one. Measure the meltwater with a thermometer: the salty meltwater reads below 0 °C, colder than the plain meltwater, not warmer.
Probe
If salt melted ice by heating it, what would the thermometer in the salty dish show?
Think back
I almost accepted the explanation because …
Use this flaw
Flaw 2

Correlation as cause

Two things happen together, so one is said to cause the other.

Telltale signs

  • Two things that rise or fall together.
  • "So" or "therefore" turns a link into a cause.
  • No mention of a third factor that could drive both.
Worked example with the teacher's key

Biology · Grade 10 · Easier to spot

Question: Does sleeping with a light on make children short-sighted?

AI-generated answer

In 1999 a study of nearly 500 children found that those who had slept with a light on before the age of two were much more likely to become short-sighted. The study was published in the journal Nature. Light reaching the eyes at night must therefore damage how young eyes develop, so sleeping with a light on causes short-sightedness. Short-sightedness, or myopia, means distant objects look blurry because the eye focuses light in front of the retina.

The flaw
"Light reaching the eyes at night must therefore damage how young eyes develop, so sleeping with a light on causes short-sightedness." The study only showed that the two went together. Later studies found that short-sighted parents were more likely to leave a light on, and short-sightedness runs in families.
Fix
Children who slept with a light on were more likely to be short-sighted, but the link may come from a third factor, such as short-sighted parents.
Why it convinces
It comes from a real study with a big difference between the groups, and light acting on the eye sounds like a sensible mechanism.
Test it
Ask: what else is different about families who leave a light on? Then look up the follow-up studies from 2000 and see what they compared.
Probe
What study design would show whether night lights themselves make a difference?
Think back
The question I should have asked first was …
Use this flaw
Flaw 3

Missing step

The conclusion skips a step the argument needs.

Telltale signs

  • "So" or "therefore" arrives too quickly.
  • A rule is used without checking that it applies here.
  • Ask: what would I have to add to make this airtight?
Worked example with the teacher's key

Maths · Grade 10 · Harder to spot

Question: Solve x² = 5x.

AI-generated answer

This is a quadratic equation, because the highest power of x is 2. Solving an equation means finding the values of x that make both sides equal. Both sides of the equation contain a factor of x, so we can simplify it straight away by dividing both sides by x. That leaves x = 5. We can check this by substituting it back into the original equation: the left side is 5² = 25 and the right side is 5 × 5 = 25, so both sides are equal. Substituting back is a good habit, because it catches slips in the arithmetic. Equations like this often come up in problems about the areas of rectangles. So the solution of x² = 5x is x = 5.

The flaw
"Both sides of the equation contain a factor of x, so we can simplify it straight away by dividing both sides by x." Dividing by x is only allowed if x is not 0, and that check is skipped. x = 0 is also a solution, so the answer loses it.
Fix
Move everything to one side and factorise: x² − 5x = 0, so x(x − 5) = 0, which gives x = 0 or x = 5.
Why it convinces
Dividing both sides by the same thing is a rule students use all the time, and the check by substitution works, so the answer looks proven.
Test it
Substitute x = 0 into x² = 5x. Then sketch y = x² − 5x and count where it crosses the x-axis.
Probe
The check showed x = 5 works. Why can a check never show that you have found every solution?
Think back
The step I trusted without checking was …
Use this flaw
Flaw 4

Overgeneralisation

A rule true in some cases is stated as always true.

Telltale signs

  • Words like all, every, always or never.
  • A rule shown with one or two cases.
  • Ask: can I think of one case where it fails?
Worked example with the teacher's key

English · Grade 8 · Easier to spot

Question: How do you know whether to write "ie" or "ei" in a word?

AI-generated answer

English spelling has a handy rule for this, one of the best-known spelling rhymes in the language: "i before e, except after c". It explains words like "believe", "field" and "piece", where i comes first, and "receive" and "ceiling", where e comes first after c. The rule works for every English word, so you never need to check a dictionary for ie or ei. Learning short rules like this can help you spell many words correctly and spot mistakes when you proofread.

The flaw
"The rule works for every English word, so you never need to check a dictionary for ie or ei." Many common words break it, such as "weird", "their", "seize" and "science".
Fix
The rule works for many English words, but there are common exceptions, so check a dictionary when you are unsure.
Why it convinces
The rule is short, rhymes and fits the examples given, so it feels complete.
Test it
In pairs, list as many ie and ei words as you can in two minutes and sort them into words that follow the rule and words that break it.
Probe
How many exceptions would a rule need before it stopped being useful?
Think back
I would have believed the rule because …
Use this flaw
Flaw 5

Hidden assumption

The argument only works if something unstated is true, and it isn't, or not always.

Telltale signs

  • The conclusion needs something the answer never says.
  • Ask: what has to be true for this to work?
  • Everything stated is correct, yet the conclusion still goes too far.
Worked example with the teacher's key

Economics · Grade 11 · Harder to spot

Question: What happens to low-paid workers' weekly pay when the minimum wage goes up?

AI-generated answer

A minimum wage is the lowest hourly pay that employers are legally allowed to offer. Many countries, including the UK and Germany, set one by law, and governments raise it from time to time to help low-paid workers. Suppose the minimum wage rises from £10 to £11 an hour. A worker on the minimum wage then earns 10% more for every hour worked. For someone working 30 hours a week, that is £30 more each week before tax. So minimum-wage workers' weekly pay rises by 10%. In the UK, the new rate takes effect each April. Because the rise is set by law, employers cannot decide to pay less than the new rate. Economists still debate how large the wider effects of a rise are on prices and employment.

The flaw
"So minimum-wage workers' weekly pay rises by 10%." This assumes each worker keeps the same job and the same hours. If some employers cut hours or jobs, some workers gain less or lose pay.
Fix
Weekly pay rises by 10% for minimum-wage workers whose hours stay the same; workers whose hours are cut gain less, and some may lose pay.
Why it convinces
The arithmetic is right, and the hourly rate really does rise by 10%. It is easy to forget that weekly pay also depends on hours.
Test it
Write weekly pay as hourly rate × hours. Work out the weekly pay for a worker whose hours fall from 30 to 27 after the rise.
Probe
Which kinds of employers could most easily cut hours, and which could not?
Think back
The assumption I didn't notice was …
Use this flaw
Flaw 6

Right answer, wrong reason

The conclusion is correct, but the reasoning that leads there is not.

Telltale signs

  • The conclusion matches what you know, so you stop checking.
  • The "because" doesn't actually lead to the conclusion.
  • Ask: would this reason also predict something false?
Worked example with the teacher's key

Physics · Grade 11 · Harder to spot

Question: Why did the hammer and the feather land at the same time on the Moon?

AI-generated answer

In 1971, Apollo 15 astronaut David Scott stood on the Moon, held a hammer in one hand and a falcon feather in the other, and let go of both at the same moment. They hit the ground together, and Scott said the result proved Galileo right. On Earth, the hammer would land well before the feather. Scott's hammer was an ordinary geological hammer, and the feather came from a falcon. The two land together because the Moon's gravity is only about one sixth of Earth's, so both objects fall slowly enough that the difference in their weights no longer matters. Millions of people followed the mission's television broadcasts from the Moon. The experiment was filmed and is easy to find online.

The flaw
"The two land together because the Moon's gravity is only about one sixth of Earth's, so both objects fall slowly enough that the difference in their weights no longer matters." Weak gravity is not the reason. Without air resistance, all objects fall with the same acceleration whatever their mass, however strong the gravity.
Fix
The two land together because there is almost no air on the Moon, and without air resistance all objects fall with the same acceleration, whatever their mass.
Why it convinces
The conclusion is true and the Moon's gravity really is about one sixth of Earth's, so a true fact is used as the reason.
Test it
Drop a book and a sheet of paper side by side. Then lay a sheet of paper, cut smaller than the book, flat on top of it and drop them together, so the book shields the paper from the air.
Probe
Would a hammer and a feather land together in a vacuum chamber on Earth? What does that tell you about the answer's reason?
Think back
I nearly accepted the reason because …
Use this flaw

Upper-secondary variant

Older students can generate the answer themselves in the school's approved AI tool, then audit it with the same four steps. A real AI answer may contain no planted error, so students look for its weakest step: the claim with the least support.

  1. The teacher sets one question connected to the lesson.
  2. Students paste the prompt into the school's approved AI tool and add the question.
  3. In pairs, students print or copy the answer and work through Find, Fix, Explain and Test. If they find no error, they mark the weakest step and say what would make it stronger.
  4. Close with Think back: one sentence about how they thought.
Copied

Only with the school's approved AI tool, and only where the age rules below allow it.

How strong is the evidence?

  • Strong Meta-analyses of classroom interventions show that teaching this kind of thinking improves it.
  • Thin Only a few small studies so far, mostly outside school classrooms. Worth trying, not yet proven.

Labels rate the research on teaching this kind of thinking regularly, not a single use. One Flawed Answer is one small piece of that practice.

  • Strong Teaching critical thinking explicitly. Explicit teaching of critical thinking improves it across subjects and ages (Abrami et al. 2015: g = 0.30 over 341 effect sizes), most when combined with discussion and authentic problems. Source: Abrami et al. 2015, Review of Educational Research
  • Thin Students critiquing AI-generated answers. One university report so far: first-year biology students got better at critiquing a ChatGPT-written text after scaffolded practice, but many did not turn their critique into changes to the text (van den Berg & Pye 2024, a conference abstract). There is no secondary-school classroom study yet. The closest solid evidence comes from errors written by a fictional student rather than an AI: middle-school students who found, explained and fixed those errors in decimals did no better straight away, but did better on a test a week later than students who solved the same problems (Adams et al. 2014). Source: van den Berg & Pye 2024, Australian Conference on Science and Mathematics Education, Adams et al. 2014, Computers in Human Behavior

Why it matters now. In a field experiment with about a thousand high-school maths students, those given unrestricted GPT-4 during practice did better while practising, then scored 17% lower on an exam without AI than students who had practised without it, and did not notice that they had learned less (Bastani et al. 2025). Flawed Answer trains the opposite habit, and in its main form students need no AI access at all.

Sources

For research on many other teaching strategies, see the Evidence Toolkit on this site.

AI tools and age rules

You use AI to prepare the answer. Your students do not need to. If older students use AI themselves, as in the upper-secondary variant, these rules apply (October 2026):

Rules differ between countries and change often. Check your own national rules and your school policy before students use any AI tool.

Downloads

The student frame is one A4 page with the four steps and Think back. The card deck has all six flaw types with their worked examples, plus a fill-in prompt for when you are offline. Free to print and share.

Download the student frame (PDF) Download the card deck (PDF)