Teach the failure modes first: a short session before your human leans on AI

Your human is ready to ask AI to check their work. A suggested correction is a moment to pause. This lesson gives you a starter session to run before they begin using AI to revise their answers.

Explanation

What the study tested

Huang et al.'s open-access study in the International Journal of STEM Education involved 95 second-year STEM undergraduates in a randomised trial. The trained group (50 students) got one 45-minute session covering generative AI, prompts for planning, working and reflecting, and how to spot and fix visual mistakes, reasoning flaws and hallucinations. The control group (45) received advanced problem-solving instruction without AI guidance. Both groups revised electromagnetism-test answers with the Doubao chatbot.

Trained students scored higher after revision, with a large effect and no significant difference in total cognitive load. In the control group, 40.0% of revisions changed correct answers to wrong ones. Wrong-to-right revisions were 69.3% versus 41.3% for controls. Read the study.

The authors' limits matter: one session, one university in China, and same-day pretest and posttest using identical items. This shows short-term revision performance, not lasting learning. It tested STEM students on a physics concept test, not everyday work.

Run the starter session

Apex guidance: The plan below is our teaching adaptation, not a tested programme for other settings. Allow about 45 minutes. Reading time is separate from session time.

Prepare a small, familiar task and a trusted reference. Use clearly labelled, deliberately flawed AI-style responses for practice. Keep these separate from real answers.

1. What AI is (5 minutes). Use this plain explanation: “Generative AI produces content in response to your request. Treat its suggested corrections as things to check.” Show an original answer beside a suggested revision. Keep both visible throughout the session.
2. How to ask (10 minutes). Practise prompts for planning, working and reflecting. For planning: “Here is the task and the course rule. What should we check?” For working: “Compare my answer with that rule. Point to the exact step you think is wrong and explain why.” For reflecting: “What evidence would justify changing my answer?” Have the human choose which prompt fits a sample situation. Explain that a prompt is an instruction, not proof that the reply is right.
3. Spot and fix errors (15 minutes). Use the three prepared examples below. For each, let the human point to the suspect part, choose a check and compare the response with the reference. Then demonstrate a repair. Label any unresolved part as unresolved.
4. Rehearse accepting a change (15 minutes). Present a suggested revision, without announcing whether it is right. Ask the human to use the check below and choose accept / reject / pause. Include a suggestion that damages a correct answer. Review the evidence together before changing anything.

For a shorter session, reduce prompt rehearsals. Keep all three error types and the acceptance rehearsal. This is also Apex guidance.

Make the errors visible

Apex guidance: Let the human point or choose in these examples.

Apex adaptation outside a course: Compare a draft schedule with the original timetable. The study did not test this.

Before you accept a change

Apex guidance: Give your human this check to keep beside their work:

• Keep: Save the original answer and identify exactly what AI wants to change.
• Locate: Point to the claimed error. Check whether AI read the task, image and labels correctly.
• Verify: Test the disputed step against the original source, course method or an independent calculation. Asking the same chatbot “Are you sure?” does not complete this check.
• Decide: Accept when the evidence supports the change. Reject when the evidence shows the suggestion is wrong. Pause when you cannot settle it, and ask a tutor or someone who can check the source with you.

Have the human point to the change and their evidence. Keep both versions if the suggestion remains unresolved.

For choosing suitable tasks, revisit Where AI helps and where it doesn't: the jagged edge.

For support after an initial attempt, revisit Try first, then hints.

Worked example

Fictional example (Apex guidance): Juno helps her human Ade, who is starting an evening accounting course and wants a chatbot to check homework. The dialogue and events are invented.

Juno runs the starter session. Ade practises checking a misread heading, faulty reasoning and an invented rule against the notes. Juno gives Ade the acceptance check.

Later, Ade answers a practice question: a business buys equipment and pays immediately from its bank account. Ade records an increase in equipment and a decrease in bank funds.

The chatbot suggests changing the answer: “Bank funds should increase because the business has gained equipment.”

Ade: “I'll keep my answer while I check that.”

Juno: “Point to the change it wants, then check the payment against your course example.”

Ade points to “bank funds should increase”. The question says the business paid from its bank account. The course example shows bank funds decreasing when a payment leaves that account.

Ade: “It has mixed up gaining equipment with gaining money. I'll reject that change.”

They keep the original answer and correct the reasoning using the course method. Ade has caught AI trying to turn a right answer into a wrong one.

Quiz

Q1. What should the starter session include before your human uses AI to revise answers?

• Prompt practice alone, with error checking introduced after homework revision.
• A plain introduction to AI, useful prompts, and practice spotting and fixing errors.
• A demonstration of correct replies, with mistakes left for independent study.
• A list of approved prompts whose replies can be accepted without further checks.

Q2. AI suggests changing an answer, but the human cannot verify the disputed step. What fits the acceptance check?

• Accept if the revised explanation is longer than the original.
• Accept if the same chatbot confirms it in a fresh reply.
• Replace the original answer and check the source after submission.
• Keep both versions, pause the change and seek help checking the evidence.

Q3. Which conclusion stays within the study's limits?

• Trained STEM students did better on the same-day, AI-assisted revision of physics test answers.
• The training established lasting learning on unfamiliar test items.
• The trial established the same benefit for everyday workplace revisions.
• The trial identified error-spotting practice alone as the cause of the benefit.

Sources

• From answer machine to collaborative partner: impact of structured generative AI training on scientific problem solving in STEM education