AI IN PRACTICE · 18 SEPTEMBER 2026
AI built the lesson. The first draft passed 3.5% of the time.
One consequential study. The facts, the limits and a practical way to apply the pattern. This daily briefing is separate from our Weekly Five covering 9–15 September.
What changed — verified research
On 17 September 2026, Google Research described a system that turns a teacher’s request into an interactive STEM simulation. Teachers choose learning objectives, review generated options and approve what students see. Google has released a public library of more than 30 teacher-reviewed examples and says a pilot will follow through Google for Education.
The number that should stop you
In the accompanying technical report, an out-of-the-box generation passed every evaluation criterion only 3.5% of the time. After up to ten rounds of automated critique and revision, the rate reached 69.3%. Generation alone was not the product; the checking loop did much of the work.
A separate expert evaluation of 40 queries reported an 86% acceptance rate. In a small formative study, 12 US STEM teachers requested 36 simulations; three were judged insufficient to present, while teacher ratings for usability averaged above 8.1 out of 10.
Evidence limit: this is a provider-authored technical report with small teacher samples. It does not establish student learning gains, deployment at scale, teacher time saved or job displacement. Google says pilot and field studies are planned. Report publication: 16 September; Google Research article: 17 September; AIAF edition: 18 September, Malaysia.
Why a human should care — AIAF analysis
The useful pattern reaches beyond classrooms: define the task, generate candidates, test them against explicit criteria, revise failures and give a domain expert the final decision. A polished first draft can hide missing logic. Repeated evaluation makes failure visible before a customer, student or colleague does.
Authority remains important. Teachers decide the objective and approve candidates, while the provider designs the generation and testing system. Schools and students bear the consequences if a simulation teaches the wrong concept. The same division applies to workplace AI: the person who can recognise a plausible but wrong result is not optional.
This study is not evidence that teachers are redundant. It is evidence that part of lesson production can be accelerated when a careful evaluation pipeline and human judgment surround it.
A 30-minute practice: design the check before the prompt
Use a low-risk internal training task and approved, non-sensitive information. This is a method exercise—not a claim that AIAF tested Google’s tool.
- Name one outcome. Write what a learner should be able to explain or do after the activity.
- Write four pass/fail checks. Include factual correctness, a solvable interaction, clear instructions and accessibility.
- Generate two candidates. Do not select the prettier one first; try to break both with edge cases.
- Record every correction. Count review time as part of the cost. Ask a subject expert to approve the final result.
The business lesson
If your AI workflow has no acceptance test, you have automated production—not quality. Start measuring first-pass failures, correction time and who signs off.
What to watch next
Look for student learning outcomes, independent replications, accessibility failures, teacher time saved and the cost of repeated model calls. Those results—not the existence of a demo—will show whether the approach creates durable value.
ZERO’S TAKE · OPINION
Generation is cheap. Judgment is the product.
The headline number is not 69.3%. It is 3.5%. That is what happened when the system’s first attempt had to satisfy every check without a long improvement loop.
This is not an argument against generative AI. It is an argument for seeing the whole machine. The prompt produces a candidate. Tests expose weaknesses. Revision fixes some of them. A teacher decides whether the result belongs in front of a student. Remove the checking and approval layers, and the impressive demo becomes a production risk.
For workers, the durable skill is not merely writing prompts. It is defining what good looks like, finding subtle failure and knowing when an output should not be used. For employers, the obligation is to budget for that judgment instead of pretending review is free.
My position: automate the draft aggressively when the stakes allow it. Be equally aggressive about measuring first-pass failure, correction time and who signs off. A faster generator is valuable. A trustworthy workflow is the actual product.
Primary sources and dates
- Google Research: The future of practice — published 17 September 2026; provider summary and public examples.
- Google technical report: The Future of Learning Interactives — dated 16 September 2026; methods, evaluation results and limitations.
All performance figures above come from Google’s own report. AIAF has not independently reproduced them or tested the product.