Old Science, New Machines: What a Century of Learning Research Says AI Should Actually Do

The brain did not get an upgrade when ChatGPT launched. That obvious fact is the most underused insight in education technology. We have more than a century of research on how humans actually learn — and most of it points to a conclusion the AI industry keeps rediscovering the expensive way: the machine should make students think more, not less.

Before buying, piloting, or prompting anything, school leaders should know what the science already settled. It turns out to be a remarkably precise specification for what classroom AI ought to do.

Testing beats studying — and students don't believe it

In 2006, psychologists Henry Roediger and Jeffrey Karpicke at Washington University in St. Louis ran an experiment that should have reorganized every school on earth. College students learned short science passages. One group read a passage four times. Another read it once, then spent the remaining time writing down everything they could remember — retrieval practice, no peeking.

Five minutes later, the re-readers looked like winners: 83 percent recall against 71. They felt like winners too, confidently predicting they would remember more later. A week later the result inverted: the re-readers produced 40 percent of the passage; the testing group produced 61. The students who studied the most remembered the least, and the students who felt they had learned the most had learned the worst.

This is the testing effect, now supported by hundreds of studies, and it keeps beating everything schools actually do. In a 2008 follow-up, students who quizzed themselves on vocabulary remembered 80 percent of the words later; re-readers remembered about a third. In 2011, Karpicke and Purdue colleague Janell Blunt published a study in Science showing that retrieval practice outperformed concept mapping — teachers' beloved diagramming technique — even on inference tests, even when the final exam was itself a concept map. A landmark 2013 review in Psychological Science in the Public Interest rated practice testing as "high utility": one of only two study strategies to earn the top rating, the other being spaced practice. Highlighting and re-reading — students' favorite techniques — were rated among the least effective.

The implication for AI is direct. An AI that generates summaries, explains beautifully, and answers instantly is optimizing for the feeling of learning — the exact feeling Roediger and Karpicke proved is uncorrelated with actual retention. An AI that quizzes, withholds, and demands recall is optimizing for the thing itself.

Spacing, mastery, and the feedback loop

The second settled finding is spacing: distributing practice over time beats cramming, robustly, across a meta-analysis of 839 effect sizes. The optimal gap between sessions scales with how long you need to remember — roughly 10 to 20 percent of the retention interval. For a semester exam, that means revisiting material across weeks, not nights.

The third is mastery. Benjamin Bloom's 1968 insight — students must demonstrate mastery of prerequisites, around 90 percent, before moving on — produced an effect size of a full standard deviation from the feedback-and-correction loop alone, before any tutoring entered the picture. It is the same principle behind the new AI-driven schools' "goes green" thresholds, and it is half a century old.

Together, these three findings describe an ideal learning machine with startling precision: it would test rather than tell, space rather than cram, and refuse to advance the student until mastery is demonstrated — with fast, corrective feedback at every step. That machine is entirely buildable with today's AI. Most products on the market do the opposite: they tell, they cram, and they advance on vibes.

The 48 percent warning

The most important recent data point comes from a PNAS field experiment with nearly a thousand high school math students. A plain GPT-4 assistant raised students' practice grades by 48 percent. Then researchers removed it for the exam — and those students scored 17 percent lower than the control group. The AI had not taught them anything durable; it had performed the work while they watched.

But a second version of the tutor, constrained to give hints instead of answers, largely avoided the harm. Students still had to retrieve, still had to struggle, still had to think. The difference between an AI that damages learning and one that supports it was not the model — it was the pedagogy wrapped around it.

Estonia's national AI program absorbed this lesson at the design stage: its student application, built with OpenAI, was deliberately engineered to guide thinking rather than supply ready-made answers. The country's bet is that the interface philosophy matters more than the model weights. The research agrees.

Five design principles for the AI classroom

Distilled from the science, here is what classroom AI should do:

1. Retrieval first. Every AI interaction should begin with the student producing, not consuming. Configure tutors to quiz before explaining. Start class with five AI-generated retrieval questions on yesterday's lesson — answers withheld until students commit.

2. Hints, never answers. The PNAS result is the bright line. If a tool hands over solutions, it is a homework machine, not a learning machine. Demand hint-based, Socratic behavior from every AI product you deploy — and verify it behaves that way under student pressure, because students will pressure it.

3. Space and interleave. Use AI to schedule cumulative review across weeks and to mix topics rather than blocking them. This is pure software advantage: no teacher can track 30 students' optimal review schedules by hand; a machine can.

4. Close the feedback loop fast. Timely, corrective feedback is among the highest-impact moves in teaching. AI makes instant rubric-aligned feedback possible on every draft and practice set — but keep the teacher as the final judge on anything that matters, because AI feedback is fluent, confident, and sometimes wrong.

5. Preserve desirable difficulty. Learning requires effort; ease is the enemy. Every AI feature should be interrogated with one question: does this make the student think harder or less hard? Retrieval practice feels worse than re-reading and works better. The science has been shouting this for decades.

The specification was already written

There is something clarifying about realizing the blueprint predates the technology. Roediger and Karpicke, Dunlosky, Cepeda, Bloom — none of them knew a chatbot was coming, and all of them described what it should do. Test, don't tell. Space, don't cram. Master, don't advance. Hint, don't answer.

The schools that will get the most from AI are not the ones with the most advanced models. They are the ones that hold every product against the oldest standard we have: does this make the student do the thinking? The science answered that question long before anyone asked it about machines. We should finally listen.

The Two-Hour School Day: Inside the Schools Rebuilt Around AI

In July, the Wall Street Journal documented a quiet exodus: high-earning families — finance, venture capital, tech — pulling their children out of conventional schools in favor of a new breed built around AI-driven academics and explicit life-skills training. At one San Francisco campus, tuition runs $75,000 a year, and parents who could send their children anywhere are choosing the two-hour school day.

The flagship of the movement is Alpha School of Austin, Texas, which has grown from 15 to 50 campuses across the United States in a single year, with new locations in Boston, Chicago, Denver, Atlanta, Nashville, and Miami Beach. Its proposition is stark: compress all academic instruction into about two hours each morning with an AI tutor, then spend the rest of the day on something traditional schools barely attempt — workshops in leadership, entrepreneurship, public speaking, financial literacy, and teamwork.

How the two hours work

Alpha's model rests on two old ideas executed with new machinery. The first is mastery learning: a student cannot advance until demonstrating over 90 percent accuracy, where a traditional school lets a 70 percent pass — meaning 30 percent of the material never learned. The second is personalization at a scale no human faculty can match: diagnostics establish each student's true level on entry (Alpha reports that incoming "A" students often test anywhere from one grade ahead to three behind), and the system assigns targeted lessons until mastery.

A full grade level of material in one subject takes 20 to 30 hours of focused work on the platform, by the company's account. A student two years behind is roughly 40 to 60 hours from grade level — a gap closable in months, not years. When the day's academic work is complete, the interface, in the school's telling, "goes green," and students move to the workshops: fifth graders running a profitable Airbnb, second graders training for and completing a 5K, high schoolers staging a Broadway-style musical.

The platform itself is deliberately not a chatbot. It leans on adaptive algorithms and integrates materials from established providers — IXL for practice, Khan Academy for explanations, Trilogy Software for skill modules — with camera-based tracking to monitor engagement and adjust pacing. Students advance only after hitting mastery thresholds. Adults in the building serve as guides and mentors rather than lecturers.

Founder Joe Liemandt has committed $1 billion to what he calls a full-stack reinvention, with a stated long-term goal of on-device AI running on tablets under $1,000, reaching a billion learners over twenty years. Whether or not that vision materializes, the near-term expansion — dozens of campuses, tuition from $40,000 in Austin to $65,000 elsewhere — signals genuine demand.

The wider movement

Alpha is the largest, but not the only, experiment. Forge Prep, a new middle school in Livingston, New Jersey, drew 600 applications for 34 inaugural seats with a philosophy its founders describe as "creation, not consumption" — and a phone-free building. The through-line across these schools is not really the AI; it is time. By compressing academics, they buy hours for the human capabilities — communication, negotiation, planning, self-management — that conventional schools tend to assume rather than teach.

Public districts are watching and, cautiously, buying. San Francisco Unified has licensed Amira, an AI reading tutor for kindergarten through fifth grade that listens to children read aloud and offers tailored prompts on phonics, decoding, fluency, vocabulary, and comprehension — one-on-one literacy instruction at a scale reading specialists cannot provide alone.

The honest caveats

None of this is settled science, and school leaders should hold two truths at once. First, the design instincts are often right: mastery pacing works, diagnostics beat transcript grades, and explicit life-skills instruction fills a real gap. The research on attention and adolescent development has pointed this way for years.

Second, the evidence base is thin where it matters most. These are private schools serving affluent, self-selected families; claims like "learn up to 10 times faster" or "top 1 percent performance" come from the companies themselves, not independent trials. The Khanmigo experiment — covered elsewhere in these pages — is a standing warning that AI-driven personalization does not automatically produce learning gains, and that engagement is the binding constraint. As districts evaluate AI teaching systems, the central question is the one now being asked in earnest: how should schools evaluate and oversee AI systems designed to educate and influence children?

There are also questions the brochures don't answer. Camera-based engagement tracking raises obvious privacy issues. A model built for motivated, well-resourced families may not transfer to the comprehensive public school down the road. And the deepest value proposition — the workshops, the mentorship, the human formation — is precisely the part the AI doesn't do.

What every school can borrow

You do not need a $65,000 tuition or a billion-dollar backer to take the useful parts. Three ideas travel well:

Start from diagnostics, not transcripts. Alpha's finding — that grades mask enormous variation in actual mastery — is available to any school willing to test for it. A week of diagnostic assessment at the start of the year can redirect months of instruction.

Protect time for the human curriculum. The two-hour day is really an argument about priorities: academics compressed so that formation expands. Even without AI tutors, schools can ask which hours currently produce the least learning and reclaim them for communication, enterprise, and character.

Demand evidence before scale. The districts getting this right — San Francisco's careful licensing, Estonia's research-partnered national rollout — treat AI as a hypothesis to be tested, not a product to be bought. Pilot small, measure honestly, expand only on proof.

The new-model schools may or may not be the future. But they have asked the question every school leader should be asking: if the technology can finally deliver personalization at scale, what should the school day be for? The answer, increasingly, looks less like more instruction — and more like everything instruction was crowding out.

The Tutor That Wasn't: What the Khanmigo Experiment Really Proved About AI in Schools

The most important AI-in-education story of 2026 is not a breakthrough. It is a disappointment — and the rare company that published it anyway.

Khanmigo, Khan Academy's AI tutor, was the great hope of the generative AI education wave: a personal tutor for every child, built by the most trusted name in free learning, designed from the start to do what ChatGPT would not — to guide rather than answer, to tutor rather than tell. When independent researchers ran a randomized controlled trial across 18 middle schools over two years, the results were modest, honest, and deeply instructive.

Students who got the Khan Academy intervention did better in math than students in regular remedial classes. But the gains were no larger than what earlier studies had found from Khan Academy practice sessions with no AI at all. In other words, Khan Academy helped. The AI tutor added nothing.

And the reason, the researchers found, was almost embarrassingly simple: students barely used it. Seeking help remained a choice, and most students declined it most of the time. The tutor sat one click away, and the click rarely came.

The number: 0.14

The study — "One Click Away: AI Tutoring with Khanmigo in a Two-Year School Experiment," an NBER working paper by economists Philip Oreopoulos and Nina Low — focused on students at least a grade level behind in math, given an extra remedial period. Adjusted for a full year of active participation, the gain for below-grade-level students was 0.14 standard deviations.

Context matters. That figure sits squarely in the known range for math practice software, roughly 0.05 to 0.20 standard deviations. It is also measured against a control group that was itself using edtech tools — Waggle, IXL, Zearn, DeltaMath — so it represents the benefit of Khan Academy plus Khanmigo above and beyond alternative digital tools. Respectable. Real. And not, in any sense, a revolution.

To Sal Khan's credit, he did not spin it. In a detailed public response in August 2026, the Khan Academy founder praised the study — which his organization had no hand in designing or running — and confirmed what the data showed: internally, Khan Academy was seeing the same low engagement. He said he did not regret shipping the tutor early, argued the rollout had done students no harm, and explained what the company had learned. This is what serious edtech accountability looks like, and it is rarer than it should be.

What changed: from sidekick to teaching assistant

Khan Academy rebuilt the product around the findings. The 2023 version of Khanmigo lived in a separate tab students had to choose to open — the "one click away" of the paper's title. The new version is woven directly into Khan Academy itself. If a student asks for help before attempting a problem, the assistant is deliberately taciturn: little more than a hint and encouragement. But the moment a student gets something wrong, it appears on its own and works through the problem with them. Schools can also switch the assistant off entirely.

The company has gone further with a reimagined platform and a more proactive classroom assistant it calls Conmigo — less a side tutor, more a central teaching assistant. Early pilots, Khan says, are striking: classrooms that assign year-long units show roughly a 60 percent increase in skills leveled up, and in one pilot, thirty minutes of gamified work produced nearly four times the usual skill gains. These are company-reported pilot figures, not independent trials — but they show a team iterating on evidence rather than marketing.

The medium is not the method

The deeper lesson reaches back forty years. In 1984, Benjamin Bloom reported that one-to-one tutoring lifted students two full standard deviations above classroom peers — the average tutored student outperformed 98 percent of the control group. "Two sigma" became the founding myth of AI tutoring: build the tutor, get the effect.

But the myth skips the fine print. Bloom's studies had three groups, not two, and the mastery-learning group — frequent testing with corrective feedback, no tutor at all — gained a full standard deviation on its own. Half the famous effect came from the testing loop, not the tutor. Later reviews put realistic tutoring effects far lower: a major 2011 synthesis found human tutoring at about 0.79 standard deviations and step-based tutoring software at 0.76 — essentially tied. The honest range for tutoring effects, across the literature, sits closer to 0.3 or 0.4.

Read against that history, the Khanmigo result is not an anomaly. It is the pattern. A 2025 trial by Eedi and Google DeepMind found that human-supervised AI slightly outperformed human tutors alone on novel problems — 66.2 percent to 60.7 percent — but the AI was drafting messages that human tutors approved, 76.4 percent of them with little or no editing. The winning formula keeps appearing: AI extends the teacher; it does not replace the teaching.

And there is a darker data point every school leader should know. In a PNAS field experiment with nearly a thousand high school students, a plain GPT-4 assistant raised practice grades by 48 percent — then exam scores fell 17 percent once it was taken away. The machine had done the learning's job. A version redesigned to give hints instead of answers largely avoided the harm. Engagement without pedagogy is just screen time with better marketing.

What leaders should ask vendors

The Khanmigo story gives school leaders a template for evaluating every AI product that comes through the door:

Ask for the control group. Gains against doing nothing are easy. Gains against good existing tools are the real test. Oreopoulos and Low's control group was using real edtech — that is why 0.14 means something.

Ask about engagement, not just efficacy. The best-designed tutor in the world adds nothing if students don't click. Demand usage data, not just outcome data.

Ask whether the AI increases thinking. Hints or answers? That single design choice separated the PNAS experiment's harm from its help, and it separates Khanmigo's redesign from its first version.

Ask who supervises. The strongest results in the current literature keep a human in the loop — approving, guiding, deciding. "Autonomous" is a cost story, not a learning story.

Khan Academy deserves credit for running toward the evidence instead of away from it. The trial proved something more valuable than a breakthrough: that in education, the honest measurement is the innovation. The schools that insist on it will be the ones whose AI investments actually teach.