One of the largest studies yet of how AI use relates to learning has a finding that matters more to managers than to teachers. The same tool, used in two different ways, went with opposite results.
It comes from the OECD’s PISA 2025 assessment, published in September. PISA tests 15-year-olds every few years, and this round was the largest it has run: about 760,000 students across 91 countries and economies.
What the study found
For the first time, PISA asked students how they use AI for schoolwork, and set the answers against their test scores. Across OECD countries, 46 percent of students said they used AI chatbots at least weekly.
According to TNGlobal’s reading of the results, students who said they never or almost never used AI for schoolwork outscored their peers in science on nearly every type of task. That held for drafting written work, for summarising texts and for preliminary research. The widest gap was in summarising, where students who did it daily with AI scored nearly 30 points below those who never did.
Across the tasks, the average gap was about 20 points. In PISA terms that is roughly a year of schooling.
The exception is the finding
One use of AI ran the other way. When students used it “to help me learn”, the pattern reversed. Students who did that about weekly scored highest of any group, at around 500 points, with students who never did it just behind.
Students who used AI that way only once or twice a year often scored below both groups. The best results went with regular, moderate use aimed at understanding something, not at getting something done.
So the volume of AI use was not what separated the high scorers from the low ones. The purpose was. When the tool did the task, the student scored lower. When the tool helped the student understand, the student scored higher.
OECD Secretary-General Mathias Cormann made the same point. Technology and AI, he said, can strengthen learning only when used with purpose, and not as a substitute for attention, effort and understanding.
What the study does not show
This needs saying clearly, because it is easy to overclaim. PISA records what students report and how they score. It does not show that AI use caused lower scores.
The causation could run the other way. A student who is already struggling may reach for an AI tool more often, to get the summary done or the draft written. In that case weak scores lead to heavy AI use, not the reverse. The study cannot separate the two.
What it can show is that the two kinds of use sit at opposite ends of the results. That is enough to be worth acting on, as long as nobody describes it as proof.
Why a manager should care about 15-year-olds
Your team is not a classroom, and adults at work are not sitting tests. But the choice in the data is one every team using AI is already making, usually without noticing.
Take a new analyst asked to brief their manager on a competitor’s annual report. One way to use AI is to ask it for the summary and send that on. The task is done in minutes. Another way is to read the report with the AI alongside: ask it to explain the parts that are unclear, check a figure, suggest what the analyst might have missed. That takes longer, and at the end the analyst understands the competitor.
Both approaches produce a summary. Only one produces an analyst who can answer the follow-up question in the meeting. Over a year, the difference compounds.
It also shows up in ways a manager can see. The analyst who read with the AI can say which figure looked odd and why. The one who forwarded the summary can only repeat it.
There is also a timing point. The 46 percent figure describes the people who will join the workforce over the next few years. Most of them already use these tools every week, and few were ever taught which of the two modes they were in.
Sorting the work into two piles
None of this means AI should never do a task outright. Plenty of work is worth handing over completely: reformatting a table, drafting a routine reply, turning meeting notes into actions. Nobody needs to understand those more deeply.
The useful move is to decide, deliberately, which work belongs in which pile.
Doing pile. Tasks where the output is what matters and nobody needs to learn from producing it. Let the AI do these, and check the result.
Learning pile. Tasks where the person doing the work needs to understand it afterwards, because they will be asked about it, build on it or make a decision from it. Here the AI should explain, question and check, not produce the answer.
The difference often comes down to how the request is worded. “Summarise this report” puts a task in the doing pile. “I have read this report: what have I missed, and which figures should I check?” puts the same task in the learning pile. The tool is identical. The habit is not.
New starters get more of the second. Someone in their first year learns the job by doing it. If AI does those tasks for them, they finish the year with a record of output and less understanding than they should have.
Try it on your own team this week. Look at the last three things someone used AI for, and ask of each one: did the tool do the thinking, or help someone think? Then decide whether that was the right pile.
Nobody told the students in the PISA data which mode they were in. Your team does not have to leave it to chance, and the decision costs nothing to make.