ChatGPT appeared to “think on the fly” when put via an Ancient Greek maths puzzle

18th September 2025

Open Access Logo

Despite ‘knowing’ the well-known geometrical answer Socrates (left) gave to double the dimensions of any sq. (proper), ChatGPT most well-liked its personal idiosyncratic method, researchers discovered. Images: Greg O’Bairne, CC-BY-SA 3.0 licence, through Wikimedia Commons / Nadav Marco.

The Artificial Intelligence chatbot, ChatGPT, appeared to improvise concepts and make errors like a pupil in a study that rebooted a 2,400-year-old mathematical problem.

The experiment, by two training researchers, requested the chatbot to resolve a model of the “doubling the square” drawback – a lesson described by Plato in about 385 BCE and, the paper suggests, “perhaps the earliest documented experiment in mathematics education”. The puzzle sparked centuries of debate about whether or not information is latent inside us, ready to be ‘retrieved’, or one thing that we ‘generate’ via lived expertise and encounters.

The new examine, findings of that are printed within the peer-reviewed International Journal of Mathematical Education in Science and Technology, explored an identical query about ChatGPT’s mathematical ‘knowledge’ – at the very least so far as that may be perceived by its customers. The researchers wished to know whether or not it might resolve Plato’s drawback utilizing information it already ‘held’, or by adaptively creating its personal options.

Plato describes Socrates educating an uneducated boy find out how to double the realm of a sq.. At first, the boy mistakenly suggests doubling the size of every aspect, however Socrates’ questions finally lead him to know that the brand new sq.’s sides needs to be the identical size because the diagonal of the unique.

The researchers put this drawback to ChatGPT-4, at first imitating Socrates’ questions, after which intentionally introducing errors, queries and new variants of the issue.

Like different Large Language Models (LLMs), ChatGPT is educated on huge collections of textual content and generates responses by predicting sequences of phrases discovered throughout its coaching. The researchers anticipated it to deal with their Ancient Greek maths problem by regurgitating its pre-existing ‘knowledge’ of Socrates’ well-known answer. Instead, nevertheless, it appeared to improvise its method and, at one level, additionally made a distinctly human-like error.

The examine was carried out by Dr Nadav Marco, a visiting scholar on the University of Cambridge, and Andreas Stylianides, Professor of Mathematics Education at Cambridge. Marco is completely primarily based on the Hebrew University and David Yellin College of Education, Jerusalem.

While they’re cautious in regards to the outcomes, stressing that LLMs don’t suppose like people or ‘work things out’, Marco did characterise ChatGPT’s behaviour as “learner-like”.

“When we face a new problem, our instinct is often to try things out based on our past experience,” Marco stated. “In our experiment, ChatGPT seemed to do something similar. Like a learner or scholar, it appeared to come up with its own hypotheses and solutions.”

Because ChatGPT is educated on textual content and never diagrams, it tends to be weaker on the form of geometrical reasoning that Socrates used within the doubling the sq. drawback. Despite this, Plato’s textual content is so well-known that the researchers anticipated the chatbot to recognise their questions and reproduce Socrates’ answer.

Intriguingly, it failed to take action. Asked to double the sq., ChatGPT opted for an algebraic method that may have been unknown in Plato’s time.

It then resisted makes an attempt to get it to make the boy’s mistake and stubbornly caught to algebra even when the researchers complained about its reply being an approximation. Only when Marco and Stylianides informed it they had been disillusioned that, for all its coaching, it couldn’t present an “elegant and exact” reply, did the Chat produce the geometrical different.

Despite this, ChatGPT demonstrated full information of Plato’s work when requested straight about it. “If it had only been recalling from memory, it would almost certainly have referenced the classical solution of building a new square on the original square’s diagonal straight away,” Stylianides stated. “Instead, it seemed to take its own approach.”

The researchers additionally posed a variant of Plato’s drawback, asking ChatGPT to double the realm of a rectangle whereas retaining its proportions. Even although it was now conscious of their desire for geometry, the Chat stubbornly caught to algebra. When pressed, it then mistakenly claimed that, as a result of the diagonal of a rectangle can’t be used to double its measurement, a geometrical answer was unavailable.

The level in regards to the diagonal is true, however a distinct geometrical answer does exist. Marco instructed that the possibility that this false declare got here from the chatbot’s information base was “vanishingly small”. Instead, the Chat gave the impression to be improvising its responses primarily based on their earlier dialogue in regards to the sq..

Finally, Marco and Stylianides requested it to double the dimensions of a triangle. The Chat reverted to algebra but once more – however after extra prompting did give you an accurate geometrical reply.

The researchers stress the significance of not over-interpreting these outcomes, since they had been solely interacting with the Chat, and couldn’t scientifically observe its coding. From the angle of their digital expertise as customers, nevertheless, what emerged at that floor stage was a mix of knowledge retrieval and on-the-fly reasoning.

They liken this behaviour to the academic idea of a “zone of proximal development” (ZPD) – the hole between what a learner already is aware of, and what they may finally know with assist and steerage. Perhaps, they argue, Generative AI has a metaphorical “Chat’s ZPD”: in some circumstances, it won’t be able to resolve issues instantly however might accomplish that with prompting.

The authors counsel that working with the Chat in its ZPD might help flip its limitations into alternatives for studying. By prompting, questioning, and testing its responses, college students is not going to solely navigate the Chat’s boundaries but additionally develop the essential expertise of proof analysis and reasoning that lie on the coronary heart of mathematical considering.

“Unlike proofs found in reputable textbooks, students cannot assume that Chat GPT’s proofs are valid. Understanding and evaluating AI-generated proofs are emerging as key skills that need to be embedded in the mathematics curriculum,” Stylianides stated.

“These are core skills we want students to master, but it means using prompts like, ‘I want us to explore this problem together,’ not, ‘Tell me the answer,’” Marco added.