OpenAI CEO Sam Altman continues to praise GPT-5, the latest large language model (LLM) from the company, as nearing human-level intelligence. However, when tested, it often appears surprisingly unintelligent, though verbose.
For example, Gary Smith, an economics professor at Pomona College, recently conducted an experiment that revealed GPT-5 became increasingly confused during a game of “rotated tic-tac-toe.” The game is simply a tic-tac-toe grid rotated once, 90 degrees to the right before starting, which doesn’t change the rules or outcomes.
GPT-5 began with extensive commentary, suggesting that although the mathematical possibilities remain unchanged, a rotation might psychologically affect players’ perceptions of the game.
GPT-5 also incorrectly claimed that starting at the center is the strongest move, despite folk game theory favoring the corners, and further suggested that players might misjudge moves due to rotation.
When Smith asked if the rotation makes the game harder for humans, GPT-5 initially agreed that it’s identical strategically to standard tic-tac-toe but added a verbose and confusing explanation about human perception differences.
This exchange highlights how more verbose and personable GPT-5 has become compared to its initial terseness, likely due to OpenAI’s adjustment to user feedback preferring the friendlier style of its predecessor, GPT-4o.
In Smith’s experiment, GPT-5 made notable errors, conflicting with OpenAI’s claims of the model interacting like a PhD-level intelligent friend.
The situation worsened when GPT-5 attempted to graphically illustrate the rotated game board, resulting in an incomprehensible and error-ridden output.
Smith concluded the experiment, not responding further, remarking on GPT-5’s confidence despite frequent errors, likening it to Sam Altman.
, as approaching to human-level intelligence — but when you actually put it to the test, it...){kind=link}