Become an AI Grandmaster: How Chess Reveals How to Get the Best from AI
Reading time: 3 minutes
Imagine teaching someone chess by showing them thousands of games written in the notation that just tells the coordinates of the moving pieces - never explaining the rules, never mentioning that it's a two-player game on an 8x8 grid, or never even telling them it’s a game at all. They’d just see endless sequences like e4 e5 Nf3 Nc6 Bb5 a6 with the instruction to “figure it out”.
Would they learn to play?
While certainly not the most intuitive way to learn chess, this exact nightmare intelligence test was enough for a simple AI system to figure out how to play chess.
The Chess Revelation
Researchers fed a basic language model 16 million chess games from the public Lichess database with no other instructions. Just raw move sequences of moves treated as text, exactly like any other document the AI might encounter.
The results were somewhat surprising.
The model didn't just memorize moves - it unpromptedly developed an internal representation of the chessboard. When researchers examined the neural pathways of the trained model, they discovered the AI had actually constructed a virtual 8×8 grid in its mind and encoded chess rules into its processing layers. It provenly understood how knights jump in L-shapes, how pawns capture diagonally and how castling works.
All this emerged organically from textual pattern recognition alone.
The trained model could play legal moves in positions it had never encountered, demonstrating genuine understanding of chess mechanics derived purely from observing game patterns. What this means is that when deciding (or “predicting”) the next move, the model would consult its internal rule book of how chess is supposed to work in order to print out the correct letters to make the move.
Beyond Autocomplete: The Birth of Understanding
Think of an AI model as a layered cake of processing, each layer building more nuanced understanding from the patterns beneath.
In the human world, where we deal with words and concepts as opposed to chess moves, the early layers of the AI model might connect basic associations - fire with hot or water with wet. Deeper layers transform these into complex concepts: fire as transformation, passion, or destruction. By the final layers, the model has woven rich, interconnected webs of meaning, tapping into cultural references and domain knowledge such as the chemical reactions that cause fire in the physical world.
The chess study, titled "Emergent World Models in Chess-Playing Language Models," proved these systems don't just memorize but they construct internal models of the domains they encounter. These world models allow them to navigate new situations using learned principles rather than brute recall.
The Professional Implications
The way AI develops understanding explains both AI's impressive capabilities and its dangerous overconfidence. The AI has been trained with all the text on the internet and, much like in the chess example, it has built mental models of the rules of law, medicine, marketing, and virtually every professional domain.
Yet here's the crucial distinction: understanding mechanics doesn't guarantee mastery. The chess AI learned the rules but couldn't necessarily solve complex puzzles or execute brilliant strategies. Similarly an AI that has learned to write code is not automatically a good fit for making ethical decisions regarding a self-driving car software.
This creates a professional paradox: AI systems demonstrate extensive and detailed domain knowledge while lacking the judgment, ethics, and accountability that define true expertise. Normally to get to that level of knowledge comes hand in hand with seniority in the profession - the AI is more akin to an intern who’s really, really, good at googling.
Navigating the Knowledge Landscape
Understanding how AI builds its internal models informs how we should interact with these systems. By default the AI models are not searching databases or consulting references - they're applying learned patterns from their constructed world models and memorized content (which it sometimes makes mistakes on, much like a human).
This means:
Framing matters enormously. How you pose questions determines which aspects of the AI's world model activate. Ask about "pharmaceutical applications" versus "alternative medicinal properties," and you'll trigger entirely different knowledge frameworks.
Context shapes everything. The AI's understanding emerges from a plethora of possible patterns, but it can only use a few at a time. You can help the AI by mentioning e.g. principles, books or domains you would like it to use. You can effectively choose which parts of its training you would like the AI to dip into by just mentioning them.
Expertise detection works. These systems can distinguish between casual conversation and professional discourse, adjusting their responses accordingly when you signal your expectations.
The modern AI systems are powerful but, unlike humans, they have understanding without wisdom, knowledge without accountability, and confidence without humility.
Bridging Knowledge and Wisdom
The chess experiment shows us that AI systems can develop remarkable understanding from textual pattern recognition alone. A key question isn't whether they understand - it's whether their understanding aligns with truth and fits the bigger picture. The AI merely learns and doesn’t inherently distinguish between what we humans consider “real” or “good” - and on top of it the AI is sometimes bad at guessing what you want from it.
At Ada Create, we've learned to harness this pattern-recognition power while compensating for its limitations. Our platform works from your approved materials, ensuring the AI's underlying understanding serves your specific knowledge base rather than going off track with its general training.
In the next post, we'll explore why AI systems prefer fabricating plausible answers to admitting ignorance, and how different knowledge systems create the "multiple truths" problem that confounds even sophisticated AI models.
This article was written by Timo Tuominen, CTO at Ada Create. This is part 2 of our 4-part series "Understanding LLM Limitations." Next: "Multiple Truths: Why AI Confabulates Instead of Admitting Ignorance”
"Emergent World Models in Chess-Playing Language Models," - https://arxiv.org/abs/2403.15498
Image created with ChatGPT.