Learning Coach Justin Sung tested ChatGPT Study Mode, and his review revealed some important insights about how learning works with tools like AI. Here’s what we can learn about when — and when not — to use AI in the classroom, and what that means for how educators should maximize the benefit of classroom time.
Terminology
Artificial Intelligence (AI) is a broad term encompassing tools like Machine Learning (ML), Large Language Models (LLMs), and systems like AlphaGo or IBM’s Deep Blue. While Artificial General Intelligence (AGI) remains speculative, the most immediate impact on education comes from generative AI systems like ChatGPT, Claude, Gemini, and DeepSeek. These tools are increasingly used by students and require educators to adapt.
Throughout this series, I’ll use “LLM” and “generative AI” interchangeably, though generative AI can include more than just language models. But these models will be the focus in this series because of their high usage rate among students.
“Writing is thinking. To write well is to think clearly. That's why it's so hard.”
Educators have been navigating this AI transition without a map for nearly two years. Even before the upheavals of COVID and AI, keeping students engaged was a challenge. The shortcuts AI offers make that challenge even greater. It’s a truism that you get out of your education what you put into it, and yet we’re told that AI doing work for us is the future of work. The reality of AI performing students’ work for them creates a double-bind for both educators and students. Educators face the paradox that basic skills are not becoming obsolete; it may be counterintuitive, but the AI-powered future workplace actually accelerates the need for higher-order thinking skills. For their part, students also face a catch-22: to use AI effectively, they need higher-order thinking skills. But those skills depend on mastering the basic ones—skills that AI is aiming to replace.
Learning Coach Justin Sung is a medical doctor who has spent years tutoring students and has developed a learning system for mastering complex subjects and materials. In his review of ChatGPT Study Mode, he offers valuable insight on when AI can support learning and when it can be an obstacle.
Sung divides learners into two categories: basic learners, who are still acquiring core knowledge and can’t yet self-direct, and advanced learners, who can identify their knowledge gaps and think critically about their own thinking—what we call metacognition. This metacognitive threshold determines when AI becomes a useful learning tool. For a more complete explanation, I recommend his video to understand when learners can benefit from using ChatGPT Study Mode.
Following this thinking, we can imagine two separate stages for students to follow as AI becomes more prevalent. The first stage is to acquire core information, context, cognitive skills, and habits. The second stage is to be able to use those skills in collaboration, either with human colleagues or an AI. Sung thinks the threshold for transitioning from the first stage to the second is metacognition. When learners are capable of metacognition, they can benefit from AI tools like ChatGPT study mode.
Stage 1: Basic Learning and Offline Cognition
According to Sung, students who are still working on their vocabulary, core information, and mental models of the subject will not benefit from the kind of Q&A and quizzing that ChatGPT Study Mode offers. Avoiding digital technology altogether actually reduces friction, accelerates the process, and lets students focus on cognition. Educators should look at reintroducing offline tools.
Use Pen and Paper
Read Printed Paper Materials
Perform Handwritten Work - Blue Books and Note Cards
Mental mapping, organizing, and encoding core concepts and keywords
Oral Discussion
Oral Defense for Testing
Oral Collaboration with Peers
Use Discussion for reference and retrieval exercises
Once students develop cognitive fluency and metacognitive awareness, they may be ready to engage with AI tools in more intentional and effective ways in stage 2. To understand whether or not students have achieved this is to ask them to present their mental models of the topic and describe how they arrived at that mental model. They need to be able to coherently describe why they mapped the relationships the way that they did. If not, they may not be ready for stage 2.
Stage 2: When AI Becomes Valuable
Language models will never be able to tell you how many R’s are in ‘strawberry because they understand language as probability, not words and letters like humans do.
We also know that language models hallucinate because of various prompting problems, such as insufficient context, knowledge gaps, and vague or contradictory prompts. AI training rewards guessing with certainty over questioning with uncertainty. You can test the boundaries of prompting with exercises like:
Testing the limits of what AI can and cannot answer correctly
Learn context-rich prompting to provide more information in the prompt
Establish semantic relationships in the prompt to encode that information for the language model
Learn how to use language models to address complex systems
Ironically, the rise of AI systems has compelled us to rethink our understanding of how our minds work, how we learn, how we communicate, and our fundamental relationship with technology. Educators will continue to experiment with pedagogy and AI tools, but the important thing is that we learn more about how we learn and use AI less as a crutch and more as an extension of our cognitive strength.