Hacker News

Overall, I do agree with the thesis but:

> Now… how does Sal Khan want my kid to learn?

> Watch a video.

I find this a little uncharitable. I learned a lot of math at a young age from Sal Khan's early videos (back when he used to erase by switching to a black pen and scribbling across the entire screen) and, to put it in the author's terms, it served as an easy-to-digest scaffolding on top of which to build a deeper understanding. I don't think Khanacademy ever sold itself as a magic learning resource that removes the need to work out problems yourself.

From what I know of the Khan approach, it's akin to "flipping the classroom" [0], which has gained pretty wide acceptance. It was pioneered by Harvard physics professor Eric Mazur.

(I use the flipped classroom approach extensively. But the legal profession being the eager embracers of novelty that we are </sarcasm>, I refer to the approach as "enhanced Socratic method," given that Socratic method was something we all endured in law school.)

For math and engineering courses, the basic flipped-classroom approach was used at West Point for over 150 years: In the so-called Thayer method, students study textbooks in advance, then in class, they work on problems on chalkboards. The instructor moves from chalkboard to chalkboard, coaching the students in real time. [1]

(I do much the same thing in my contract-drafting classes: In small groups, students do drafting exercises using online whiteboards. I lurk on my laptop, quietly coaching in person as needed.)

[0] https://bokcenter.harvard.edu/flipped-classrooms

[1] https://flippedlearning.org/syndicated/avoiding-flipped-lear...

This articles core argument appears to be that a live instruction is better than a video because there can be feedback at the moment of confusion while a video can't. But that depends on student's impulse being triggered and I'm not sure if its any better or in some cases can be worse if the teachers content is not as thorough as Khan's or anyone else's video - as these benefit from a worldwide audiences feedback on correcting any confusing or unclear parts.

In the classrooms I see through my kids, the teacher is doing roughly what a Khan video does: walking through a worked procedure at the front of the room. If a video fails to provoke the "wait, why does that work?" moment, then a teacher following the same flow likely fails in exactly the same way as the author claims and missing the opportunity for hands'on learning. A video actually allows the learner to go back and review content if anything was unclear, and presumably any interactive AI agent can help with the feedback part even though the teacher in the video isn't.

I also wish the piece had stayed with Khanmigo, where nobody has a good theory yet on where the ai tutoring loop actually breaks and why people fail to engage it and that putting it in their face may not work, why exactly personalization at that scale hasn't produced the gains it promised.

Also irked by the title built on a pun on Khan's name, sets up an awkward framing that kind of lets me know it may not be unbiased.

Is the article about something else than how math on Khan Academy was thaught?

I remember minimal videos (3-13 min, usually 5-8), and then exercises drilling the techniques until you understand it and don't make mistakes. If you got a perfect score (no mistakes whatsoever), you went a little faster.

That's learning by doing, or am I wrong? Or are there no exercises for things other than math?

Khan Academy has a problem lately:

Go visit a page like https://www.khanacademy.org/math/linear-algebra and count the number of:

  * cookie banners
  * donation panhandling modals
  * account signup widgets
  * gamification modals to tell you about leveling up
  * etc

Whenever this topic comes up I feel the urge to point out that not enough people have read Audrey Watter's "Teaching Machines: The History of Personalized Learning", or her blog. We might stop repeating the same mistakes over and over again if we did.

[0] https://direct.mit.edu/books/book/5138/Teaching-MachinesThe-...

[1] https://2ndbreakfast.audreywatters.com/

As someone who has earned over 3 million points on Khan Academy over a 3-4 year span about ten years ago, I am very appreciative of my experience. What I appreciated most is when Sal would derive formulas such as the quotient rule in Calculus from the power rule by using negative exponents. Deriving formulas rather than memorizing them opened up a new perspective for me.

This article appears to focus on the chat bot which I have not used and was not part of my experience. At the time, I had a subscription to Wolfram Alpha that was helpful if I got stuck on an intermediary step.

The high level of goal of the post which is directed at asking good questions raises a point that seems valid to me.

Education is a very difficult problem. People like to learn and people want to learn. Mostly improve themselves over their friends and society, and they want to do it quickly and easily. This proven by the countless influencer videos and influencer culture present in youtube on topics such as finance, health and many others.

However, true education is really hard. It requires hard work, sticking with something beyond when it gets boring and difficult. Being stubborn to get it right. These qualities are not very common and instilling this is extremely difficult from outside.

What author is asking is, he wants Khan Academy to be a true education platform , while dismissing it was video platform. He is wrong in both counts.

Khan Academy is a gateway for a student to draw the map or scaffold as the initial phase. The start of the education happens when the student works on the exercises, and tries it again and again to get it right. They are driven by the mastery based education which they try to ask the student to do.

A good general article on education, but weirdly wrong when it comes taking a dig at khan academy platform. And w.r.t to AI use in Education, students who know how to use AI are using it exceeding well for their true education.

> What he has never had is pedagogical knowledge: an understanding of how people learn

As anyone who’s met Sal will attest, this characterization is incredibly off the mark.

The author extrapolated from the product as opposed to understanding the person and how the product came to be the way it is.

K.A. has always helped me immensely. Often I will add a video as a 2nd "I do" before our "we do." If there is extra time after small groups we typically answer the math unit questions as a class. Students are thrilled to come up in front of the class to defend thier reasoning before finally seeing if thier choice was correct.

What is with these personal attacks? I don’t understand the insistence that every educational tool must offer some perfect, complete learning experience.

I grew up in India watching the early Sal Khan videos because I preferred them. They complemented school and helped me understand things I’d ignored in class or simply hadn’t been taught well.

You can critique the limits of videos without dismissing teachers who have genuinely helped millions of people learn.

This seems to be wrestling with the wrong opponent.

> And this is what I think the edtech world keeps getting wrong. The assumption is that if you can deliver the right content, in the right way, at the right time, learning will follow. It won’t.

I think it depends on the goals of the student and what you're trying to learn. Some things are procedural and can be learned by being told, some things require a more hands-on approach to really "get."

If I want to learn how to drive a tractor, I just read the Farmer's Guide to Tractor Drivin' or find a YouTube video (I learned how to drive an excavator and fell trees like this—watch and listen and then go do what they said; hot damn, I learned how to do it).

If I want to learn how to produce a modern masterpiece (or novel invention) in some field or art, I won't find the "how to" in a book or video [1]. That's very much a personal journey that can't be replicated or recalled as a series of steps you tell someone to follow.

The Sal Khan way is prescription-driven and has its place in the world and I don't think it should be dismissed. But that way won't give you the "10,000 hours" level of understanding or experience (and arguably, shouldn't be expected to—even in the shape of an intermediary chatbot).

[1] https://www.youtube.com/watch?v=mvgVA_neH04

I have been using Khan Academy for over 7 years growing up. It has helped tremendously and taught me to be a critical thinker. There are exercises, quizzes, and unit tests to test a student's understanding and if they get something wrong, KA marks what videos or articles they should go back to and review. I agree with one of the commenters here, this article is uncharitable.

I watch Khan Academy videos because they explain topics like mathematics in an easy-to-understand way. But it's just one tool in a student's arsenal. Khan Academy is by no means the one-stop-shop for education and should not stop learners from inquiring, constructing, expressing, and communicating as part of their learning journey. I hoped that the author would at least make that distinction a bit more in their article because it seemed a little unnecessarily critical. I _think_ the article makes the assumption that most students are just passively watching videos and _are_ using Khan Academy as the one-stop-shop and that the site (and Khan) are missing out on an opportunity to provide a better experience that is more in line with John Dewey's principles of learning.

The AI voice of this article irked me. Pangram says 44% of it is AI, for the chunk I was reading.

For generic trope-y content like this, the voice can't be left this AI-ish. It has to be about something really interesting for this AI tone to be bearable.

I have nothing to say on the merits of Khan Academy (I have never looked at it, nor have I known anyone taking a course), but from 35 years of teaching, I know that we mostly learn by doing. It's why university courses have labs, problem sets, and required essays. All else is memorization (not that memorization is inherently bad, part of learning assembly language programming is memorizing the available instructions, while learning French includes memorizing vocabulary).

Educational technology that is focused on delivering “content” is unlikely to be of much value. Technology that induces the learner to experiment, to discover, to discern patterns, and to ask further questions, is going to be much more valuable.

The article starts by talking about how interactive learning with Khanmingo failed, and then goes into depth about how watching videos isn’t enough. That’s disappointing. Khanmingo should be superior to passively watching a video because you can interactively dig deeper just like the author says is necessary.

Off-topic and may be found rude - I find multiple instances of "Kahn" in the comments. Is it really that difficult to put in the effort to learn the correct spelling of "Khan" specially when it is everywhere in TFA, the post title, and other comments?

It feels weird and unfair to focus solely on Kahn here when he’s competing with some random teacher or professor’s droning, confusing, non-interactive lecture that’s less engaging and just as lacking in interactivity or application.

I also don’t think he ever purports to be “all you need” to learn.

It reminds me of the people who criticize Duolingo because you can’t actually reach perfect fluency through their app--like wasn’t it always obvious that it was never meant to be the only education you need; wasn’t it always obvious that you need to actually _speak_ a language to learn it? And yet it’s still a great app that has helped millions of people learn languages.

I think the core thesis - that videos aren't the full package - makes a lot of sense. But I don't see why Khanmigo couldn't in theory fill in some of the other pieces.

I'm reminded o A Young Lady's Illustrated Primer from The Diamond Age - and how LLMs haven't been able to achieve anything similar. With the correct supervising AI agents that have their own goals - I don't see why you couldn't have an LLM try to motivate you, or gamify education tailored to your interests and targeted at the areas you are struggling with. Somehow weaving in their own education videos seems way too difficult though - it feels like it should be an entirely different thing

Maybe Khanmigo just kinda sucks? Having a ton of funding doesn't mean its the best possible solution. I've never used it, so it's hard for me to judge.

I think there is more to this.

Here's a presentation that, kind of claims the opposite. More specifically, it claims figuring it out on your own is not as good as having a teacher tell you, and then having you use the info, and it claims to be backed up by research

https://www.youtube.com/watch?v=g1ib43q3uXQ

I'm not a fan when schadenfreude indulges in someone failing to make the world a better place.

I think there is a simple reason why AI tutors are not terribly successful.

AI is just not good enough yet, they are only just now capable of performing tasks, getting them to do those tasks involves a degree of careful guidance and the ability to handle frustration calmly and rationally.

Those things require a high motivation to have the task done.

They are not the properties you want in a tutor, a few students have the drive to learn in conditions that are not ideal. They are a small minority.

What you need is a AI that has a deep enough understanding of the ideas behind the metals it teaches and is engaging witty and charismatic enough to get that information across to a student.

When it has those abilities there will be a lot of parents that won't want it to have access to their kids.

Not sure what actually killed it but Khanmigo seemed to have removed the state of being stuck. Wrestling with a problem you cannot immediately solve is where the learning happens at least for me in programming and math.

Seeking help after a while is obviously fine but when the help is one keystroke away and always available, you take it before the useful part of being stuck has happened. Not ideal for learning tbh

A comment I had made on Dan Meyer's article, also relevant here (Also, as others are pointing out here, there is a crucial difference between the videos and their chatbot; you can inspire with videos because your human interests are on display):

This resonates so much with me. I am the co-founder of a microschool and also a neuroscientist. And these two have intersected so much in fascinating ways. Four years in, the big lesson for us is that for learning to make sense it should have the following progression: meaning, motivation, mechanics, measurement.

Conventional schooling, and everything designed to help with conventional schooling like Khanmigo give in to this systemic gravitational pull of measurement (we must know what they are learning). And from that ensues your standardized curriculum, measuring the visible and easy, because that's easier than something like "mathematical thinking", and the focus on mechanics (multiplication tables, fractions, chemical names, whatever). And in doing this we leach out all meaning and focus on the drills.

Chatbots, the way they are set up now, are doomed to fail because the entry point is mechanics. "How do I solve this particular quadratic equation?" It's a meaningless question to a child because it simply does not connect to their world. We cannot start with that and work Socratically towards meaning. We must instead offer or build a world where meaning and sense-making are possible. (Wrote about this here https://blog.comini.in/p/why-khanmigo-will-fail )

Personally interesting for me is how meaning-making and sense-making are essentially the core of conscious experience. We, bundles of experiences, cohere and conspire and collaborate to create a meaningful experience from the sensory data that assaults us. Data that would be meaningless without a perspective.

So in education we are doing the same thing that the mind does so effortlessly by existing and being in the world and grappling with it. But we shoot ourselves in the foot by taking a reductive lens and split it into elements.

That's worked well for us only until we encountered complex adaptive systems that need us to also focus on the whole. But the system has been built to focus on the narrow and hope that these come together at some point. And so we end up with this meaning crisis we are all navigating, in school and outside.

More about Meaning before Mechanics here: https://blog.comini.in/p/schooling-has-a-meaning-crisis-para...

Nothing beats a well-written book.

I'm curious if maybe the mistake was in assuming we needed AI for the teaching part of the process.

Maybe we should focus only on having AI for the correcting/internalizing part instead? A purely deterministic grader and problem creator creates an endless stream of problems and solutions. The LLM's job is just to help you get to the point that you can assign a random problem from this problem set and the student will be able to answer it without assistance. Hopefully the teachers lecture at the beginning of class helped with understanding as well (I suspect that it would), and also provides a more broad "theory of mind" about what we are doing than the myopic LLM's "Let's get you solving this problem" does. Also, humans are social creatures, and being taught by another human being (even if it were less efficient) is likely beneficial to most people.

tldr; Normal human teacher, deterministic grading to gauge progress, and LLM assistants to help the student get to the point that they can pass the deterministic grading.

Yeah? That's what the style of teaching is about.

Think less science and more a conversation; establishing syntactic and semantic ground rules for the speakers grasp of a behavior relative to problem.

It's not necessarily about teaching something concrete. It's about teaching a social style in an abstract way. It's discussing a painting or piece of music, in math the symbols used to represent possible number/geometry generators.

I never just watch the videos. I rewrite what is written out. To recreate the song and dance so to speak.

Pragmatically speaking life is just performance art. Only a few specific realms of science move the needle for humanity; biological health maintenance (across all contexts; housing, food, not just medical) and our research in energy infrastructure.

Everything else is just one's relative outcome. Result of their sensory experience. Feels like this should be obvious at this point.

The studies that show brain activity goes down as we use AI. Yeah. So? Is it great to stimulate ourselves so much with ultimately arbitrary content our biology immediately tries to forget due to entropy? Heart attacks are up in <60 crowd relative to last generation. Constant interaction with flame wars and tabs v spaces (functional languages and Nix or gtfo!) computer config wars level inanity is not good Bob.

Isn’t education often a you-can-lead-a-horse-to-water-but-you-can't-make-it-drink problem?

Comment https://news.ycombinator.com/item?id=49411243 by @dabedee is dead from downvoting but shouldn't be.

The only thing more irritating than the AI-assisted unfair attacks on Sal is the font it's written in.

That article is spot on: we learn by doing and experience, not being told the high level outcomes (which is not without value as a guide).

Why are there people in this comment section who are just downvoting opinions that they don't agree with?

I looked at Khanmigo fairly early on. I tried it out and thought: this is ChatGPT with a system prompt. And not a good system prompt either. In addition to what felt like rote math it had some had some experiences, like you could "talk" to Plato or some other figure. Each of these was also pretty awful; simplistic prompts that spent all their time on guardrails and none on the experience itself. (What it the student thinks they are ACTUALLY talking to Plato?! Avoiding this seemed very important to the makers.)

In the more core experience they were super focused on not giving kids answers. I suppose built on a fear of subverting the teachers or classroom environment. But again it became a fixation that took all attention away from teaching itself.

But I figure you put something out in the world and see what happens. Yet I came back to it a year later and it was exactly the same. Nothing had changed. It baffled me then, and still baffles me... I had done enough to know the whole experience was easy to manipulate and yet I couldn't see any evidence of effort. It felt very much like "we tried nothing and it didn't work". Maybe I was missing something, I don't know.

I think there's a few different problems here:

1. I think the author is correct to highlight that underlying motivation, which is so important to teaching. We can focus on a high-minded Constructivist ideal of intrinsic motivation, but motivating students is also what grades are for, and classrooms, and learning with peers, having someone pay attention to your work, and so on. I do think that the AI chatbot is not capable of providing that. But the AI can still ENGAGE with that motivation. I have a feeling Khanmigo never did. I suspect some of their privacy controls also kept it from ever creating a good model of the learner.

2. The entire experience was very focused on answers, on solving problems. And yet it focused on that so reluctantly. The obsession with not giving away the answers had the unfortunate result in not engaging with anything but answers. It felt like it was always teasing at something it would withhold. But questions are cheap, especially rote questions, they shouldn't be treated as precious.

3. This may be unfair, but I infer the makers of Khanmigo did not build it with love and craft, which is unfortunate. I don't know enough to say why. Though there's a weird confidence to the product and to all of Khan Academy that I feel does not bring the necessary humility.

4. The chat experience was asymmetric in the wrong direction. In each turn the AI can and will spew out paragraphs of text, and the student types a couple words. Chatbots DO NOT have to work like this, they operate wonderfully with large chunks of user input. But you have to have the modality to receive it, the openness of experience to elicit that response, and the respect to make use of that response. Speech can be good, but it's a real challenge in a classroom environment (though if you are investing in Khanmigo, you can also invest in the other hardware to make it possible).

5. In some fairness, individual tutors don't represent any pedagogy at all. It's usually just someone making it up as they go. Motivation is a huge aspect, studying with someone sitting next to you, watching you study, will do wonders for keeping you focused! But also a tutor will have a natural theory of mind, hopefully the ability to see a student get the wrong answer and get inside the student's head to model their understanding and find a path to a correct understanding. That's a very challenging task, not beyond GPT-4 entirely but still quite challenging, especially if you are optimizing cost or time to response.

6. A good experience will do a TON of prework. I don't know if Khanmigo does that, if they've reshaped and expanded each educational module into a thorough playbook for the AI. That's where you have a chance to model things like wrong answers (there are actually SEVERAL books enumerating wrong answers in math), and use that to plan out responses.

If Khanmigo is not successful, I think that's on Khanmigo. Still teaching is very challenging. And maybe all of Khan Academy fails in this way: it does not seem to acknowledge the challenge of what they're trying to attempt. They seem far too confident in what they do, and not nearly creative enough.

Sal could not get to his cousin's house every afternoon to do math with her, so he recorded the sessions. She said she preferred the recordings, because she could rewind them and because he wasn't sitting there watching her not get it. He put them on the internet. Then he spent about fifteen years building an exercise engine, a "mastery" model, and a teacher dashboard around them, and told everyone who would listen that the point was to move the lecture out of the classroom so the classroom could be used for something better.

And the essay's premise is: this man wants your child to watch a video.