Foreign. School Podcast. My name is Bill Selig. He him. I'm our director of technology and I

am here with Skyler Anderson, history teacher at the Upper school. Yeah.

So your name has come up a whole bunch. People are like, you got to talk to Skyler. Like he's doing some cool stuff around generative AI. We were just talking like right before he hit record about Notebook lm. That seems to be the thing that when people are describing how high school students at Hillbrook are using generative AI in meaningful ways, NotebookLM comes up a lot. You come up a lot. So walk us through. What is NotebookLM, how are you using it? What do you like about it? And we'll go from there.

Sure. I think the best place to start is to think about how NotebookLM is different than a general purpose LLM like ChatGPT or Claude. Whereas those, when you're chatting with them, they're pulling information from their training data and you can't really see or know exactly where that training data comes from

or even if any of what you're being told is accurate.

Yeah, it's better, but still.

Right, right.

And it's sort of like effectively for human understanding, their training data is unlimited. Right. So it could pull information from anywhere, which may or may not be relevant to your question. NotebookLM is a little different. It's still, you interact with it in a chatbot format, but you can upload PDFs, Word documents, it will skim text off websites if you just give it a URL. Transcripts of videos, if they exist for something like a YouTube video, and that sort of confines its universe of information to what you've uploaded.

So as a teacher, this is perfect.

Yeah.

Because the video thing is like, that's not insignificant to say.

And I haven't even begun to talk about those elements which I think are useful in their own ways. But like, the history teacher is always worried about is the information that students are getting from ChatGPT. Is it made up? Is it hallucinated? Is it something that is limited access and you can't confirm the authenticity of so.

And just quality of sources. Right. Like that's a big thing for you as a teacher.

Yeah. And especially historians, they're always like, source criticism is massive. It's a huge part of the discipline and getting students to think about it. So I mean, a lot of teachers have this fantasy or ideal that if I could just control where the LLM is pulling information from, I'd be a lot more comfortable with it. And so NotebookLM kind of fills that gap. And so like I just figured out

the name of it, right? Instead of large language model, it's a notebook. Instead of a large language model of like trillions of data points, it's like literally a notebook size language model.

And then like you can use it to generate notes and you can upload your own notes too. And so you could just have it analyze your own hand, like typed up, handwritten notes, whatever. And I mean I first came upon it and there's a New York Times article like I think it was titled How AI is Changing the Way We Write History. Literally. And it's like. And there they talked about historian writing his fourth or fifth book on the California Gold Rush. He uploaded PDFs from Internet Archive and grabbed this primary source and that one and then just started talking to the Notebook LM to get ideas for his book.

Super interesting.

And when he ultimately published the book, he published it. You can share your notebook LM so you can have other people basically interview the same source set that you used. And so it's like there's an interesting idea about like sharing your book, not just sharing your book, but like the archives and the source space you use to write that.

What.

Which has a ton of potential.

Yeah, yeah.

But what I really liked about that whole article for the purposes of teaching is like a lot of the friction for teaching history is like skilling someone up to be able to dig in the archive, right? And like you give them a 300 page PDF. A 9th grader or 12th grader is not, that's not, they're not equipped to like find the needle in the haystack. But that's what Notebook LM can do. So it can kind of like speed up the reading process, which is where like some of my hesitance as it's changed over time is coming from is a lot of students do use it exactly for that to like just read for them. Which of course is a whole nother problem.

Well, yeah, so let's dig into that. So like how do you, how do you live with that? Because you're living with that now, right? So I'm like, I hear you talking and like your whole face lights up and you're like, you can actually like read these and interact with. It's not just reading, right? It's interacting with like, yeah, this Gold rush notebook.

And we like, we can, we should explain that because those are like, for teaching those. I see massive potential with NotebookLM in particular for like teaching students background information because like history, there's like Two things. There's content knowledge and then there's like interpreting sources and argument construction.

Okay.

And you only have limited class time. You would love it to be like the Matrix where a kid could be like Trinity, I need to know about the California Gold Rush.

Oh yeah.

Just downloads it in, right?

Yeah.

And so like with NotebookLM, you can kind of, you can give it the textbook on whatever you're studying and then you can ask it to generate like multiple choice quiz practice tests. For example, you can have it generate an audio podcast that they can listen to in the car. And then they recently have introduced the video feature where it generates like an explainer video with graphics. Right. You can have it provide like one page study summaries, flashcards, all this stuff it will generate based on the text content you uploaded.

Right.

And so if you're, if you just. If the goal is just have the student learn that information. Now you have like this program which can distill it and have the student engage with all that information in the mode that is most like successful for that particular student, whether it's video or audio or flashcards or quiz testing yourself.

Right. So first of all, that's amazing.

Yeah.

Secondly, I'm totally picturing Keanu Reeves now going, I know kung fu.

Right.

It's that moment of suddenly they have access to this knowledge that moments ago was so just like out in the ether. And then the third thing is a clarifying question I have for you. Are you creating this for the students? Are you doing it whole class with them? Are they creating their own versions of like everyone's going to create their own LLM or notebook. Lm like project about Gold Rush. Let's just stick with that because like, I think most people listening, I'm like, I can picture Gold Rush.

Yeah.

Like, how is that working? Is it you? Is it we?

Is it them Individually you can do all those things. I will say I've done it with units in a history class. You know, your unit, one test is coming up. I put up all the readings we read for that unit, the chapters of the textbook that were relevant to that unit, and then you can share the notebook LM link. You can pre generate the video and the quiz questions and the flashcards and check it all before you share the link. Then you share the link and you can say you can use this to review this unit's content and they can access all that generated stuff. So it's great for review, student self assessment, that sort of thing. Alternatively, like the 9th grade project they did at the Beginning of the year was they're basically supposed to tell the history of the world through a commodity or a plant, sort of pre Columbian exchange. Columbian exchange and then like industrialization. And so I showed them Notebook LLM said, here's JSTOR and Google Scholar. Make sure you're not just ripping Wikipedia, but also like uploading PDFs of articles and journal articles, all that sort of stuff. And then they can click, you know, make me a video, make me a podcast. And it's pretty intuitive about like, I mean, this is both the, the promise and the problem of generating that whole project for them and explaining like the whole history of the world through the plant. And you listen to a couple of these, you're like, these are perfectly competent. Yeah, it sounds as good as Planet Money, you know, and. But if the goal later is then to turn that into a physical exhibit, then is the student just replicating what it has already done for them rather than re synthesizing and all those sorts of questions.

Yeah, well, so what do you do?

How do you.

That's the tension, right?

Yeah. And I sort of think about this like the Internet. There's great things about the Internet.

Oh, are we talking like 2001?

Yeah, like when it was first introduced, you're like, this is great for education. They have all this unlimited information. It also is like you can go to a Flash website and play games, you know, so it's like, there's great potential, but also it. The same platform that is good also allows distraction and avoidance of work.

Well, and when I was teaching, like, this is where our age difference is going to show up. Right. When you're teaching, like, there were for sure groups of people who are like, you cannot use the Internet for research. The Internet is not research. You never reliably learn anything on the Internet ever. Right. And then there are those people that are like, oh my gosh, like you can find anything. Not any at that point, not anything,

but like stuff you couldn't get before

you didn't have access to. Like with your high school class, suddenly you had access to. And it was just like, the potential here is amazing. Right, right. But like the, the institution of education had, I mean, realistically, like 20 years of like this slow burn to tease apart. Like, how do we figure out the Internet? Like the way we think of the Internet, right. In education. Well, and all that's been compressed to what, like two years now? Yeah, right.

And like a lot of the solution to the Internet is like, oh, put up a filter and block this function of it. Right. And it's like, can you do the same thing with LLMs such that they can be used in one way and not another? Like, can you have it, you know, interview students for feedback on their essay, but also block it from. You can't copy paste out of ChatGPT. Right. So it's like figuring out how to limit it. Because of course the Internet blocking kids just find ways.

You're not feeling that. Right. And so I was just talking about that with Julian. I was like, we're not, we're not playing whack a mole here. Block this website. This is bad. Internet bad. We do block. We block lots, but we're not going to block 10 billion websites.

Yeah. So I mean, the Notebook LM, I mean, that was totally experimental. I had no experience using it. I'd read about it in the New York Times and I'm like, what's for this function? You know, if the kids really, if the goal is synthesis, not necessarily producing new knowledge, it's like accumulating information from different sources and presenting it because they had to create a physical exhibit, then maybe this is a tool that feels appropriate for that. You know, if it's, here's a document based question and I want you to find evidence to Support an argument, NotebookLM will do that perfectly well.

Right.

But it, like, that is the moment when I actually want the student to do that all themselves versus the Notebook lm.

You lose that struggle of them actually having to figure at the thing. Yeah.

And the friction, like, of course, the students abhor friction and like challenge and work. But we know like, that is where you like grow your skill sets and develop as a learner and all this sort of stuff.

And that's where like your brain physically is rewiring neurons as you're struggling and making sense of the thing. Yeah, yeah.

So I mean, and we, you know, we've had to have conversations with ninth grade about like, oh, you know, because we've introduced NotebookLM, like, yes, it's AI and you still need to ask permission and all this sort of stuff. Right. But I think now students are trying, like, are some platforms okay and other platforms aren't okay. And yeah, and I. NotebookLM is probably not the only tool that is like a more narrow version of an LLM.

Right, right.

And so, like, will we just keep having these conversations going forward about like, which particular tools are useful and which ones are off limits?

Is it fair to say that, like, this will be like an ongoing tension for the rest of our careers? Like, we're not Going to solve it in this podcast. Right. You don't have the answer.

No, but I mean like ask a teacher whether you can cite Wikipedia for your paper. Right. And now I would be grateful if they went to Wikipedia rather than going to ChatGPT. Oh, that's because it's like it's at. And of course now Wikipedia slowly. Its information slowly being replaced with AI generated content.

Yeah, yeah.

Under the guise of human editors. But as a. I tell them like, what is a good quality source? It's something that is peer reviewed, accessible to everyone and has stable content. And Wikipedia, because of the gatekeeping on those web pages, like it fulfills a lot more of those metrics. Whereas AI, it's like you don't know where the information came from.

No.

You. Is it going to be that if you ask the question you get the same answer every time?

No.

Is it accessible to everyone? Well, if you bought the pro version, you're getting different service than. Right, yeah. And so like by those metrics, I'd much rather have a kid on Wikipedia.

We just came full circle with Wikipedia. Yeah. Wikipedia is not an academic source. And I'm not saying that you're saying that Wikipedia is an academic source, but like given that or just. But a complete unprompted. Just answer my question. ChatGPT.

Right. But a lot of teachers will still tell you like, no, don't touch Wikipedia. Right. So we're still having that conflict with the Internet. And Wikipedia, as far as Internet websites go, is like one of the most antiquated.

Right, sure, sure.

I would hope that we're not. I know it feels like the AI boom is raising a lot of questions as educators. Like, you know this. I'm sure you've read about Alpha Academy down in Texas and you have.

Tell us about it.

It's this school, I think it's K to 8, maybe K to K to 5, where students basically cram the core subjects. Math, history, English, science. They're basically on an AI platform for two hours.

It's like a two hour school. Speedrun.

Yes. And I guess there's also like NotebookLMS generative quizzing. And so they're using the testing effect to try and lock in information. And the idea is, okay, the teachers are no longer teaching. I think they may call them guides or something. I jokingly call them screen enforcers because like their role is basically to force the kids to look at the screen.

You're not wrong. Right.

And then for the social emotional side, I think I read this in the New York Times as well, like they have social emotional instruction the rest of the school day. But like literally in the article I read, if, if you look it up, they talk about assembling IKEA furniture to build resilience and is my future career just being a screen enforcer. And on the other hand is like,

well, yeah, a screen enforcer plus IKEA furniture builder.

Yeah. Like a frustration manager.

Yeah.

But you know, if that's, there's an argument to be made that if that's actually truly more effective and that's what Alpha Academy claims, like better test results and all this sort of stuff, then maybe it is better for me to become a screen enforcer than a content specialist teacher.

Yeah, I'm gonna go hard no on that one.

I don't think people are ready for that. And of course all these tech CEOs send their kids to no tech schools and all this sort of stuff. But the scalability in the school I just came from last year, Waterford, they started out as an education platform in the 80s for like and they still, they split in like the early 2000s between the school and the, the digital service to provide like you could get a CD for your like first grader and teach them at home.

Right.

So like, but that idea of like scaling up high quality education, there's potential there in AI, but it looks sort of dystopian too.

It. Yeah, sure does. And yeah, one theme I'm for sure getting. So this season, basically the school year around this podcast where we're focusing like deeply in just AI and education. And one thing that's come up in every single conversation has been like, at its core, learning is relational. And so even though we're always starting like let's talk notebook lm, let's about talk, you know, like, look at the episode notes. Like that's going to be the first thing people are going to see. But we always come back to like, it's actually about your relationship with the students.

Yeah.

You know, and that's, and that's what's, that's what people are going to remember. That's what facilitates learning. You know, when I think about all the like the things I remember from high school, it's always around a project or a teacher. It's an emotional thing.

Yeah. I mean it's the motivation component. Like, and we have lots of discussions about how to improve student learning. I mean, or like designing an assignment. And how do you design an assignment to maximize student excellence. And at the end of the day it's like if you can get a student to be enthusiastic and curious. Like you could have a really poorly written assignment and they'll produce something amazing.

Yeah.

And so I think relationships provide a lot of that. I do think like there's some truth to like kids who just can't bite on a, on a topic. Like it's. Teachers have limited approaches to teaching things and maybe AI makes it easier to provide different modes of accessing that content, which could lead to the enthusiasm and engagement.

Yeah.

But so much of it seems to be like I'm turning to the AI, the LLM, because I don't actually care about the work and it's just busy work and this isn't interesting to me. And, and AI is not necessarily going to solve that problem. Right. People might, you know, but I do think there's a lot of the relationship component is why these guys, tech CEOs send their kids to no tech schools because they want, they, I think they appreciate the human experience and they want their kid to have something similar to what they had.

Yeah, well, and that's kind of the. Right. You're giving us actually both extremes. Yeah, right. The no Tech and the Alpha Academy.

Yeah, yeah, yeah. And I mean, I asked, I mean I don't have kids, but the conversation comes up with my wife, where would I send my kid and probably a no tech school. Like not even laptops, all that sort of stuff. Right. And I, I like, I do think the laptop's awesome. You can go on Internet archive, pull up a text from the 17th century and you know, 17th century English. It's not easy to read. But before if you wanted to do that, you would need to like be going to Harvard and have access to special collections, you know, and like literally

put on like the white gloves and

go down like to the, and maybe look at it behind glass and all this sort of stuff. And now you can actually do real historical archival research in high school and middle school through the Internet. And now you can try to compile enough of that and access it through notebooklm that you can write a book or a paper or a chapter or something using the actual methods that historians use on a smaller scale.

I have one small question because we could talk about this one question for like an hour. I think I want like a three sentence answer and then I want to know like what you're excited about next. And then for us as we're recording, it's almost lunchtime. Right. So we gotta get to that. So like I'm fascinated with the idea of like the Gold Rush book and then sharing your Notebook lm. Is there potentially a new style of book where it's not actually a written book, page one to 200, and it's actually just like a curated notebook LM or whatever it may be, and it's actually like, interact with this thing that I curated instead of read the thing that I wrote.

Yeah. I mean, I do think what it means is the work that the historian did there was actually curating the sources, not doing the writing of the book. And that is like, as a teacher, that's really what I'm grading in all these circumstances when it comes to source criticism is like, what did you think was a good source? What did you pair it with? Like, the paper at the end is like just a way to basically force you to synthesize stuff.

Okay.

But the student has already, like, if before they've written the paper, if they've done their job well, then the paper is just a formality, you know, if they've chosen the right sources and found them. Right. And so I think in the case of this Gold Rush book, it's like the non historian is not going to know what books are out there and. Or what archival sources are available. And of course, AI is like, eroding that as well.

Sure, sure.

But then, so then the thing that the non historian gets out of going to the historian is like having that curation.

Interesting.

And I, like, I'm not that connected to my grad school buddies.

Yeah.

But it is like obliterating PhD. Like the process of getting a PhD and the dissertation and all that stuff. Because, like, that was all the work you're spending three years doing.

Right, right.

Or four or five years doing and curating the thing. Yeah. Like going to, going to the archive and like finding the thing. And then, and, and then, of course, the writing of the dissertation was like a year or two of that. And now it's like all of that. If you're not doing it, you're not competitive, you know, and so it's really. I think it's changing that discipline. And it does lead to this, this thing where people who are curious can get access to your. Your source base and like, whatever questions they have, they can ask of it, which is like a much more relational, like, way of interacting with information. It's like, here's what I'm curious about. I don't necessarily want to hear your writing, but I'm curious about asking questions of the stuff that you've explored. So it's like your presentation's no longer as important. It's having Access to what this historian has already curated. I'm sure lots of this is happening. If you look up, they've done studies of articles and the infiltration of AI written work, and it's like above 50% now in some fields. So it's like, I do think a lot of it's sort of slowly replacing like the human centered writing. Interesting.

But then like the artistry and the expertise becomes around curation, not about creating the content. Yeah, that's a really. I have not heard anyone kind of articulate that that way.

But I mean, you can ask ChatGPT what sources on 17th century coffee houses are out there and it will give you some stuff and maybe some stuff you is not available online is difficult to access. But like that curation and the skill of curating is also being eaten by the LLMs these days too. So I don't know how long that stage will last. Interesting question, but like, I had my students last semester in the econ class propose jobs of the future that will exist in three to 10 years. And one student came up with this idea of, we jokingly called it the slop curator, which is like a human out there who decides what AI generated content is actually worth being seen by a human. Right. Which is like, maybe that's the next step where you're like, you're not curating the sources anymore, but you're deciding which AI generated books are actually worth reading and all that.

So the slop filter. Yeah, that's a super interesting idea. All right, last thing. What are you excited about? What are you curious about now? Like, what's the next.

Yeah, I mean, I do think what excites me we've had this long standing tension in history between covering like skills, argument, construction and content. Like knowing the years of things, who people are, where things are, geography. And it feels like with limited school hours you can't do both. You can't have your cake and eat it too.

But like, that's a history quote.

Yeah, but, you know, so you want the student to have the context to like write a good argument with evidence.

Right, Right.

But you don't necessarily have time to like cover the context and the content. Or if you do that, you don't have enough time to do the argument over the writing of the essay and all that stuff. And so what I am excited about is potentially AI allowing us to have our, our cake and eat it too. Like get the student, you know, like the matrix updated on the context. So in class you're really focusing on the skill work.

Right.

And I. I hope there's a world where that happens. And it's not just AI replacing both

things, but it's our role to make.

Well, people, you know, they have to want that, to support it.

Yeah.

And, you know, if. If they don't want it or don't care, like don't care that a human's in the loop, then, though, there won't be a human in the loop.

So. Yeah. Super interesting. Skyler, thanks for joining us.

Yeah, thanks for having me,

Sam.