Sounds good.

Hey, everybody. Welcome to the Hillbrook School Podcast. My name is Bill Selleck. He him. I am the director of technology at Hillbrook, and this is season eight of the podcast.

Wow.

So excited.

Me too.

Who are you?

Me too. And wow. I am Jay Van Buren.

Hi, Jay.

I am working here at Hillbrook, teaching in the Hub, which is the makerspace, only for a limited time until June. And I am also really heavily into augmented reality. So I'm bringing augmented reality into the Hub and using that.

Yeah. We are so excited to have you in the Hub and to dig into your expertise around AR augmented reality. We're just going to go back and forth, I think, and mostly say ar, but what do you know about ar?

So, really quick, just for people who don't know, augmented reality is different from virtual reality. Virtual reality is the one where you got your goggles on. You're completely separate from the real world.

And that's where, like, people can, like, pretend to, like, poke at you, and they're like an inch from your face and you have no idea.

And you have no idea, which is actually one of the. Socially speaking, it's one of the things that is a real barrier to people wanting to put that headset on because you feel really unsafe. Augmented reality doesn't have that problem because you are not cut off from the real world. You are looking at the real world. You're just looking at it through your phone or through your iPad. Right now, eventually there's going to be augmented reality glasses that become popular. Right now, there's like, vanishingly few people who are using them, but you're looking at the world. You're seeing the real world with extra stuff. So that's why it's augmented. There's extra stuff on top of the world.

Well, so Pokemon Go.

Exactly. And that is the example that everybody knows. It's funny, you know, if you had told me when Pokemon Go launched, I don't know, it would be five, six years ago, probably something like that. If you had told me that six years later, it would be still the only augmented reality thing that had broken through into general public consciousness and knowledge, I would have said, you're crazy. And yet here we are, and it's still Pokemon Go is the Go to example, because it's the only one. And I have a lot of theories about why that is. And I kind of think everybody's mostly using augmented reality wrong. But, yeah, it is like Pokemon Go. And what we're doing right now with the students at the Hub is essentially doing Pokemon Go. But for other things.

Yeah, well. And so when I saw Pokemon Go, I'm like, how do we get this in the education space? And the answer was, you don't. It was like, you get to. You get to, like, walk around and find, like, the Pokemon where you happen to be. We had a gym, and maybe it's still here, like, somewhere in the village. Oh, yeah, at Hillbrook. You know, I remember, like, vaguely walking around our city when we went to go in Monterey. It was like, oh, there's a bunch of Pokemon here. And that was about it. But always, like, you know, people sometimes share, like, the icebreaker, like, what's your superpower? And mine is one of two things. It's either, like, seeing an app or a new piece of technology and immediately knowing how we could integrate it into education. Like, instantly I see a thing. I'm like, oh, that's how we do it. And it took me actually, a very long time to realize this is not how normal humans think. Right. My second superpower. And I actually did this my first employee orientation at Hillbrook. I thought it was like, if you could have any superpower kind of thing, I was like, oh, a tail. I would love to, like, a human with a tail and, like, climb up a tree or, like, be able to, like, type with two hands. Then, like, sip coffee also with my, like, yeah, that would be so cool. That would be cool. And then turns out everybody else was like, I'm a great listener. I'm great with working with students. I was like, oh, this is what we meant by superpower, right? I was like, can. Can I go again?

My worst flaw is that I'm too much of a perfectionist, and I do everything too perfectly.

But. But I went. I went like, superhero kind of right on, not, like, time. I was like, oh, you know, I'll be like, I'm not gonna do time travel or, like, pause time or, you know, whatever. Super fast rudder. Like, I'll go with the tail. That's a good one.

That is a good one.

And turns out, that's not. That's not what the prompt was.

No.

So anyway, so, like, so I see that I was like. Like, what I can imagine all these possibilities of with AR just. There isn't a thing. As it turns out, you actually have developed an app with it. Tell us briefly about that. And I promise this isn't, like, you know, sponsored by Mimbid.

Right?

But the idea, like, you actually know a whole lot about it, and you're at a really exciting, interesting, like, super exciting cross Section, like Venn diagram, as I'm drawing in the air of, like, education and AR space.

Yeah, well, and just to follow on what you were saying, I'd say my superpower in that space sense is I actually come out of a fine art background originally. And so it's my. And my knowledge of, like, art history and humanities and art and all of that stuff is the thing that where, like, I see technology and I think about, how does this technology relate to all things that are common across all human cultures and things that are common across human time? And one of the things that is special about, you know, that humans always do is they make pilgrimages. They go to certain places. There's certain places that are meaningful to them where something happened, and there's places that they go to be in community with other people. And so to me, that's what I built Membit around, was the idea that it's actually the connection to the location that becomes meaningful. And so Membit is built to be able to put things in the world very precisely, and it's built to be able to do it anywhere. And not to get too technical, but maybe it's okay to get a little technical on this. The biggest problem in augmented reality is relocalization. And what that means is that most AR experiences, you might see something and you're seeing it, but no one else ever sees it in the same place or what's happening. You could have something that was basically using gps. What most people don't know is GPS is kind of garbage. Like, if you really pay attention, like, the blue dot on your map thinks you're down the block from where you really are.

I mean, it's close enough. Like, my wife was just on a trip, and I was like, oh, she's in this city block, right?

That's right.

Great. I'm on the freeway going 70 miles an hour. I'm about here, and in the next 30 seconds, I need to merge. Right?

That's right.

That's great.

And that's good enough for that kind of thing. But if you wanted to see, like, if I'm telling you a story about, like, I fell in love under that tree, and it's like, well, it was that tree. It wasn't that tree over there. It wasn't that tree over there. It was that one. Like, it's important to our stories, to our human lives, and to things that matter to us exactly where it is. And so what Membit does, we have a patented form of relocalization called Human Positioning System. We show the user a photograph. The user lines that photograph up with their video feed and then they hit the button.

Oh, so this is. I know this because I've placed with Membudu a lot. We've been talking for like, six months about it. And I was like, we finally need to just record this. If you've done stop motion, it's called onion skins or onion peels.

That's right.

Where like, you see the frame before it at like 50% opacity. So it's kind of a ghosted image. That's right. It's that same sort of idea. So someone takes like, a photo as, like, I want the AR thing. I want my little Pokemon thing to appear here. And then you take a photo and then the other person, like the creator, takes the photo. Then the user walks up and goes, oh, I have the onion skin. I need to line up this corner of the tree. And then the road curves this way. I need to step back, I need to step forward, I need to angle. And then they hit like, I'm here or something.

Yeah, the button says here. That's my little bit of poetry in the interface. There.

Perfect.

Here, here. But basically, yeah, it's like, you're here. And then we can reinstantiate whatever somebody created in that place. It could be like three 3D models, videos, images, whatever, in any combination all around you. And they're exactly where the person who created that experience for you meant you to see it. And that enables a kind of meaning creation that you just can't have when you don't have that level of specificity. Now, the other things that are out there, like, Niantic has their own version of this and their own version of something. Apple has one, Google has one, where they're basically creating a high. A high quality mesh 3D model of the world. You have to do a whole bunch of scanning. You have to walk around with your phone and do all this stuff, and it creates this model of the world.

And like, I've done this in the Clips app, where, like, it has, like, the disco balls and the disco lights, like, go up the couch and over the couch.

Yeah.

So it creates, like an actual 3D.

It's creating a 3D map. And I mean, that's great, but what it means is that someone has to have done that in every place that you want to create AR content. The other problem with it is that it fails. It fails a lot. It fails if someone moved the couch. It fails if someone parked a truck on that corner and there was no truck there before it fails, if there's a bunch of people standing in the way. If you're trying to do ar. I was actually. I had another app that was using that system, and I had a big party and a bunch of people were doing it. All of them. The AR wouldn't work because there were too many people.

Oh, too many people in the room.

Too many people in the room. And I had to say, like, okay, I got up on a chair. Everybody move out of this way, please, so that the AR will work over here. Right. It's like, that sucks. The reason that our system is better is that basically, like, you have a human being with a human mind trying to match up something I've never had to say to someone, match up this picture, but ignore the truck.

You just line up the picture.

You just line up the picture. Yeah. Because you know that well.

And so, like, to quickly get this into education, the reason that you're able to actually hit the ground running.

That's right.

Is that students can hold up an iPad, line up the thing, and tap. I'm here.

Exactly. And they can create anywhere they can go. I didn't have to map this whole campus. I didn't have to do any of that stuff. Hand them the iPad. Go put stuff where you want.

Well, yeah. So you and I have been talking. Like, we can never talk for just five minutes. It's like. And then there's this whole idea. And then there could be a separate app for all this thing. Like, give us one or two kind of projects you're already doing with students. And I think people listening are gonna be like, oh, my gosh. Like, I could be doing that tomorrow.

They totally could. Okay, so number one, what I'm doing with the fifth graders. Cause the fifth graders are doing a whole bunch of work about themselves. So identity their own identity. So we are doing an augmented reality thing we're calling the Self Endure. I have to give credit where credit is due. Samuel, who's in my class. I was saying, okay, guys, we're going to make cylinders that are about yourself, but we need to think of a better name for this. And he's like, self Endure.

Perfect.

I'm like. I'm like, that is what it will be called forever now. That is what it is.

You showed me this one. It's like a giant almost ring. Like a ring on your finger, only it's giant 10ft tall.

Exactly. And then what we did for the door, which I thought was really fun, and it was great. Got the kids to learn Some new skills in Illustrator. You know those things that are like two faces looking at each other and it's also a vase.

Yes.

So basically they made a little image of the profile of their own face and then the space between the two profiles, like the vase part is the door. So you walk into the cylinder through a door of these two faces that are the face of the person of the kid who. Who did it and then go inside and then all the interior surface of the sphere, or sorry, of the cylinder is all graphics that they designed. They are putting 3D models inside there. They're going to put videos inside there, they're going to put songs inside there. And it's all just telling people about them. And then they're going to be able to go take that whole thing and they're going to put it on campus in their favorite spot.

In their favorite spot.

Right.

So it's the place. And so if we take just a step back. Back. Right. This sounds like what I used to do with my second graders, like, all about me. This is what I've done with like grad students in school of ed for, like, you know, create a presentation in Google Slides and share it with me. So, like, we always do these things about. Let me introduce myself. If we go a little bit deeper, it's about my identity. Right. But then taking that and actually having it in a physical space changes it.

It definitely changes it. And it makes it big and spectacular and powerful. And the thing is, what the biggest thing I want to say about why AR in education is that it empowers kids. It makes them feel like they're creating this thing, which is huge, and that people are walking around inside of and they love it. I mean, one of the things about being a kid is that you feel, you know, you're. You're small, you don't know everything. You don't have all the skills that the adults have and you don't feel like you are powerful. And one of the things that is just really great for these kids to experience is like, I made this thing and now it's this huge thing that people are seeing and they're walking around in it. And that's one of the things about AR is that like, yeah, it's not really there. It feels like it's there, but it.

Feels like it's there.

Yes.

Yeah. Well. And so that's why we continue to have iPads in grades junior kindergarten through eighth grade. Because there's a bias towards creating.

Yes.

The ability to create photos, videos. And at this Point, all our middle school devices have Apple Intelligence on them to be able to quickly, quickly generate something just from text and then get an image and use something like Meshi AI to take an image and make a 3D model. They can do all of that on an iPad. And so we've kind of covered check, check, check. We can do Google Slides, we can create images on our own. That's cool. If we go over to a separate circle of Venn diagram, we have the hub, which is 3D design, laser cutting, making big, big things that often take a long, long time. Right. And require certain expertise to get them to like, really feel like, oh, that's the thing. Right. Or like a lot of iterations that we often don't give kids time for during the school day. We could, but we don't just like, we have to move on to the next thing. It's like, oh, let's do the thing again. Right. So like, and that's an amazing thing to have a space where kids are making big physical things. Things. And this does, like, a really cool version in a lot of ways, I think does the best of both. Like, it feels like a physical object, even though it's only on your phone because you have to stand in a physical space.

Right?

Right. So you get like all the benefits of, like, very quickly being able to create something digitally. But then with ar, you get the benefits of, like, it feels like a real thing because it's. It's locked into a place.

Right. And when people are experiencing it, they're walking around holding the phone and they're looking at it, and when they are in it, they're in it. The fact that they're only seeing a tiny bit of it at a time does not matter at all because it just. And this is actually, we could talk for a long time about this. The reason is that it just integrates directly into our visual perception system. And there's a lot to say about that, about why AR feels like it's real. But we don't need to go into that right now. But yeah, they can iterate and they can create. And the thing that I love about it too, is that they're actually learning. What they're learning is they're learning 3D experience design. How do I create an experience for somebody? And it's very different from 2D and it's coming and AR is going to be a huge part of the next 20 years of how things are going to be, how communication is going to work, and these kids are going to have a Head start against anybody else because they've actually done it and they've actually put something together, tried it, had people look at it. Oh, you know what? I'm going to change this. I'm going to make it this way, that way, change it around, you know, and like, and really iterate and do something different.

Well, and even without the promise of like, AR will be everyone in 20 years, there's something very much to be said for like the design experience.

Yes, absolutely. And you know, and just getting to think about, you know, how do people perceive things? And I want to make sure I get in quick, quickly. Just mention the other two classes because what the sixth graders are doing is they're actually going to make a self guided augmented reality tour of campus. So we had them talk to the school administrators about the tours that actually do happen on campus.

They have like weekly tours, prospective families.

Yep. And then we talked about like, what do we want these people to come away knowing? What do we want them to come away feeling? And the students are working on creating that tour that I've told them, basically, if you do a good enough job, it's actually going to get used. And then the last one is the seventh and eighth graders are doing a project that's in conjunction with the new Museum of Los Gatos. And it's been really great day. I want to shout out Constance Buttons. She's been partnering with us at the new museum and they're going to create a puzzle game, a little sort of informational sort of scavenger hunt game in membit that is going to let people learn about objects that are part of the collection at the Los Gatos Museum. And why, how do they fit into the history of Los Gatos, which is part of their mission.

Oh, very cool. And then a museum is a space, right? So you can go to the physical space.

That's right. And actually one of the things that we figured out is the space where they keep all these objects is kind of cramped and small and like kind of not great for ar. This is one of the things like, you know, people do AR wrong. So much of it is small. You know, like there was like, you could have Wolverine could appear if you scanned a Coke can. Right. And I did it. And first of all, it took like 20 minutes to get it to actually scan. But then I had this tiny Wolverine in my kitchen. You know, he was like, he was like 2ft tall, you know, And I'm like, why, you know, make Wolverine six feet tall? Right? Like, why not? You know, and so like one of the things that I, what we're gonna do with the museum is that actually the tour is going to actually take place outside the museum.

Oh, that's really cool.

So you're going to start.

Well, so that's the kind of thing a museum could never do.

Right, right. Because basically one of the things that AR does for a museum is it says, hey, you know what, the whole city is your museum now. You can put stuff anywhere. And basically it's difficult for museums. I'm really grateful that the new Museum of Los Gatos is open to this because a lot of other museums can't get their head around the fact that they aren't a building. Like, the museum isn't really a building. The museum is really a group of people with knowledge and expertise who want to share that knowledge and expertise with.

Other people and about building community. Right. That's your fine arts background. Yeah. Is that art? Did you say art is about community art?

Absolutely. I mean, if you think about, like, if you think from just stepping back for a moment and thinking like from an evolutionary, evolutionary perspective, why do humans, why is art creation, all the arts, Right. Music, dance, painting, everything human, universal. Every single society in the whole world does it. Why? Because it brings us together, it creates bonds. Art expresses values, it expresses community values and it tells us who we are. And you, you create a community around other people who also love that art. And that's why it's so vital and so important. It creates community and it brings people together. And one of the things I love about ar, especially place based ar, brings people together in actual space because all this stuff that we're doing, where we're sitting at home looking at our screens and we're all completely separate from one another, it's not good for us. Like, people need to be in actual spaces with each other. And like, I think to the extent we can use digital media to actually bring people together in real space, that's a positive.

Well, and for me, I have a slightly different take on that. Like it, I think it's a yes and thing instead of a less screen time because for me, so many of my close friends now are educators that I've either initially met online, most of it is back in the day with the Twitters, or I've only met online. And so for me, having that and having those connections and finding those like minded educators that can push my view of education is super important. And having that physical space was something I know a lot of people really felt in 2020. Right, right. And so like taking a lesson from that of like, where can we have place based community AR does that. I think there's kind of one thing for somebody at a school right now to take away. I think the school tour is really, really cool because so much of tours, certainly when I've done them, for people who typically want to see how you integrate iPads into education in a meaningful, visionary way, I'll just do a tour and we'll just pop into various classrooms and you see what you see, right? Like, it's authentic and I'm not going to like knock on doors and say, hey, we're gonna do a tour and can you do something cool with an iPad? No, no, no. You see what you see, right. And there's a real authenticity to that that I think people appreciate in ways that a lot of tours are really cherry picked. But I remember a number of years ago, we actually had the Ministry of Education from Singapore come, oh, wow. And did kind of a more kind of traditional like executive briefing and showed them our book that we published as an Apple Distinguished School and showed them kind of all the examples and said, we're going to go and look. And fortunately, I prefaced it with, you might see some amazing things with technology, you might not. But here's what technology does here. It's not, we don't do tech for the sake of tech. It's always to like center student choice and voice. And so you're going to see that it might be with a pencil, it might be with a whiteboard table, it might be with an iPad. And we actually saw. I'm going to give everybody a chance to think, like, how many amazing technology experiences they had. All right, it was zero. They saw zero pieces of tech. But they were like, this was an amazing tour. Right. Like, they could see the whole story of it. And so that was great because it.

Wasn'T about the tech.

It's never about just the tech. Exactly how students and teachers can leverage it. Right?

Totally.

And so that was great. But how great would it have also been if I had said, hey, go ahead and download this app, join this channel, and we're going to pop into first grade, see some amazing learning, and then on the wall you can tap on it and you're going to hear some first graders. Because I'm not going to interrupt a class and be like, hey, three first graders, come talk with me.

Right?

I could. Depending on the teacher and the lesson.

That's right.

But probably not.

Right.

You know, certainly on a school tour every Wednesday, they're Not going to be like, knock, knock. Right. Can three first graders please come out like that? That's going to feel weird and disruptive. But then you just turn to the wall, you're like, oh, there's a pin right here. Tap on that. And there's a first grader being like, hey, let me show you what I did for fluency check. And they just read. And you're like, that's really cool. That's really simple. But that's really cool. You know, let me pop over to fifth grade. Oh, there's a photography class. Oh, I see them taking photos. Okay, whatever.

Right.

But then to actually walk down, like, the hallway or like the outside of the building and have like, photo after photo after photo, you're like, oh, now I see it. Right. And so I think the ability to take student photos and videos, not even 3D models, like, that's a whole different world. We should talk about that also.

Right.

But just to be able to, like, slap some photos on the outside of a building so as people walk by, they're like, oh, that's what they're doing. That's really cool.

Totally.

You know, it feels like such a next level and relatively simple thing to do.

Totally. You could do it. And I should have mentioned it's all no code. The kids can learn to do it in a few minutes. They take to it immediately. Yeah. And you know, the example that you were talking about where you. That's. We're actually going to do that. We're going to have. Some of the kids are going to have videos of themselves talking in different places about what they learned in that spot and why that place is important to them. Oh, and the other thing I was thinking about, I hear from a lot of educators, I don't want the kids on screens just using those words. I don't want them on screens. And the thing that I always try to get across is that it's not like screens are all the same. Right. And I think this is something that I really admire about the way you handle technology in the school.

Quality of screen time.

It's quality of screen time. What are you doing on the screen? And with augmented reality, the screen is not taking you away from where you are right now and what you're experiencing right now. It's actually augmenting the place that you're standing in.

And what was it giving you? Superpowers. Exactly. Nice.

Oh, that was nice. You took it right around. Exactly.

That's really what it is. Suddenly you're able to put like a cylinder and as you walk in this virtual cylinder, like, you feel like you're in it. Right, right. That's a superpower that humans so far can't do without a tool like that.

That's right, yeah. And it's mimicking. It's mimicking structures and things that have been common to humanity forever. You know, you create a space, you create a door. You create a liminal experience of going into the door. Being in a new space, it makes you feel different. And in this case, you know, with the Selfinder, it makes you feel connected to that student. Now they've made a space to tell you about them.

I love that so much. Well, Jay, thank you for joining us. And thank you all for joining us. For season eight, we're going to keep diving into augmented reality while we have Jay here. Like, we're just going to go deep into this. We're definitely going to connect with AI. Like, initially, I was thinking about doing a whole season around artificial intelligence, Apple intelligence, all the things. And that'll for sure there'll be places with that. But we're just going to go deep into, like, art and space and, like, authentic audiences. Yeah, it's an exciting time for that. So, Jay, thanks for joining us.

Yeah, thank you.

Everybody else, be sure and check out Membit.

M E M B I T co CO Yeah.

Not C O M. Right. Justifies much time. Not typing that. M. Exactly. Oh, man. Anyway, Jay, thank you for joining us, and thank you all for listening.