WEBVTT 0:00:00.000 --> 0:00:24.660 Programming Throwdown Episode 184: Asynchronous Programming. Take it away, Jason! Hey everybody. 0:00:24.660 --> 0:02:21.880 This is going to be a fun topic. I think this is something that I don't think they teach. This—well, you know, this gets back to my like I took so many theoretical classes; I didn't really take anything practical in college. But they didn't. I don't remember learning about this at all in college. It was extremely important and useful, so I'm excited for us to get into it. But before we do all that, I wanted to talk about something that kind of blew my mind that happened recently to someone in our neighborhood. Okay, so imagine you get a call and it's your son's voice, like it's actually—I don't know if it literally was his voice or if it was just a voice of a child his age, but basically it's this voice telling her it sounded like her son saying, 'Hey, I am...' Actually, I'm not going to say her son's name even if I could remember it, but—I'm so and so your actual son, right? And you know, I fell and I need some money to, like, you know, help me get to the hospital or whatever, right? And it's a scam, but it's like in your family member's voice. Like they've cloned their voice and it says it's from them because they probably go on your Facebook and figure out if you've ever posted publicly like, 'Hey, me and my son,' let's just say Doug.' So they call you and it's like, 'Hey, it's Doug. You know, I fell and I need you to wire me some money or something like that.' So, you know, the person who was a scam, but it really kind of blew my mind. I've been getting a lot of AI scams on my phone where they text me. I don't say—I want to fall for it? I fall for it in the sense like I didn't lose anything, but the first time it happened, it was like, 'Hey, you know, I'm in Austin at the Sixth Street parking garage. Where are you?' And I was like, 'Well—' 0:02:22.877 --> 0:02:53.961 Who is this? It's like, oh, you didn't save my number.' I'm like, 'Okay, no, I didn't save your number. Who is this?' It's like, 'Oh, I'm Jolene,' or some whatever the name was, a name I didn't recognize. And I was like, 'Okay, yeah, this is clearly—but when I feel like you know I could have proved that that one's AI, but I feel like people are using AI to scam people, and the one over the phone was definitely some AI thing.' Um, and yeah, I just wanted to throw that out there and get your thoughts. It's pretty mind-blowing. 0:02:54.501 --> 0:03:12.080 Yeah, be careful. I mean, scams in general are kind of an unfortunate—well, bad but interesting topic. I don't know how you say that. Like it's actually really unfortunate, and scammers suck. Um, I don't understand anyway whatever that's an aside, but I think I mean— 0:03:12.338 --> 0:03:13.857 It's big money, right? That's that's. 0:03:13.857 --> 0:05:13.179 Yeah, but still like um I a lot of the people that they scam, right? So part of it that's confusing to me is like, oh, this is an obvious scam. And it's like, well, that's part of it though. They want to kind of weed out people who are going to be like they want someone who's not going to catch on later and so you know they want someone who isn't going to ask questions, like, 'Are you sure? Who is this?' Or potentially looking for you to fall into it. It's sort of the, you know, when you're sometimes when you're young you tell these jokes where like you're hoping the person will give a certain reaction so that you can sort of like say something funny in response. And so they're sort of instead of saying, 'Who is this?' and they say, 'Oh, you didn't save my number.' And go, 'Oh, is it Sarah?' And then like, 'Yeah, it's Sarah.' And then you're like, 'Oh, it's Sarah,' and then but it's not right. Like they were just letting hoping you would say a name first. And like, you said it can be really easy to fall on these, and I do worry like you're pointing out AI on two fronts: scaling these out so that they don't even need someone to be attentive to all of these different threads that you know they can just mass-do them and have auto-responders and keep people busy until they like figure out something is scary. And then the voice cloning, eventually video cloning. The video stuff is getting to the point where everyone goes, 'Oh, it's so obvious.' It's so obvious. And they do like some frame by frame, like look at the—and it's like, but who in the small little window when you're scrolling through social media can see like this frame by frame pixel analysis where you know the pinky finger glitches out for one frame or whatever? It's, you know, I remember you and me discussing on this: The Will Smith eating spaghetti. Oh yeah, that's right. That wasn't that long ago. Yeah. Now now, like I saw some video from a football game over the weekend where one of the coaches had an issue and somebody made an AI video of him, like basically super vulgarity, like dropping, like saying how bad it was or whatever, and it just turns out it was like completely fake. And oh wow, they—the— 0:05:13.179 --> 0:07:09.160 they said it underneath, but you had to like click it and look for it. A lot of people probably reshared it thinking it was funny. Like, 'Look, the coach,' like basically going off on reporters and stuff.' And so it's becoming more and more prevalent, and it only takes you being off guard for sort of a moment to get wrapped up on this. Um, so yeah, definitely be careful. But I actually have a flip observation about this. Okay, so you mentioned like, you know, we all kind of go, 'Oh yeah, voice cloning, it's dangerous.' You and I have lots of, you know, audio out there. It's, you know, whatever. But now like you see demos on YouTube or people showing, hey, from even just maybe 10 seconds of an arbitrary sentence, they can sort of—you can use a different voice or clone the person's voice, you know, do voice-to-voice sort of transition. Yeah. I so I've seen these before, so then I wanted to do this for for basically like an audio enhancement thing. I was—I had some audio I had recorded somewhere and I was like, 'Oh, I want to use voice clone,' which I know should be possible to, like feed it my own voice from a podcast recording or something high quality, feed it the like not good quality and have it basically reread that stuff in the high quality voice, which is a use case that's supposedly possible. And then you immediately bump into the fact that like it's actually really difficult. Like as much as people say, 'Oh, there's gonna be this,' it's non-trivial. Like I guess there's a lot of paid solutions, but it's often unclear if the paid solutions are any good or if they're just like the open source stuff that someone's charging for. But like cobbling together all of the—like you go download this model, you run it on ComfyUI, you do that. Like it's still in the realm of like this is difficult, reasonably difficult to access.' So for all of these AI things, everyone's like, 'Oh, it's so easy to do this,' and then or generating an AI video, like we all know we've all seen some crazy AI video that was hilarious or partly convincing. And then you go try to do it, and I don't know, maybe I'm just not smart enough or paying enough. 0:07:09.160 --> 0:07:20.367 Attention. Um but they seem a little cherry-picked. Like people may have gone through a lot of work to cherry-pick the one convincing video, and if you go do it, you think it's going to be magical, and it's actually a little disappointing. That's been my observation. 0:07:20.451 --> 0:07:55.719 Yeah, I think there's two things there. One is um, you know, definitely it's the open source stuff is not trivial to get up and running. I do think that the paid solutions are like, you know, pretty accessible, but—but you're also right that that um you often have to iterate a lot, and you have to get it's almost a skill in and of itself getting it to sort of behave and feeding in the right sort of negative prompts so that it doesn't go off the rails and all of that. So, yeah, that's a whole skill in and of itself. 0:07:55.719 --> 0:08:29.031 Yeah, I someday it is getting better fast, so I think we are getting close. But it's one of those things where I was thinking about this the other day. It's like if you could describe where we are right now with AI, people be like, 'Oh, that's going to be so magical! It's going to change everything in the world!' Like and then we got here, and it's like, 'Oh, huh?' You know, like I—I don't know. Like an AI could clearly probably pass the traditional sort of Turing Test. I mean, maybe that's controversial. And then now it turns out like, oh yeah, but it didn't just solve all of life's problems, like. 0:08:29.031 --> 0:08:45.800 You know, yeah, yeah. I think that to double down on that, I think AI is just not going to create the kind of monetary value that people thought it would. You know, like the thing about it is the reason why Fang got so incredibly— 0:08:48.741 --> 0:09:35.097 rich is that now people can like be super productive anywhere, you know? And so like you could be waiting in line for a ride at Disney and like working. You know, like you could be answering emails, talking to people on Slack. Like that was something that in, you know, 1999, you couldn't do if you're waiting in line for an hour at Disney; you couldn't do your job, right? And so, you know, combination of Apple, Google, for Search—these other things like allowed people to be like super productive when they're even when they're idle. And for AI, it might it might do that, but it hasn't done that yet. It might take a lot longer to get there. 0:09:35.772 --> 0:09:51.195 Okay, we're way off the AI scam thing, but I have heard some recent interviews along the lines, and they said actually like the the thing, the like what will be the inflection point or the turning point is when all of these advances sort of make it into robotics. I 0:09:52.039 --> 0:09:52.799 Could see that. 0:09:52.799 --> 0:10:19.039 Do you think that's like a different economic equation? Like when not just oh, you can be taking the AI and making programmers more effective or making people's email responses better, or like sure all of those things. Maybe economic value is a little, but like if you had the level of improvement we've seen there in sort of like robotics, is that going to be a major unlock? It totally. 0:10:20.035 --> 0:10:33.855 Totally. Yeah, so here's a very simple example. Like think how many parents have to either change their work schedule or do something kind of wonky to just shuttle their kids around. Oh yes. 0:10:33.855 --> 0:10:34.919 You know. Yeah. 0:10:34.919 --> 0:11:04.332 Yeah, just driving. It's like oh, I pick one of my kids up from school. I drive them to the soccer field, and especially when they get older, they don't even want—you know, they don't want parents to be sitting there like watching the practice. Yeah, exactly, go sit in the car so everyone can see you. Yeah. Yeah, so like you either go home then or whatever. Like that's just one example. Like you could have a robot that goes up and down the street taking everyone's trash to the curb. I mean, there's just so much. You get on lock with robotics, but I think it'll take a while all. 0:11:06.120 --> 0:11:25.071 Right. Yeah. Well, let's keep—let's keep moving through the agenda. All right. So yeah, don't get scammed. Don't get scammed by careful folks. Be careful and warn your parents and grandparents or you know family that like as well who may not be as aware that that's a thing. Yeah. Very true. Very. 0:11:25.071 --> 0:11:28.969 True. Okay. All right. What's the first news of the show? All right. So? 0:11:28.969 --> 0:13:25.180 I have—I have no news this time, but I do have two good video links that I wanted to shout out. And so the first one was a recent video by Sebastian. I actually didn't look up how to say his last name. I know. I'm not going to try to say log log. Okay? Okay. Anyways, and it's 'Coding Adventure: Ray Tracing Glass and Caustics.' This is a series of videos that Sebastian's been doing. They're sort of like in a similar vein as, I guess, like three blue one brown. Like you know, it's like sort of a nice quiet pace but like deep content and just sort of like working through something. So they're not like five-minute YouTube Shorts; they're sort of longer form videos. And he's walking through the process of coding. He's been working on a ray chaser recently, but going a little step further than normal. And so in this one, he's sort of talking—he gets to talk a little bit about like physics, physical-based rendering, and sort of like how glass has certain attributes and indexes of refraction. And sort of if you build up from like really simple sort of rules, you get very complex interactions. Almost becomes a physics simulation, right? Like the caustics is the like very bright shiny thing that if you have like a glass object, you get—you know, sort of like rainbow diffusions in some areas and some areas you get, you know, like very bright shiny spots or whatever. And if you play video games often, you won't see those kinds of things because they're not really meant to be simulating physics; they're meant to look nice. And right fast. Yeah, but in ray tracing, you can be kind of more physics-based, but a lot of videos on his channel that are just like really, really good. But this one's really nice. And then the reason why I wanted to bring it up here other than like, you know, I mean maybe this is really cool to you, it's great background photo. This is kind of tends to be the YouTube I watch. I try to explain to people there's people who watch YouTube and it's like, you know, more the Shorts content or entertaining. 0:13:25.180 --> 0:15:10.920 But there's a lot of this like science engineering kind of stuff out there that I feel is at least a little bit better than—I'll just call it brain candy. And it's sort of replacement for, you know, the TV growing up. We were like what Bill Nye, The Science Guy, or something that would be like, you know, a little and even that was high strung, I guess, but you know teaching science concepts in an entertaining way. And there's a bunch of people in this vein, but I will say that I also began to realize that a lot of the classes I took in university, you can kind of end up not building a curriculum, but if you just watch some of these channels about how microprocessors work or people kind of building their own circuits—some of this stuff about ray tracing—you get a lot of what a traditional at least when I was in college sort of like Computer Science degree would cover. So if you can kind of get through the basics of programming, learning about like graphics, learning about like data structures, he gets into debugging because by necessity he like oh, here I didn't check the sign properly or I did a dot product and I didn't think about like this set of inputs. And so you actually end up backing your way into a lot of that education. And so I think that's really an interesting thing to say if you can—if you find this at all entertaining, I think there's like a definite learning ability that can come here from sort of watching this and sort of seeing how how they kind of go about problem solving and working through things. And it is a little bit more entertaining than just watching, you know, I don't know if he does it anymore, but not choose to do like live streaming of him programming or whatever. Like and I guess oh really? We'll do. Yeah. Oh, we should do that. No, no, no. I don't know. I've never been able to watch one of those. This is a more edited form of that. 0:15:10.920 --> 0:15:19.634 Cool. Um, I'll have to check that out. I used to do a lot of that ray tracing stuff back in college, and it's very. 0:15:22.232 --> 0:15:29.219 Fun. All right, my news is actually okay. So it's kind of related to the voice. 0:15:30.332 --> 0:15:31.092 Cloning? Oh. 0:15:32.492 --> 0:17:06.030 okay um there's a new open source model called higgs audio v2 and it's from a company called boson ai and um it's it's like really really good um so historically this company called 11 labs had been kind of dominating the space so if you want to do voice cloning actually 11 labs is something i've never seen before to this day which is you can describe a voice and then say what you want that voice to say and it'll do it so you could say uh you know an evil smelly ogre says who's in my dungeon and it'll actually like make up a voice for that description and then say who's in my dungeon and that voice is just made up um this doesn't do that but but 11 labs is paid though right 11 labs is paid that's right um but this is totally open source and um it can clone a voice so you give it you know an mp3 of you kind of narrating something and then now it has your voice it could say anything um it handles like you know you can do kind of like in a movie script or in brackets you say you know laughs or chuckles or whatever and it'll actually laugh while it's talking um it's really really really good uh it's definitely just like heads and tails above anything i've seen um in the open source world so um i'm gonna try and play with it i have some funny things i want to do i'm not i'm not gonna run a scam call center oh no no no please 0:17:06.030 --> 0:17:06.942 Don't, but I have 0:17:07.060 --> 0:17:28.699 Some other funny things that I would like to do with it. Um so I'm gonna try and mess around with it, but I saw a demo that one of my co-workers presented where they cloned their own voice, and it sounded really believable. So it's like we're getting to that place you talked about where anyone with the GPU could just do this. 0:17:29.909 --> 0:19:20.220 Yeah. So what is the I guess for examples? Like a lot of these they don't need to be like super fast real-time. Like I know they could run in a CPU, but I don't feel maybe we're just still too early. So for text LLMs open source, and and maybe it's just because I've never spent time there, there's something like LM Studio kind of gets there. So there's like a GUI that you download, and then from there it offers you links, and I hope someone in my family was trying to use some offline LLM stuff, and so I was able to walk them through pretty easily. Like you go there, it offers you models that are appropriate for your computer, tells you if you're big, tells you if they're too small, has filtering, then you download it, gives you the nice traditional sort of like ChatGPT style, like you can type in text in the bottom. It has prompts in it; it'll answer, but it has like you can go deep on it, but it sort of harnesses a bunch of the open-source stuff under the hood so you can do like traditional chat app with one of these offline LLMs. You can switch models midstream and it'll just feed the context into the new model. Like it kind of just works as a graphical user interface, but for a lot of these other ones, like the image generation, I don't feel like it's as good. Video generation? Not at all. These like audio ones sort of—I've never seen anything like that. I feel there's still like a bunch of this. Maybe we're just still too early, but like single front end. So we're audio, right? You're either wanting text, you want like the thing, the target, the source. What are they? Like there feel like there's only a few moving pieces. Maybe there's just not enough users, but it feels like it'd be really easy to just build an app that's like a front end and handles—like I'm looking at the GitHub link for the one you were just talking about, and it's how to set up Conda or virtual environment, or it's like. But I'm never going to describe to someone in my family how to do these steps. Like they will never be able at this level to use this. Yeah. 0:19:21.149 --> 0:19:38.969 We're still—we're still not quite there yet on the user experience. Totally agree. Um I think the quality is getting there though. Like the Flux text-to-image model is really, really good. It's mind-blowing. It just needs a better user experience, and 0:19:40.082 --> 0:21:02.466 The new one we don't have it in our news, but the new one Google did NanoBanana or whatever. Have you seen this one too? I played with it a little. Yeah. Uh specifically for like image editing tasks. So like saying, 'I have an image and I want to change it in some way.' Like I was asking it to do like people removal from background, and it's—it works really good. Yeah, it's kind of scary good. Yeah, now that one's paid, right? That's a Google product. But still like yeah, you're right, we are getting really close on the image stuff. Yeah, totally. Okay, my next news article is a YouTube channel as well. Okay, that's a thing anyways. And this is on my—as people have listened to the podcast before. No, uh Machine Learning AI is not my background, but um understanding sort of how the backpropagation works, how the gradient descents and sort of the model trainings is all very like interesting to me. I had a book of the show a little bit ago that you know kind of similar or maybe it was even a news article, but I think that I'm continuing to try to learn and sort of work my back not just as a how do you call it practitioner, like user of these things, but trying to understand some underlying thing. And I think there's some concepts there that can cross-apply to other sort of like traditional computer science sort of applications, so it's just definitely like an interesting thing to learn about. 0:21:02.466 --> 0:21:14.886 It is pretty wild to see AI go kind of mainstream. It's been just a really weird experience. Like all these people are now talking about stuff that was like really not that cool not that long ago. Yeah. Oh, 0:21:15.882 --> 0:21:25.028 I—I'll tell you off that it's just an observation about that, but I know you've been in the field for a very long. You were in the field before it was cool and now it's cool. So like that makes you cool. 0:21:25.551 --> 0:21:30.225 Maybe? Yeah. I don't know or maybe it makes me too early. I don't know. We'll 0:21:30.512 --> 0:21:38.055 Find out. But too early everyone's too early for something, right? That's the thing. So even now you might be too early if it grows another 10x. So yeah. 0:21:38.055 --> 0:21:39.979 That's a good point. It's like Bitcoin. Oh yeah. 0:21:40.384 --> 0:21:42.527 Well, I wasn't okay. Yes, that was what I was. 0:21:44.350 --> 0:22:58.853 Um, okay, back to my focus. So the channel name is Welsh Labs. Um, and they have a sequence of videos that um they were doing called How Models Learn. So this is the first video that I've linked in the show notes. You can just look up Welsh Labs. Um, it's called The Misconception That Almost Stopped AI. Now, I am not Jason, so I cannot vouch for the sort of like how truthful versus narrative the backstory is or any of that stuff, but they're kind of talking about gradient descent and how if you give the sort of how I understand gradient descent to work—the very simple, you kind of heard about it in computer science classes, you've seen it on the internet once or twice—and the way that you would kind of assume it might work, it just feels like you would get stuck in local minima all the time with how big these models are, right? Like these models, the dimensions are really high, the parameter counts are obscene. Like you would just say there's intuitively—that was sort of what I thought too. I didn't really understand the unlock of deep learning and sort of like what happened because in my mind, you know, you kind of picture the 3D scene with all the noise in it and some well in the middle, but how do you get there? How do you know where it is without just searching? 0:22:58.853 --> 0:23:06.801 There's the what is it called? The like it's called a Gaussian, but like that's sombrero hat function where you kind of get trapped outside of the hat. 0:23:07.257 --> 0:25:05.420 Yes. Yeah, so I—you're right, and the video series kind of walks through the same thing. Like this is the simple explanation about gradient descent. Here's why it would work, except you like as you point out, gotcha, like it doesn't really work, not really a gotcha, but it doesn't really work. Except wait, it did work? Why? What happened, you know? Like they walk through how some of the things about higher dimensional spaces, about batching, about some of the techniques that are applied sort of help actually almost always guarantee you sort of get where you want to go to some degree. Um with—you know, obviously probably some caveats—but just a really well-handled explanation. And they do a couple really interesting things I've not seen before. So rather than just sort of give it as a simple explanation, they actually take some deep models, some like early versions of GPTs, and extract some of the layers and sort of talk about what is happening sort of like visually, saying, 'Hey, at this layer, if we go here in this actual model—maybe not state-of-the-art, you know, I trained it—and this is what you kind of see developing over time as a consequence of this technique.' Um and then later on, even after this one of the videos, is talking about sort of how you would take coordinates and divide them up with a sort of classifier into boundaries for a really complex country boundary. It's one of these ones where like there's a city which one polygon of the city is a part of one country and the rest is another. It's like some really weird place, and how that as you sort of take these linear equations, stack them up, do these other techniques that you can sort of start approaching a generalized representation of even very, very complex sort of shapes and and spaces. And I just really think it was well-handled. It really helped me. Um it's one of those things like I kind of had to watch it again just to kind of like, 'Oh, okay, I see.' 0:25:05.420 --> 0:25:43.840 Where it's going, but definitely also kind of entertaining. So if you—I guess like me have some understanding of some of these things, but there's a gap, right? There's a gulf from, 'Okay, I kind of know what gradient descent is; I could have a casual conversation about it,' but how that ends up getting applied to giving us, you know, these AI models that we have today. Is there some steps missing in between? And this isn't going to take you all the way or make you sort of like able to design state-of-the-art models or improve them, but giving you better glimpses into sort of some of the techniques that are used to cross that chasm. 0:25:44.869 --> 0:27:43.760 Yeah, totally. Um yes, it's really interesting. I think um it is pretty remarkable that it works. I will say that people get hung up on a lot of these like 'It's not optimal,' or 'You know, theoretically this can't work,' or 'Doesn't work,' or etc. And my always my response to that is like it just has to be better than what's out there now. So if what's out there now is nothing, it just has to be better than nothing. Um and so the bar is just a lot lower than like the theorists want it to be. You know, I actually think the thing that almost made AI not happen was people's fear about getting wrong answers. Like remember when you know GPT was in its infancy and Facebook released this thing that like wrote research papers? I don't know if you remember that. Um I forgot what it was called, but you know it was panned, and someone made a research paper where it's like um the benefits of like drinking urine or something. It's just some bogus thing. And you know because it's like um you can talk past the sale with AI. You can say like, 'Hey, there's incredible benefits in drinking urine; write the research paper that kind of explains this phenomenon.' And it'll kind of take what you said for granted and then just go off and do the thing. Um So you can trick people, you know? That's what people are doing with the research paper thing. But you could also just encounter problems like the famous 'How many R's are in strawberry?' or 'Should you put glue on pizza?' It was all these famous examples where they always gone wrong, and people were just really fixated on um on all these on on the worst-case scenario. And there's actually like a—there's like an I don't call it a mental disorder, but there's 0:27:43.760 --> 0:28:42.480 like a—I went to this like career coaching thing. It was like a big seminar, a lot of people, and the lady who was giving the seminar spent a bunch of time talking about worst-case scenario thinking. And she's like, 'This is one of the biggest problems I see as a career coach and as a life coach is people doing worst-case scenario thinking and not thinking about average-case scenario,' or also best-case scenario, but more importantly, average-case scenario, right? And so there was a—there was a phenomenon of worst-case scenario thinking with AI for a long time. And people you know just didn't want to release something like ChatGPT, and I will give OpenAI a bunch of credit that they basically put it out there, and got it to be really popular. And as you know, they got a bunch of pushback. They dealt with that in a way that allowed that to eclipse that problem. 0:28:42.480 --> 0:30:10.229 Yeah, and I think it is still like people talk about the hallucinations and the incorrectness. I mean, I think it's not fully solved, but like you said that's not meaning it isn't beneficial. We just have to learn, you know. I think the same to your point. You kind of say take almost anything we have today—oh, a car. Like a car is dangerous; like cars are going to kill people; they're going to drive fast. And it's like, 'Yeah, that's true.' Um but like there are still benefits. There are still other things, you know, we have to learn collectively, individually how to be responsible with the use of tools. And yeah, I think always with new things there's a debate about usefulness versus danger, I guess. And you said worst-case thing. You can always hyperfixate anything we do has sort of danger, right? Going outside and exercising. People someone told me, 'Oh, you know, you exercise, you can have a heart attack.' Like people say often when you exercise, you like you can cause a heart attack. And it's like, but by X—like but also by exercising, you are lowering other issues that lower the probability of having a heart attack. So like, you know, so your solution is like sit inside, don't go outside, don't exercise, don't move. Like you're gonna you're gonna die from that for sure, you know? And so it yeah, these balances can be really difficult to find. So okay, wow. We took that in a different direction, but totally. 0:30:10.229 --> 0:30:26.007 Well, I guess it segues nicely to my last news, which I won't dwell on because we spend a lot of time talking about news, but it's basically an interview with Peter Thiel, and there's a lot of content there. But the thing that I took away that was really interesting was you 0:30:27.762 --> 0:31:41.980 know at one point the interviewer says, 'Why AI? Why is there so much focus on AI?' And I expected some kind of answer like unlimited productivity or, you know, wages can go to, you know, the price of oil or something. But actually what he said was um it's the only thing—like AI is like the only really exciting thing out there. And that was pretty mind-blowing. It's like, 'Is that, you know, if that's true, like why aren't there more exciting things?' And and maybe like I do feel like there is a tendency for everyone to kind of chase one thing at a time. Like remember when CarPlay and Android Auto and just getting stuff in the car was such a big deal, and all the companies were copying each other? And now we're kind of seeing that with AI, and it's like we just for some reason as a society we get like just fixated on one thing at a time. But I thought that was really interesting. I wonder if if this is like a recent—he's he's trying to make it sound like this is a recent thing, but I wonder if we've just always just been fixated. 0:31:41.980 --> 0:33:16.731 On one thing. Interesting. I think the in a similar vein what I've heard a bit as well, like the focus, the drive towards it from a variety of facets is it feels close to something like akin to an escape velocity. So like getting into a post-scarcity world where you can get whatever level of like engagement you want with it, but basically if we can develop an somebody can develop an AI system that becomes sort of superintelligent and then you can assign it to the task of improving the AI systems—somebody will basically like break past the barrier, right? Get into orbit, whatever reach escape velocity. And then they it'll be like winner-take-all. And so you see a press from country level, from company level as almost an existential risk because if some other company does that, then they will have a system which can basically build any system. And so search, you know, Chat, image—like we'll just get won by somebody who can basically devote all of the computational power they have to just building all of the other competitor stuff, and then those competitors will only have whatever resources remain from their sort of war chest to spend to also get there, or they'll just be drained of you know financial inputs. And so maybe it's a bit the same. It's like the only interesting thing, but like there is also this and 0:33:18.790 --> 0:33:40.255 maybe it's like focusing on the danger part. You were saying it's like there's this focus that if you could get to this sort of recursive AI that improves itself, then that will be so transformational that whoever—if someone gets there first, like before everyone else by enough of a margin, that the like everything changes. Yeah, it's 0:33:41.234 --> 0:33:45.840 Interesting. I wonder to what degree that's true. 0:33:48.574 --> 0:34:13.735 You know, I feel like it. Yeah, it's a really interesting perspective. I think that people, I think that okay, even if the AI can do general things, when you go to evaluate it, you're going to evaluate it on your specific thing that you care about. And so in that sense, like that part is not going to scale, but 0:34:16.570 --> 0:34:55.838 But maybe we can get the evaluation to be somehow like okay. In our case, you know, the Earth is our evaluator. You know, it's like we went out and we hunted a tiger, and one person, you know, got the tiger, and the other person didn't, and that second person goes hungry. And so we have the planet Earth as our early man's evaluator. But you know, now we have to have synthetic evaluators for synthetic problems. I wonder if maybe that's like the next big job title, you know, AI agent evaluator? It's like the new Data Scientist or something. I mean. 0:34:56.378 --> 0:35:42.548 There. Yeah, people are talking about in a similar vein, I guess moving to completely synthetic input instead of using sort of Wikipedia, The X tweets—well, X, right? What they're called anyways. I think it's X. Yeah, the X's, the X... Oh, The X is on X. Yeah. I don't know, the Reddit posts, you know, basically instead of all of the corpus, it's just not enough. Like can you move equivalent to AlphaGo? You know, moving from the recorded games of Go to just self-evaluation? Like can we shift that and like what happens if you invent basically like a parallel universe of input and train on that, and like what does that unlock or enable? Yeah. 0:35:43.172 --> 0:36:29.562 So to use the robot example, you know, can the robot effectively know if it when it goes to take your trash can to the curb, can it effectively know is high accuracy whether it did a good job or not? You know, I mean, I guess it could take a picture of your trash can on the curb, but like if the robot did a bad job, maybe it also does a bad job of knowing if that picture is correct. Like maybe what if it dumps your trash out in your driveway and then takes a picture of that? It's like I did a great job. You know, so it's it's like it starts—it's just kind of you have you've sort of moved the problem, and you still have a problem, but 0:36:29.849 --> 0:37:37.889 Maybe that is actually going to be one, and that's the self-driving thing you were kind of alluding to. Car stuff like it's a bit difficult fitting in current systems, but I mean, I think maybe the what about if you think about the whole solution so that the job well done is did the trash end up in the dump? And that's did the robot system take the can to the curb? Did the garbage truck robot pick up? Was it able to successfully pick up without like—so maybe the garbage can knows the mass of the trash that was in it and the mass that comes out at the end, and if that ratio you know is less than one, you get a bad score. If it's over one, you get a bad score. It needs to be exactly the mass in is the mass out at the end. Maybe you have to account for evaporation or something, but like yeah, and then actually it becomes this end-to-end measure, and you have to attribute to like you can improve the garbage truck, you can improve the like roller autos, you can—you know whatever it—and it tries to iterate collectively. So for a while we just end up with like giant dump trucks that back up to your house and try to like go into your garage door. You know, you're crazy stuff. Yeah. 0:37:39.863 --> 0:38:00.670 Yeah, I think I think that's kind of where we're stuck now is we're using human—even you know OpenAI, all these companies are using people to evaluate the models. And that's where we have to get beyond it. The models need to be evaluated by the Earth somehow, by the environment, but 0:38:01.547 --> 0:38:08.989 I don't have a counter to what else is super interesting right now for everyone to focus on, so I can't disagree with the thesis from 0:38:09.158 --> 0:38:21.940 Yeah, me neither. You know, that's it. That's what made it so fascinating. Like if I could point to like oh, you know, gig economy is still a thing or something, then you know, but there really isn't anything else which is kind of wild. 0:38:21.966 --> 0:38:23.282 Like VR, I mean. 0:38:24.025 --> 0:38:28.041 VR is dead. Maybe AR? Potentially. AR would be cool. Space. 0:38:28.699 --> 0:38:31.517 Travel, but like it feels high barrier to entry. 0:38:33.255 --> 0:38:55.660 Yeah, you know space travel. I have a hard time getting excited about space. Um, I don't know why it just doesn't really like it doesn't sit—it doesn't excite me for some reason unless there's like gold on the Moon. You're like dead inside, dude. Well, maybe I can only be excited about one thing, and that's AI. Maybe it's a me problem. 0:38:57.994 --> 0:39:03.023 What about AI in space? Oh, mind is blown! Dyson's Sphere program to power out. 0:39:03.023 --> 0:39:47.033 AI. I can't get back into that. All right, all right. Time for Book of the Show. All right. I will confession corner. I have not had a ton of time to read recently. I have not made a lot of time to read recently, so I went back into my history of books and I'm going to give a shout out to The Hobbit. If you have not read The Hobbit, if you've not sort of dove jumped into The Lord of the Rings—delved, dived, yes, spelunked—if you've not picked up your hammer and gone with the Dwarves, the dwarves to Mine, and okay, I'm gonna stop because someone's gonna call it out. You haven't. 0:39:47.033 --> 0:39:48.552 Gone to Moria. That's what I was. 0:39:49.160 --> 0:41:12.900 Gonna say, but I was like, I don't know. I'm gonna—I'm gonna use it wrong because like I don't remember which one's the one that had the problem anyways. Um, The Hobbit is a sort of good entry into the world of The Lord of the Rings of J.R.R. Tolkien. Obviously, a classic. The Hobbit is, I will say, like a pretty easy self-relatively self-contained read. Um so definitely worth picking up if you've never done. I do think there's a lot of folks and disagree who kind of said this. It's J.R.R. Tolkien sort of set up a lot of the archetypes that we have today of sort of dwarves, elves, humans, and like some of their characteristics. People complain those tropes are overplayed today, but if you read modern fantasy, there's a lot of built-up stuff that you can kind of trace back um to the origins, and I think it is a good story. I think sometimes people will say, you know, like for me the first time I watched The Three Stooges as like, but I didn't find it that funny. I'm like, I've seen these shticks before, but it's like yeah. But when they did it, it was like they were the people, like they invented it, but other people took it on. I think The Hobbit to me doesn't really—I don't necessarily like suffer the same thing. Like you can read it and it is still good even though other people have continued to like pick up and and sort of run with those concepts. Like itself is still a sort of fun and exciting read. 0:41:12.900 --> 0:42:07.011 Yeah, totally. I think if I remember correctly, it's been a long time since I read it, but there is a ton of world-building, and um um because it's older, it does go at a slower pace. I feel like you know we're such dopamine fiends in the 21st century that like it's uh it feels like um like we just can't have something that goes at a slow pace. If you've ever seen like old movies from the '50s or even like Space Odyssey, you know these kind of movies they go at such a slower pace. It kind of blows your mind. It's kind of the pace has crept up over time and it's like a boiling water frog type of thing, but um so it is going to be a slower pace and a lot of world-building, but if you stick through it to the end, I think it's a phenomenal book and it leads you into all the other Tolkien books. Yeah. 0:42:07.905 --> 0:42:40.460 I've always heard it like called prequel to The Lord of the Rings, but it was written before The Lord of the Rings, so it's I don't think it technically qualifies as a—as a prequel, like it was the book written and then Lord of the Rings was written after. So I don't know what you call that, but it does the story contained does precede the rest of the story of The Lord of the Rings, which is even a larger tome, even I think actually a little slower, a lot more world-building. But Hobbit is is definitely shorter. I think it's even considered kind of a kid's book, but you know don't let that scare you off if you're an adult, but it it should be a pretty quick read. 0:42:40.460 --> 0:43:35.909 Yeah, totally. All right, my Book of the Show is even more of a cop-out. It's what I've been doing instead of reading, which is maybe a shame, but it's an New York Times game, so this is an app. A lot of the games are free. You can subscribe to get access the crossword puzzle, the famous crossword puzzle of New York Times, and some other stuff, but um but it's very fun. It's a simple app. It works offline, so if you're off the grid, you can still use it. I think I think it's something where like after a couple of days it'll stop working, but if you go off the grid for, you know, an afternoon or something, it's fine and uh it just kind of keeps your keeps your mind going. It's got Wordle, the famous Wordle, but it's also got, you know, a few other ones, and it's a nice kind of thing to help you kind of build your vocabulary and literature skills. Um if if uh if you want to kind of continue to sharpen that. 0:43:37.157 --> 0:44:14.029 Axe. So I did see something that Wordle will have to end in 2027. Did you see this? They're gonna run out of words. Yeah. So basically they have like a dictionary of 2300 words. They're on like whatever in the mid thousands, and so like in 2027 they'll—they'll run out of like words, and then people point out, yeah, but they could just expand the dictionary sort of change the rules. It's only like under their current rule set of not allowed to repeat words um and stick to the words in that in that sort of like word list. Um but yeah, I guess in its current form it it has a finite number of like puzzles to have from its sort of. 0:44:14.957 --> 0:44:21.690 Dictionary. Man, it's like we thought Y2K was fake, but it's real. It's just coming in '27. Yeah, the world's gonna 0:44:21.960 --> 0:44:26.449 end. What's gonna happen? They'll have to release Wordle V2. They'll have to add an extra letter. 0:44:26.820 --> 0:44:27.620 Yeah, that's right. 0:44:27.620 --> 0:44:36.220 Oh man, they'll split it. It'll be Wordle Lite; they'll take away a letter and then Wordle Deluxe. 0:44:39.139 --> 0:44:41.620 It'll add a letter, and each one of those is 0:44:46.244 --> 0:44:47.307 a microtransaction. Oh man. 0:44:47.307 --> 0:44:51.137 Okay, for two dollars you can have a Wordle that is not expired. I'm terrible. 0:44:51.357 --> 0:44:56.031 At word games. So, I thank you for this pick. I won't be picking it. 0:44:58.579 --> 0:45:01.296 Up. You can. I'll do it in space. How about? 0:45:01.296 --> 0:45:18.711 That way we both get infuriated. I just look up the answer because it makes me feel dumb, like I can't even get good guesses to keep progressing. Like I know I have. I just purposely make bad guesses just to try to get more clues, and it's like I know I should be able to form, I can't, and then I just like rage. 0:45:19.150 --> 0:45:49.035 Quit. Oh man, I have to confess or admit. I am a Wordle god. I am actually so good at Wordle. I actually sent one to my wife the other day where it was like the first word only had one letter is out of place and then the second word I picked had no letters. So all I had to go on after two words was one letter that was out of place, and I still got it on the third try because I just eliminated so many vowels and and other things that like there's only one thing left. I got it on. 0:45:49.035 --> 0:45:51.420 My first try. I looked up what the answer was. 0:45:54.975 --> 0:46:01.034 Yeah, take that. Remember for a while Scrabble was really big with friends or whatever? And oh, I just like 0:46:02.873 --> 0:46:12.576 Straight up cheat. Yeah, I used to play. Oh man, I played so many games of Words With Friends, but you're right. It had to be with friends. You couldn't play anonymously or you just get wrecked by a person who's cheating. 0:46:14.432 --> 0:46:54.983 All right. Or let's move on. Oh man, all right. And if you want to support our New York Times Addictions or our trips to space, please give us some money on Patreon. I'm just kidding. Actually, none of that money actually goes to us. We put all of it back into the show. We've literally haven't taken a dime of it, but we have used it to do advertising and reach out and get guests, and all of that good stuff. So if you like the show, if you want the show to reach more people, please support us on Patreon. It's not technically a non-profit, but it is a non-profit. We go and put all that money back into supporting the podcast. All right. 0:46:56.873 --> 0:46:57.970 For tool of the? 0:46:58.140 --> 0:48:03.340 Show all right. What's your tool of the show? Patrick, it's a game, Shocker: Escape Academy. I looked—I thought for sure I must have shouted out this game like on the show before, and if I did, one, I don't care because it's worth it too. I searched through all of our show notes and website, and I couldn't find it, so I don't care. I'm double down on it, doubling down on it: Escape Academy is a great game. It's about playing an escape room with a story, a narrative about being in an academy that challenges you to solve puzzles and like, you know, has a kind of life or death setup situation about problems, but in reality, it's just like a really good, well-paced, you know, escape room. Some I've done before, and it's just like really—like how was I supposed to know? You just need to randomly kick around until you find this thing where the path to progress is non-obvious. I don't like the sort of just search around and find; that's not my cup of. 0:48:03.340 --> 0:48:55.977 Yeah, I was talking about this with a friend of mine about my cousin when we were nine years old. He made a choose-your-own-adventure game in BASIC on our Commodore 64, and it was like you're in a town. There's a bank, and you can also walk down the road. Which do you want to do? So it shows you know, 'Walk down the road,' it's like you meet a guy, you die, the end. Yeah, it's like okay. Yeah. So when you whenever you make one of these, you know, point-and-click adventures, choose-your-own-adventures, it's like you have to somehow very subtly hint to the person what the right answer is, but it has to be subtle enough that when they pick the right answer, it's satisfying. And it's like I feel like there's an art form to that. That's—I've never made one of those, but it feels actually a lot more complicated than you would think. I'm sure it. 0:48:55.977 --> 0:50:51.120 Is. And I'll say for me the subtlety of the answer can be problematic, but even just understanding like what the puzzle is. So in your example, like making sure that it's clear that there's some bit of information you should figure out to make the choice. If you just present it as a trivial choice and I make the choice and then like, 'Oh, game over.' It's like that—that's just, I'll just quit. Like I don't want to. Like this is not—that's not fun. And I did play one the other day where there was like a hidden number like you know sort of in very faint sort of color that you were supposed to see, but I didn't see it, and then I just kept bumping my head against the wall. I asked for a hint using the hint, I was able to solve a puzzle I wasn't supposed to be able to solve yet, and then I bypassed like a third of the thing, and I'm like, 'That? I only found out at the end when it's like, oh, you were supposed to do that,' and I'm like, 'But I missed a very subtle clue.' The game didn't help me. And like I bypassed a bunch of it because I was able to kind of brute force a problem I wasn't supposed to solve because it was the obvious puzzle, and I just assumed it was what I needed. So I asked for a hint, and it gave me okay. All that aside, Escape Academy isn't really like that. It does have a good hint system. It's reasonably easy to play through if you're really expert at it—you're probably laughing like, 'Oh, this is this is really easy,' but it is fun. It has a cooperative mode, which is really cool. So for me, oh, interesting. Kids and they got really into like getting out the piece of paper to solve some of the problems or write down the notes or figure out—it doesn't require a ton of that, but there is a little bit of it, and some wordplay and stuff. But just overall, I loved it. It's probably the only game where I've paid for the DLCs and played all the DLCs. Like wow, I don't think any other game—I always hate DLCs. Yes, and can't express how much I hate them, but it's probably the only game I've ever paid for DLC. 0:50:51.120 --> 0:50:56.903 Like played it and feel like I've gotten my money's worth and absolutely would have done it again. 0:50:57.544 --> 0:51:08.294 Oh, that is amazing. I'm gonna have to try this out. You've had some amazing game recommendations. Like I never finished Dave the Diver, but I really enjoyed my time with Dave, and I haven't finished. 0:51:08.294 --> 0:51:08.850 It either? Yeah. 0:51:09.407 --> 0:51:16.815 But you know the first half of the game is amazing. Like I feel—I feel like I'm super satisfied. I don't think I really need to see the ending. 0:51:17.153 --> 0:51:30.585 Yeah, kind of the same. And Escape Academy is available on like Steam, I think. You know it's on Switch, I think it's on Xbox as well. So most of the sort of gaming platforms, and it's not super expensive because it's been out for a. 0:51:31.429 --> 0:53:31.160 While very cool. All right, my tool of the show. And this might start to become more of a trend, but it's basically using AI, like using LLMs. So I'm putting a link to ChatGPT, but you can use really any of them. But it's multimodal, so for folks who don't know that, just means combining several modalities such as text, audio, visual, etc., multimodal LLMs to make calendar meetings. So for example, my kids do Taekwondo, and I'm always forgetting like what times because they have different times, they're different ages, and all that. I literally just took a picture of the Taekwondo schedule and I said, 'Create me a calendar.' There's this thing called an ICS file. I guess the C is for Calendar, I don't know, but it's a file that you can then when you open it, it creates a calendar meeting. So I said like, 'Create an ICS file that has like a recurring meeting for me at these times.' Like I have one kid who's this age, one kid who's this age, and so find from the picture find the times that I have to be at Taekwondo with them and create an ICS file done. Done. Like now in my calendar every whatever day there's there's meetings for that, and on Saturday and everything. You know, one of my kids does—he's like a vocalist, right? And so he does a bunch of like choir shows. So I took a picture of that schedule. I'm like, 'Just put on my calendar all the choir shows from this grade done.' And that has been amazing, like just taking pictures of things. I took a picture of my trees in the backyard and I was like, 'Are these trees healthy? You know what type of tree are they? What should I do to make them healthier?' And it gave me really solid advice. Um the whole thing is amazing, but especially the calendar thing. I talked to a few friends about this, and they had never thought to. 0:53:31.160 --> 0:53:35.379 Do that, and so I figured it'd be a good thing to talk about on the show because it helps so much. 0:53:35.379 --> 0:53:53.601 Yeah. I um I think this is one of those cases. I was tempted to bring up an example where I tried doing this and it didn't work very well, but have you done it recently? Yes. Oh, okay. That's okay. It was a bit more complicated than what you were saying, but I think that. 0:53:55.289 --> 0:54:46.032 I won't take that bait. And when instead I'll say is what do you have to lose? Like if these ICS files suck or like they just delete it. Like yeah, like the bar is really low. Like if it screws up the fallback is what you wasted? What maybe three or four minutes trying to prompt it to give you what you want and then like it didn't work. Okay, just go back to doing it by hand anyways. Yeah, so I feel like I have been trying to encourage people I work with that whatever the bar isn't, and I think you alluded to this earlier, the Bard AI isn't the magical everyone's, oh, I gotta be 10x better, 100x is gonna replace all the engine. No, like can you be 10 better? Can you find a way or a place or a set of things it can do to make you 10 better, five better, one better? Like can you enable you to do something when you wouldn't otherwise do it, right? Like I think and to your point there a 0:54:47.837 --> 0:55:10.260 lot of the barriers is trying it. Like just try it. Like who cares? And it's not like that stuff's private. Like it's a Taekwondo schedule; it's probably on their website. Like yeah, there's no privacy concern there either. So like in my opinion, either I guess maybe it knows where you are so it can send Uber Eats food to you? Uber Eats food. No, I just like advertising, hey, we know you're at Taekwondo. Would you like french fries? 0:55:10.260 --> 0:56:30.387 Oh man, it sounds delicious. This is getting better and better. I think that you know this is another thing I've done kind of also off-topic but related is I actually put in a loop—I said find Python files that don't have a corresponding test_ file and create a test for them, and then verify like that the test passed. And I actually ran that in a loop so like it just kept doing that, and I just let it go for hours. And when I was done, it created like 80,000 lines of tests, actually probably more than that. And it's just like that's amazing. Like these are all now I'm gonna have to go through them today, and a lot of them you know might be crap, but some of them aren't, right? And like you said it's—these are things like I wouldn't have done because I just don't have time, but if the AI goes and does it, I can check the work, and that's easier than doing all the work. And then eventually we'll get this evaluation problem fixed; I won't have to do that either. So are you saying it helped you asynchronously write the tests? Yeah, that's right. Yeah, I had a global lock on my time. It's called family, and while I was locked, it went off. 0:56:31.315 --> 0:56:38.352 So what would you say is the what and why of asynchronous computing? No, I'm just kidding. We're transitioning into 0:56:40.005 --> 0:58:39.801 our main topic. All right. So here's the reason why I decided I reached out to Patrick to cover this topic because I've been asking this in interviews. Um we were looking for one more person to add to the team. I think we found them, but I was asking this kind of question like what's asynchronous? What's multi-threading? And very few people knew the answer, which really surprised me. So so I thought, hey, this is a good topic for us to cover. I like to think of it, and this is a term I kind of made up, but I think it it kind of sums up pretty well: asynchronous programming is multi-threading between the lines. So in other words, you know, you're writing lines of code, and one of only one of your lines can really be executed at a time, but in between the lines there might be areas where you can do multi-threading. So for example, you might have a line that says download this image off the internet, and so the system can only read and understand that line sequentially; it can't do other things while it's doing that. But then the part that actually does the reading over the internet and waits for those packets to come from across the world—that actually could be done kind of off to the side. And so you're doing multi-threading in between the lines. You don't have to worry about semaphores and mutexes and all that stuff because of this constraint, and so that makes your life easier. And so you get kind of the benefit because really like when you want to—it depends on the problem you're trying to solve—but for web applications, for example, what you really want is to not be waiting on the internet. And if you're not waiting on the internet, then you're 0:58:40.358 --> 0:59:09.417 pretty close to optimal for your web service because usually you're waiting on the internet to get your request, you know, complete; waiting on the internet to send that request to an intranet? Maybe to send that request to some database, you know, waiting for that database to come back. And so if you eliminate all that waiting, you don't also need to do a lot of multi-processing. You don't necessarily need to do a lot of instructions at the same 0:59:12.336 --> 0:59:16.920 time. Um so that's—that's kind of the big motivation for async, and so 0:59:20.132 --> 1:01:08.504 you need a couple of core building blocks to do asynchronous programming. So you need thread-local memory. So in that example I gave where you know you're waiting to get some data, you know, over the internet, that data needs a place to go that no other routines can access that data. So for example, one way it's done in Go is through something called goroutines. And so a goroutine is not like a new process on your OS; it's not like a whole new process, but it just kind of keeps track of the state of execution in this code, and it also has some thread-local or routine-local memory. So if you say something like, you know, I want to download this packet of data from the internet, and then I want to store it in this variable, you know, it has a place to go that it knows is kind of protected. And then while it's doing that protected operation, reading from the internet to some protected thread-local memory, it can kind of say other threads can do other things. Other threads can be reading other things from the internet or they could be doing arithmetic or whatever because I know that none of those other threads can affect this—this operation that I'm doing right here. And so because I know there's no side effects, I can now let those other things go, and so you could have a hundred routines all downloading things from the internet at the same time even if your program is single-threaded. 1:01:11.288 --> 1:01:39.554 So you'll hear about like blocking versus non-blocking operations. And that basically means a blocking operation is one where there it's not kind of releasing that lock. So if you just say imagine you have two kinds of routines and both of them are just running a for loop and they're just adding two really large arrays airwise, right? Well, those 1:01:41.595 --> 1:03:31.671 are blocking operations. So it's going to happen is the first routine is going to run, and it's going to do that entire for loop, and that second routine is just going to have to wait. And then when the first routine is completely done, then it'll finish, and the second routine will do its for loop. And so you haven't achieved really any parallelism, right? But now take that same scenario but instead of just adding things in a for loop, you're going to have a for loop that's downloading from the internet. Well, that operation—that download from the internet operation has been built to be non-blocking. And so in the case of let's say TypeScript where it's single-threaded, you know, you have a way of telling the TypeScript or the JavaScript VM like, hey, I'm about to do something now that is isolated, that's non-blocking. And so other routines can go and do work while this routine is just frozen on this operation. And then once you've downloaded what you needed from the internet, then you go back to the virtual machine and say, hey, I'm ready. You know, I'm ready to block and take this—take some time. And when it gives you that time back, then you can go and copy that information into that variable, and all of that. In the case of Python, Python actually has coroutines and threads. And so what that means is you can actually have multi-threading in Python where you have several different threads accessing memory at the same time, but 1:03:34.135 --> 1:05:29.520 there is a Global Interpreter Lock. So here's—here's a good way of thinking about it. In the case of let's say TypeScript, if you're inside a TypeScript function, you kind of have to wait until there's a non-blocking call. So the example I gave with the for loop, you're just stuck, right? In the case of Python, it would be a little different. If you're using threads, what would happen is the first thread would do part of the for loop and then it would release, and then the second thread will do part of the for loop and then release, and they'll just kind of checkerboard this. And so it's not the first one goes till completion. There's sort of this striping effect. Now it might take the same amount of time in the end as the TypeScript version, but you know they're both making progress at the same time versus TypeScript is single-threaded. And the way Python achieves this is through a mutex. So this is called the Global Interpreter Lock or GIL, and it basically says you can have many threads that are all sharing your VM, but only one of them can execute at a time. So here's if you wanted to accomplish the same thing with TypeScript, imagine the for loop example I gave with TypeScript. So you have a for loop goes from one to ten million, adds these arrays together, but after every iteration of the for loop, you put a yield. So yield is a special keyword in asynchronous programming. Yield basically says, 'I just want to give up my time slice. Like I'm not done yet, but I feel like the right thing to do is to give up my spot and let another thread do some work.' And so as a programmer, you might have various reasons for doing this, right? So imagine if your 1:05:29.520 --> 1:06:02.500 for loop said, you know, for one to ten million, take these two arrays and add the element together and then yield. So now you're going to get that striping, right? So now the first process will yield, the second one will be able to do one iteration of the for loop, and then it will yield back to the first one. And so you know the Python multi-threading is the equivalent of like having a yield after every operation. And so you know the GIL kind of enforces all of that. 1:06:05.487 --> 1:06:08.305 What do you think, Patrick? Any holes in? 1:06:09.030 --> 1:08:04.400 That we could cover. I think for me, my introduction is probably a bit different, but I think it parallels. So I don't think it's contrary to what you're saying, but for me, it was—and you kind of bump into this if you ever do like Arduino programming, but like embedded programming. And there sometimes the mapping is actually really weird from something like Python to something like running C++ in an embedded microcontroller because it's just sort of like different paradigms. Like networking is well now it's more common when I was doing it, it was very rare that you would know kind of do networking-related stuff in these contexts. And there we would talk about like interrupts. And so the idea is if you imagine like a microcontroller wants to talk to a sensor, that sensor—it communicates over something called I²C. All it is is like a set of wires and voltages that move up or down in a certain pattern to send data from like your piece of silicon to like another piece of silicon glued to your like thing with wires or on a printed circuit board or something else. And so you want to measure, let's say temperature of the room, and you want to record this. And sort of the way that it used to kind of always be done was something called bit banging. And so bit banging would be, 'Hey, I'm a microcontroller. I need to read this, so I'm going to set up an I²C library in code that says move the voltage on this wire from low to high wait this amount of time where a write is go into a loop that is like a certain number of instructions long time to be like the right amount of wall clock time based on the like rate of the processor. So it's just sitting there doing like no-op instructions, but it's running the instructions. It's 1:08:04.400 --> 1:10:03.440 using all of the power of the full CPU which has like all sorts of peripherals on it. So it's sort of consuming the most. And you sort of go high for a certain amount of time, then you go low, and you're moving along the sort of data to be sent out to ask for a temperature reading. Then you have to wait, and then it starts coming back in. But you can imagine like thousands, tens of thousands of like no-op cycles waiting for like the voltage to move up and down due to capacitance and the rise rate and like the data transmission. So the CPU is really bored, but it's sitting there doing nothing right. And so this is the equivalent of like polling or single-threaded, and you're doing all of the work in one context. Then people realize, 'Well, we have more sophisticated silicon. Why don't we make like a sort of finite state machine, an accelerator? Like another set of transistors where you—you sort of said it's thread local memory.' The equivalent here is we have to call it a register. So you have a set of registers, and you could like write into them, 'Hey, I want to send out this data stream,' and then you would say go. And what you could do then is put the CPU into a low power state, like go to sleep. And then you're going to save power, right? And then or you could go to other processing if you had it, but saving power is also really, really important in a lot of these contexts. What's going to happen is the little bit of circuitry is going to like send the thing out. It's going to wait for it to come back, and then it's going to interrupt, and you're going to have a little function that is mapped to say when this thing is done, it's going to call the instruction that you've loaded into this register. So the instruction there needs to be jumped to a function that you wrote, and then that function when it runs is going to push the current thread, the current running CPU thing into the stack. It's going to allow you to run some piece of code, and that code could be copy the data out of the registers into 1:10:03.440 --> 1:12:02.000 a set spot in memory or into a queue or just handle like write it into—write it into the, you know, flash memory. But that could take a long time, so maybe you don't want to do it. And you can imagine as you're handling 10, 15, 20 sort of like peripherals all coming in at different rates doing all of that and sort of making sure that none of it gets dropped becomes harder and harder if you don't sort of have some mechanism that allows the sort of distributed handling to kind of happen. And so the asynchronous context maps pretty cleanly to what you're saying. You're sending some data to the network and saying, 'Hey, I need you to transmit this.' It's going all the way across the world. It's coming all the way back. And you can sit there and wait for it, and that's called blocking. But under the hood, the blocking is still just basically sitting there, 'Are you done? Are you done? Are you done? Are you done? Are you done?' It's not doing any work. It's not bit banging anymore. It's just waiting on the other little processor to do all the work. And so I think as you kind of pointed out, there's like this clear mapping about blocking versus non-blocking. I think the rest of the conversation becomes in a bit what you're—you're kind of getting into is like there are kind of two ways to handle it. One is sort of inline in the code. 'I want to ask for something to happen, and while it's happening that set of consciousness that execution is just going to kind of pause there, and maybe other stuff can run.' And then when it's done, I'm going to pick back up. And that is one method. The other method, if you think about like interrupt as a form of like a callback, you could just have like almost a DAG, which is 'I'm going to ask for something to occur. I'm going to transfer control to that thing. When that thing's done, it's going to call the next stage.' Oh sorry, Directed Acyclic Graph. So it's just a pipeline. And so you call to the send networking, and when the networking is done, it's going to call some function, but that function is completely different part of the code, but it's responsible and it has enough context to say 1:12:02.720 --> 1:13:40.538 and maybe even pass a little bit of context. 'Hey, I'm getting a response, and when I get a response, this is what I'm going to do.' So if you think like a graphical user interface, and I say type in my text to my friend, and I click send, and then I don't need like control to return really when my friend sends a response back. That code just needs to know to write it to the screen, like write it into the next line of the message conversation. And so you can kind of have this feed forward, and so you don't really return back to the same point of control. And there are these kind of—and they can be mixed—but these kind of two paradigms where you—well, I guess there's a third one, but like there's this sort of block and wait, and then control returns there. And that can be blocking or asynchronous for allowing for cooperation. You can have this like feed forward sort of DAG approach, and then the final method is like kind of spoiler, we'll get to in a minute, but like the sort of future promise where you're getting some state back that somebody else is filling it, and you're going to go check on it later. You're going to go ask later, 'Hey, was that done yet?' Or 'Wait for it to be done,' or something else. I think that's like a little bit of like a different handling. So there are these kind of different methods, but under the hood they're all really the same thing. They're just how the programmer wants to relate to that work and how—how you want to set up and what other kinds of things in your system. And if you have nothing else to do, letting it sleep can be a really good answer in some contexts, but in other contexts, you have other background processing or other things that need to be handled. And so depending on the paradigm of your program, you'll kind of choose a different asynchronous behavior. Yeah, yeah. 1:13:41.500 --> 1:15:34.832 That makes sense. I mean, I'm reminded of—and I should have made this a news article—but if you read like the history of NGINX. So NGINX, it's spelled N-G-I-N-X, but basically it's the super fast, you know, web handler, web service handler, and it does routing, proxying, etc. And when NGINX was invented or, you know, was initially being developed, the best thing that that was out there for doing web stuff was something kind of like Tomcat. Tomcat is a Java thing. There was other—it's written in other languages—but the way these things would work is, you know, when you started Tomcat, it would spin up like let's say eight processes, like OS-level processes on your computer. And it's like, 'Okay, here's your eight processes.' And then the main process, when it sees there's a new web request, it would just send it to one of those eight or all eight of them would listen. I'm not sure at that point what exactly happens there, but basically one of those eight, you know, receives a web request, handles it, which might include like going to the database and doing other stuff—maybe it's over I²C, it's like talking to a temperature sensor or something—and then responds with the response. But because you had eight processes, you could only do eight things at a time. And it could be that all eight of those are waiting on the database, and you really could add a ninth one or a tenth one without affecting that computer's load because they're all idle, right? And so NGINX was pretty revolutionary in that they basically said, 'Okay, we're going to create the same eight processes, but we're going to use asynchronous programming.' And so 1:15:36.942 --> 1:16:45.944 Oh. And so you know if—if one of these, you know, eight processes is waiting on a database, then that same process can go and like fetch another web request while it's waiting on the database. And so you could actually have, you know, eight processes handling a thousand web requests at the same time instead of eight at the same time. And this like blew everybody's mind. Like I remember we were using Tomcat on something—this is like 2003-ish—and yeah, and just constantly running into issues with quality of service where you know we'd get—I was working at an online school, and so registration day there'd be a ton of people hitting the site, and they would inevitably bring it down. And so you had this machine that was mostly idle but then also just blocking everyone's requests. So NGINX was amazing. The thing about it is if you were to do this yourself like what Patrick was saying, you know, if you were to, you know 1:16:48.323 --> 1:17:45.840 to try to implement this asynchronous thing yourself, what you would have to do is—and I'm going to use the higher level terms, this probably doesn't work in Arduino, but you could—you could help translate. You'd basically have to say, 'Okay, what things am I doing right now? Okay, I'm doing these eight things.' Okay, can I sort of peek at each of these eight things to see if one of them is ready? Oh, the sixth one is ready. Okay, you know execute that function. Oh, now like that function got a little bit further, but now it's blocked again. Okay, put it back on the list.' And so you could imagine like what a nightmare that would be to have to code up, you know, for every application. So um so in this case, like that whole nightmare of just peeking at a zillion different things all the time and finding the one that's available and doing it—like someone else has implemented that nightmare for you so that you can just kind of work at a higher level. 1:17:45.840 --> 1:19:44.320 Okay, there's a bunch there that was really good. So I think to be clear, this is all a bit how do you know like recursive fractal? I so the operating system is doing kind of what you're saying. So it has a scheduler. Modern computers normally like a preemptive like scheduler, which means exactly that you have some thread running that gets stuck or is not or is lapsed its time, and the operating system will like suspend it and run a different thread in order to like timeshare. And and sort of hopefully some of them are just doing like busy loops pulling for something, and you're getting a win. But there is like some—like you're it's not magic. None of this stuff is magic. So everyone who always it's kind of one of those two sides of the coin. In some cases it sure seems like magic, but in other cases, you're like, 'I don't get it.' It like there's still only one CPU. And and yeah, you're right, there is really only one CPU. So in like your database example, let's say you have sort of like the database is local on disk, and you really are kind of bottled—like let's say you're already maxing out the like hard drive RAM transmit speed from your CPU to RAM. Whether you have one, eight, nine, ten, a hundred like database jobs queued up, you're not gaining any more throughput by like knowing that you have a hundred jobs or knowing that you have eight jobs. Like there's nothing more to do. Hmm-hmm. The trick comes in that that isn't normally the case. It's not that every job coming in needs the database. Some jobs, you already have a cached answer. It just wants to say what is the list? In your example of like, you know, registration, it's like what is the list of all classes? Well, you could have memorized, you could have cached that like, you know, the list from five seconds ago of all possible classes. You don't know their current state, but you know all the things, and so some requests can be quickly turned back around so that the server feels more responsive. And so while you're waiting on the 1:19:44.320 --> 1:21:04.907 hard drive RAM to transmit database responses, you're sort of getting these things back, and the question becomes a little bit on like the scale: are you running? Are you the only thing running in the operating system, or are other things running? Do you have other threads running? Do you have—like there is still a sort of like fundamental limit to how much stuff can happen. And going to an asynchronous solution isn't a cure-all, but it is one of these cases as you're pointing out. Often we can know more about the problem than the generic operating system solution would give you. And so letting eight OS threads handle something isn't going to be optimal if we know that the work is very heterogeneous because if we are having one in ten come in that needs a database, but it takes a long time to get a database response, then as soon as you have the few queued up, everybody else—even that doesn't need the database—is stuck waiting. If you sort of have this sort of asynchronous thing that you're mentioning, you can just burn through the entire queue of everybody who doesn't need the database. People who need the database may still get a failure, right? Like, you know, they may still have a timeout, but overall your metrics are better. The responsiveness feels better, and you can kind of guarantee you're maximizing the proper throughput. Yeah, yeah. 1:21:05.667 --> 1:21:13.547 Totally. I mean, this is also like—there there's a dynamic at play here which is like throughput versus latency. So imagine, oh yeah. 1:21:13.547 --> 1:21:14.492 Okay, good. Yeah. 1:21:14.492 --> 1:23:07.774 Like imagine like the connection. Imagine like the database is like a giant sewer pipe, but the water can only go, you know, one meter a second. And so, you know, you could put like a ton of water through, but it can only go through at a certain rate. And so that's where yeah, having like a zillion connections to the database actually is a little better because you're just hiding all of that latency. But but you're right, with asynchronous programming, you do run into a lot of problems around, um, you know, now you can do a thousand things, but that also means that potentially there's a thousand things that are in a bad state. Or or if there's a bottleneck further down in the chain, then now all those thousand things are sort of fighting for resources. So even in asynchronous solutions, like if you look at FastAPI for example, even though it's asynchronous and you could handle a thousand requests on one thread if you wanted to, even there you would specify a limit. So you typically would say something like, 'If I have—I'm just going to make up a number—if I have 32 requests simultaneously on this machine and a 33rd comes in, I'm just going to reject it and return an HTTP 503.' And then my client will call again and hopefully get a different machine because even though the machine might not even be very busy, you just don't want to be in a weird situation where there's a thousand things that aren't getting done because that's usually a sign that something else is wrong. So so asynchronous, to your point, it's not a free lunch. Like you do have to, you know, now that you've been given this power, you do have to use it responsibly. Yeah, you bring 1:23:08.787 --> 1:25:04.920 Up a good thing. There's like an interplay here. We didn't have it in the notes, but like queue management and there's like stacked up queues everywhere. Yeah, as you kind of mentioned, you're going to have like a load balancer, and the load balancer will have like a little bit of a queue. It's sending stuff if you just consume a thousand things even though you can't handle them. Some other server may have finished everything it's doing, and you're just like holding those thousand waiting for expiration basically. And in reality, as you said, if you returned an error code after a certain amount, it'll just run back to the load balancer and hopefully get assigned to a machine that like isn't having an issue or isn't overloaded or whatever, and those have their own queues. So there's like this very interesting whole system dynamic. The other thing about asynchronous program, although it is like important to know about and be aware of, is you do have to just like multi-threading. You do have to be concerned about like race conditions, data safety issues depending on what all is happening right? So you can imagine I send a network request, and then I have an update from the user. I want to do something different. I've changed my state. I send the new networking request that I kind of want to supersede, but the old one comes back. And I think it's an answer to, you know what I mean? Like you have to manage that. Wait a minute. Like I have to be really careful I'm doing more than one thing at a time. And so you can end up with all of the traditional things you would hear about related to sort of thread safety—being careful making sure that you know you're aware of whether the data structures like queues and other things you're using are multi-thread safe, even if you're only doing async stuff, because you can end up in these weird situations where out of sequence stuff is occurring or overwrites. Or you just be aware that it does add a lot more complexity to the system, but there is a lot of gains to be made in many situations. 1:25:05.933 --> 1:25:29.085 Yeah, totally. And this is really what separates kind of like masters of their craft from apprentices. You know, it's like all of the lessons learned around—around oh, you know, we got burned this way or that way, or we, um, you know, the database kind of like fell over because we weren't monitoring it. And what it really comes down to 1:25:30.655 --> 1:26:41.378 is just setting up a ton of monitoring and constantly tweaking the knobs. You know, it's if if um if the machine is like mostly idle, then maybe it can handle more processes, but um you know if processes are getting backed up, why are they getting backed up? And so there isn't uh like a there isn't a system you're going to build that has concurrency that's going to work perfectly for the first time and forever. You know, as as the load profile changes, um you'll have to adapt to it, and it becomes kind of a living system. Um this is this is pretty unavoidable. So async, you know, helps with, you know, all of the mistakes that come from multi-process. You know, async, you know, you can eliminate a lot of those as as candidates. Like if we were doing our if we were basically building async ourselves, we'd have to wonder, 'Oh, did I get the process pool correctly? Am I releasing the semaphore at the right time?' And so because of async, you don't have to worry about that, but the consequences you still have to deal with. You bring up 1:26:43.454 --> 1:27:55.037 a really good point actually. I think if you're going to go to this approach, so there are some like UI frameworks I feel are much more commonly written in an async at the framework level because you want to make sure that the user interface stays responsive. So you shouldn't ever do anything blocking, and so they're actually really set up from the start to be asynchronous. But people who write that—well, I should say a lot of people I work don't write UIs; they're not actually even familiar with that. As you pointed out, if you ask them about it, they wouldn't really be aware of like the nuances of these trade-offs and the decisions and the complexities and measuring and all of the parameter—like having reasonable parameters and monitoring whether or not like they're set well—would just be stuff not present. So it is really weird. Like on the one hand, it's common enough that you would sort of say like it's in every UI framework to have like asynchronous stuff. What do you mean? Like people don't know what it is, but it's like this very weird division of practitioners where sort of like I think there are whole classes of people who go a very long career without really getting into 1:27:56.050 --> 1:29:52.960 any of it. Yeah, yeah, totally. Which is why you need to subscribe to Programming Throwdown. It's a perfect segue to—to our outro. I mean, we we, you know, um I feel like we gave a pretty good background. Um I think you know the sort of maybe the call to action here is really learn async programming. I mean, if you um if you're doing UI stuff, you pretty much have to—I think it's at the point now where I think Android and iOS, if you try to do a network request on the main thread, it'll just like give you an error, like a runtime error. Um it just literally will not let you use the main thread to do non-blocking stuff. Um so so it's a non-starter. But but even if you're doing anything with distributed—anything with queue processing—you'll have to you'll encounter this in some way, shape, or form. So so uh if this is all new, definitely dive in. This is an area that's worth learning. Um I think with with um I think with AI and agents and and all of these things, it's going to become even more important. You know, an agent might send a text message and wait to get a response from somebody, and it can't just block the machine while it's doing that, right? You want you want to run like 10,000 agents on one machine. So so super important area. Um uh I think that uh um it's definitely something that's worth researching. And uh um if you have any questions, you can always go on Discord. The Discord channel is is uh getting more engagement. I always try to reply to uh to stuff on there. Um so check us out on Discord. Shoot us an email, and uh it's a great cover in this. Yep. Thank you everyone. All right. Catch y'all later. 1:30:09.092 --> 1:30:11.100 Music by Eric Barndollar. 1:30:12.704 --> 1:30:31.040 Programming Throwdown is distributed under a Creative Commons Attribution-ShareAlike 2.0 license. You're free to share, copy, distribute, transmit the work to remix, adapt the work, but you must provide attribution to Patrick and I, and ShareAlike, and kind