WEBVTT 0:00:15.994 --> 0:00:23.369 Programming Throwdown: Episode 171 - Compilers and Interpreters. Take it away, Jason! Hey! 0:00:24.465 --> 0:00:49.137 Everybody, this is a super exciting. We got this request from a listener—maybe later on the show I'll find out who it was, but you know they said, 'How about you folks do Compilers and Interpreters?' I can't believe we haven't done this episode; it seems so pivotal to everything we do in software engineering. But 0:00:50.790 --> 0:01:57.801 But yeah, here we are to lead the way. I have to confess, I don't know as much about Compilers and Interpreters as Patrick, so Patrick's really going to carry the water here, but I will add color commentary as appropriate. And before we get into that, I wanted to talk about monitor setups for a little bit. So I was working next to somebody who had three monitors like side by side, and they had three different things going on, you know? Like they had one was for coding, the other one was for email, and I thought this was really compelling. I thought, 'Oh, this is a great idea.' And I just can't get myself to do it. Basically, I just have to have just one thing in front of me, and just keep Alt-Tabbing to go to different things. I'm not—I haven't figured out how to, you know, drag from one desk screen to the other and do all of that. What is your setup? Do you have, you know, like an array of monitors or what? What do you do? 0:01:58.662 --> 0:03:27.239 Yeah, eight. No, seriously, I do all my programming in VR, so I just have as many as I actually do look for it. I think that'll be cool one day. One day—that day is not today. I just have one big one, but I also have an iPad off to the side that I use for my conferencing, for video conferencing. That works well for me. So I guess it's technically two, but one big one for sort of similar reasons to what you're saying. Like that normally if you have two monitors, you just get them extra wide or on top of each other, which is too much versus one big monitor. I feel you don't end up with that awkward split in the middle or have to shift one far to the side, right? I will say for people who do like simulation race games or flight games having three for a gaming setup would be really cool because you're one thing over three. And the people who do of course the, you know, stock trading where they're actually just it's mostly focused on one task, right? Like they're all of it is related. There's just lots of, you know, dials and metrics. It's akin to I think like a cockpit in an airplane where there's many many indicators. Some are lower priority, but they're kind of very related to a singular task. To me, it seems a little bit different. Like when you're doing your programming, email is—maybe that's not fair, I don't know. I've never been those other things, but it feels like a different task. Slack or email, they feel different to the task of coding. And so yeah, I'm with you. Like I don't feel compelled to have them up on 0:03:27.880 --> 0:03:57.540 The side. Yeah, you know, I used to do—I used to have a my laptop, so like my laptop right now is just mirroring the screen of the main display. I used to have it kind of like what you're saying where it was sort of a secondary screen, but I found myself not paying attention in meetings even when I should be, and I just—I kind of couldn't do that. I guess well, I mean isn't this like a trope that people can't actually multitask? 0:03:59.065 --> 0:04:02.288 We just sequentially singular task. Yeah, yeah, I mean. 0:04:02.575 --> 0:04:12.025 I just—but yeah, I mean, I do envy the people who can have like three different screens on and not be distracted. 0:04:13.577 --> 0:04:50.859 will say i i do find i i it's not all bad i think if you occasionally need like a paper or document or you're implementing an algorithm and you want to consult something or the few times i've had the kind of i'm generating some kind of output or whatever that's visual um it's very cool to have a extra real estate where when i update something and click you know test i can sort of still see the code but see the results and you know up inside you know screen real estate can be very very useful so i'll never sort of say something bad against someone who who wants to have more screens but the variety is yeah it's it is very interesting 0:04:52.255 --> 0:05:12.353 yeah i wonder um you know now that i'm working from home it's less of an issue but when i was going into the office i was running into problems where you know my home monitor and my office monitor aren't the same and so you know it wouldn't quite extend the desktop the right way and things would get all messed up 0:05:13.517 --> 0:05:31.034 that's the thing i've been picky about is i have my monitor pretty high i'm i'm fairly tall so um i want you know my monitor nice and high so i either put little plastic risers or put it on a monitor arm and lift it up so i have it higher than i think what i see a lot of other people do just because it feels unnatural to me all right we 0:05:31.034 --> 0:05:41.850 have to have a we have to have a bake-off here my monitor is on a i'm trying to eyeball it maybe like an eight inch riser what about you oh i'm 0:05:41.952 --> 0:05:45.816 trying to look from the bottom of my screen i'm like at a like a 0:05:46.727 --> 0:05:57.814 Foot like oh wow, okay. That's even bigger. I might be—yeah, a little bit more than eight inches, but yeah, that is pretty tall. So like when you look at your monitor at eye level, are you looking at the center of it? 0:05:58.844 --> 0:06:06.454 I wish it's not quite that high, the monitor. I'm going to go quite that high, so it's—uh yeah, I'm looking at like just above something like 60 up from the bottom. 0:06:06.809 --> 0:06:18.722 Yeah, okay. That makes sense. Yeah, I'm about maybe like 75 or so up from the bottom, but yeah, if I didn't have any riser, the monitor, like my eye level would be above the top of the monitor. It's just—yeah. 0:06:18.722 --> 0:06:21.827 But but I see people do that all the time. I don't. I can't. 0:06:23.144 --> 0:06:28.189 Yeah, it's like—um, I think your neck would get hurt. I think if you do, if you keep it like that, what? 0:06:28.442 --> 0:06:34.804 I want to try is one of the very wide curved monitors. I think that would be really cool. Yeah. 0:06:34.956 --> 0:07:08.757 You know, I've—I've tried it. I have one um at work, and it's pretty good. It's pretty good. I will say that the one I have, um, the refresh rate isn't as good. Um, it's not like you know a deal breaker or anything, but you do like if you have one monitor, it has twice the refresh rate of the other. It can kind of like—sure, it's like when you're at the store and all the differences between the TVs are so noticeable, but you take it home, totally lose your frame of reference, right? So— 0:07:08.757 --> 0:07:24.029 If you have like a four to three aspect ratio or whatever, like not a 1080p widescreen 16 to 9, but like if it's curved left and right, does it also curve up and down? If you get a nice tall one, like—or I don't—is like on a sphere. I've never looked at it. 0:07:24.029 --> 0:07:32.399 Up, I guess I should. The one I have, I'm looking at it right now, um, it's it's curved, but just you know, it's just curved like—like. 0:07:32.399 --> 0:07:33.867 A cylinder. Like a cylinder. 0:07:33.867 --> 0:07:42.060 Okay. Yeah, and but it's very wide and not very tall, so the aspect ratio is not—it's not HD aspect ratio. It's much wider. 0:07:42.473 --> 0:07:58.892 Oh, okay. Yeah, I think I would want one that—yeah, I think I just want like a very large, like probably 8K, but like you know big, but then I wouldn't actually run it. You know, I would everything would be scaled up just so it's a comfortable eyesight thing. I think 8K would be too small at like minimum resolution or whatever. Yeah, right. Um. 0:07:59.500 --> 0:08:24.036 But yeah, folks out there, let us know your monitor setup. I'm curious. I do feel like I could do better. Um and uh yeah, curious to know what people's setups are out there. Just chat us, email us, post in Discord. Um Discord is getting more popular, so people posting some funny jokes there. There is this video of this professor pranking his class, which I thought was hilarious. So thanks. 0:08:24.036 --> 0:08:26.466 Rang to their class? Okay. I had it on Discord, but— 0:08:27.006 --> 0:08:28.120 Pranking their class? Oh, pranking. 0:08:28.120 --> 0:08:30.340 I just said ranking. I was like that's—I was like, 0:08:34.448 --> 0:08:37.604 Yes, we all know that they do that, but I'm supposed— 0:08:37.772 --> 0:08:40.500 To actually do it: RateMyStudents.com new website. 0:08:40.500 --> 0:09:54.149 All right. Well, we'll jump over to the news of the show, starting off since last time I mentioned this and then Jason was excited about it as well. So, so I feel we'll talk about it here. Uh, the headlines everyone's probably seen them, but but I feel like it's pretty cool for something that is a memorable part of my childhood, which is being absolutely crushed and being terrible at the game of Tetris. Um, but but now uh there has been someone who has beaten Tetris. Actually, as of recording this show, I think two people have done it. Uh so the world record didn't last very long after lasting for years and years. Um but Blue Scuddy—I think the first name is Willis—beat Tetris first, not at the earliest possible time, which was accomplished now, but uh did beat it. And here actually the reason why I think it it's worth mentioning on the show aside from just being a cool thing and I linked the video there that is the first place I saw it. Um but it's interesting how they define 'beat' here. So when you know Tetris was originally built, they didn't actually have an end game in mind, which kind of unusual. It would just play faster and faster, and I guess the assumption would just be that it's so fast you couldn't possibly keep going, and so eventually you would die. 0:09:54.705 --> 0:10:20.507 A lot of games actually are like that. Like you know Donkey Kong. Actually, uh in Donkey Kong the amount of time you have to finish the level goes down every time you have completed a full revolution of all the levels, and basically you get to a point where there just literally isn't enough time to finish the first level, even if you're frame perfect. And that's just how it— 0:10:21.891 --> 0:12:21.340 Ends. So Tetris I guess gets faster and faster, and then at some point it basically kind of caps out like it just doesn't go any faster. Um and for a long time uh people would get to that area uh that that speed and then just sort of you know die pretty quickly if their their row count was low. They would be able to you know manage to maybe get a few more lines cleared and then die, but then recently people discovered a new way of pounding—tapping the uh uh you know controller faster and faster with this crazy two-handed technique. And um they've been able to go faster and faster and began to discover which I guess people had figured out via you know tool-assisted speedrunning, so they had kind of known this, but humans began to be able to get there, which is levels where you know incrementing the color palette left the predefined color palette and would start using actually machine instructions as the color selection, which would result in just really ill-formed colors that made it even more difficult. But then eventually a portion of memory would be accessed that would just cause basically uh you know a corruption of of the data, and so when that happens the game—the game just crashes and stops and just hangs. Um and so this is what uh Blue Scuddy managed to do was to to basically cause the game to try to execute code that wasn't code, uh and then hang the game. And so first person to do that? It's just it's crazy. It's very fascinating. It's also—and it's the same you know as Jason was mentioning sort of frame-perfect analysis for Super Mario Bros., is some any sort of speedrunning game that is from around that era. The amount of analysis and reverse engineering that goes into it, and there are even patched versions of the game so that like it doesn't get weird and you can keep playing or you know just other things where people really really understand the mechanisms that uh you know take place at this end game, and the amount of analysis I'm sure the programmers at the time had no idea that you know for 30 years, 40 years people would be uh sort of thinking about what that code that they wrote. 0:12:21.340 --> 0:12:24.640 In the back of a cubicle, you know somewhere, and turned out— 0:12:25.855 --> 0:13:42.130 Yeah, it's totally wild, right? I think uh yeah. Yeah, that whole thing where they do this crazy—I mean, you really have to see it almost in slow motion this way with two hands. So the idea is every frame, you know, in Tetris there's no inertia or momentum or anything like that with the with the pieces. So you know every frame if you could get like a press left and a stop pressing left in those two frames, then you could move it over one whole column. And so yeah, they do this weird thing where I guess they they hit the joystick from the bottom. Um so in other words, like if you imagine like just pressing with your thumb on a button—like there's only so fast your thumb can know press and release this button—but then I guess they're saying well part of that is just like the mechanics of your body. So what if like you had another thumb kind of on the bottom of the controller, and so one thumb is pressing and releasing, but then the other thumb is pressing the bottom into the first thumb? And like I guess by doing that you can get more presses than you could with just one one thumb. It's it's totally crazy. You have to see. 0:13:43.176 --> 0:14:10.277 It. I mean, I guess Tetris uh lends itself to that because you—I don't know. We'd have to learn more, but I guess you only have to move either left or right as long as if you don't overshoot, and you only have to rotate the piece three times in a direction, and then you would you know just be back. So you can only rotate 90 degrees. So there's like a finite number of presses if you knew exactly what you were doing—your path planning. You could be very minimal inputs. 0:14:11.560 --> 0:16:07.760 Yep, yep, totally. All right, on to Palworld, accused of being an AI product. Um, so yeah, Palworld is wild. Patrick and I were talking about it before the show. It really like is a great area, a great theme for a game. Basically, it's adult-themed Pokémon, adult-themed. That means something totally different—Pokémon with guns. It's Pokémon with guns, and they're just—they're I think they're also just really aggressive, you know. You could actually catch people in the Pokéballs and sell them like. So it's basically a crazy, much more like Mad Max. It's Mad Max meets Pokémon. It's a great idea of a theme. This article is really interesting. Patrick actually found this, and it's claiming that a lot of the assets are AI-generated. But they have no hard evidence. So I guess there's really two things that I took away from this. One is that the generative AI is really here to stay; it's not going anywhere. You know, I mean, the days of downloading like 30 gigabytes of MP3s when you buy a game—I feel like those days are very short-lived. I think even if you did hire a voice actor, you almost certainly want to do some type of capture of their voice and, if nothing else, just to reduce the storage costs and generate their voice dynamically. So that's definitely here. And then the other thing is, it's amazing that they don't know because I've seen a lot of papers and other. 0:16:07.760 --> 0:16:19.101 Um, you know reports where you can see how something is generated by an AI. But I guess they have a point that it's really hard to prove. 0:16:20.704 --> 0:17:06.705 It. Um, you know, I think the examples I saw were interesting. Like for example, if you see a—and this isn't a Palworld thing—but if you see a picture of a face and the earrings don't match, you know, if it's a woman's face or man's face, anyone's face, and the earrings they're wearing don't match, that's a sign that it's generative AI because it's hard for the AI to get that symmetry right. But they haven't really found anything like this here, and it might be just that the person is starting with an AI model and then just as inspiration but then putting a human touch on it, and it makes it very hard to track. 0:17:07.937 --> 0:17:19.716 Okay, a couple things. If you're going to commit a crime, put on mismatched earrings and like do all the tips that are generative so if someone captures you on camera, you'd be like, 'That's clearly not me because like.' 0:17:19.716 --> 0:17:20.813 That's clearly an AI. 0:17:22.028 --> 0:19:01.800 So I sense a trend where people do this. Second—a hat tip here to Ace Rola. Israel does a YouTube video on shaders actually. Anyways, and on Twitter, or X is where I sort of first saw this controversy rising up. But if you've never checked out him, check it out, Ace Rola. And then yeah, I think Jason's right. Is like, 'Wait, first of all, does it matter? Like if it's generative AI or not?' And the answer is maybe. And it's sort of gets into one of those gray areas. If it's fun, it's fun. But at the same time, there are people trying to put work into the craft, right? It goes back to that debate which I don't think there's an singular answer to. But you know the difference between IKEA furniture and, you know, a real wood piece of furniture. They're both furniture, but you could kind of talk at length about various pros or cons of each. And so I think in the end, as I don't know—even they have to be forthright or honest about what they did. They just probably shouldn't go around claiming that it's high art or that there's tons of people behind it, right? If you tried to say, 'Hey, you should support this game because this took a lot of work and it didn't,' then there's like a sort of ethics problem. But as far as I know, as long as—I think right now they're not saying anything, they're just not talking. It's been a kind of overnight sensation, then I personally yeah, it's kind of interesting, but I don't know that I have a specific problem with it. 0:19:01.800 --> 0:19:44.487 Yeah, I think as a coder, it's hard to really relate. Like all of our code that we've done on open source, you know, like in my case, like Eternal Terminal and MAMEHub, and all these things, they're all public repos. And so you know AI is scanning those and using them to build GitHub Copilot and all these things, right? So in a way, they're kind of like copying it from me, but it's okay because I copied it from Stack Overflow. But artists don't feel the same way. 0:19:45.111 --> 0:20:52.307 That's not, but that's not. But if I'm not an artist, I can't speak authoritatively. But it's very common when people are learning to take inspiration to practice copying and to understand what it takes to make the paintings. And it's not good to copy too much, but to be derivative. Or you can even see sort of the lineage of how artists build on other artists' concepts and extend them and sort of push them forward. And so there's this very fine line between ripping off, Pokémon characters, although if you look other places like Digimon, there's a lot of Digimon monsters that look very similar, and the same kind of brouhaha didn't get that well. Maybe it did. I was probably too young to remember, but you know it—I don't know. There's a lot of music. It's the same way, right? Everyone's sort of taking riffs and ideas from each other. And it's been court cases recently where it's—I don't know the answer. It sure sounds really similar, but then again, it's a nice sounding thing if you just sit there and play music until you find something nice, you may play something that someone else has already figured out before. So yeah. 0:20:54.147 --> 0:21:14.042 I mean this is maybe trying to think if there's other examples before I say this, but I don't really think so. I mean let's say this is all AI-generated. Somebody basically threw all the Pokémon characters of all time into some AI model and now they're able to spit out more Pokémon characters. 0:21:17.249 --> 0:21:52.860 Know this is really the first time where we've seen like some huge commercial success come out of some generative AI thing. Like people are so worried about fake things, like you're trying to impersonate a politician or something like that. But this is not really that. I mean, in terms of the incentive is not to impersonate Pokémon; the incentive is actually the opposite. It's to generate something that has plausible deniability that it came from Pokémon. 0:21:52.860 --> 0:22:14.590 I mean yeah, how many go on the App Store? How many Flappy Bird with a, you know, oh so true. Like and you—but to Jason's point, those don't experience the sort of commercial success. The numbers I've seen are absolutely crazy for how many people have bought this game. I haven't bought it yet. I've been. 0:22:14.590 --> 0:22:22.800 Tempted because it actually does look hilarious. But yeah, my neighbors are actually going to play it, so I'm feeling kind of compelled to buy it. 0:22:25.322 --> 0:24:22.760 All right. Well, hard pivoting off of this one, I found a blog article here which is something that's meant—a sort of, I don't even know how to call it—not a meme in like a picture with an image macro or whatever pasted over top of it, but just a sort of meme-like going around which is kind of absolutely silly ways to solve determining whether a number is even or odd. And so I link the blog here. The title of the blog, it says the blog's name is Blabbing, but this is the only post I saw on this blog. So, I think a new blog, but anyways, the link is will be in the show notes. But this person was saying they saw on TikTok someone get kind of ripped which again, I'm not on TikTok, so I can't third-hand at this point, but getting ripped for saying that they determined if a number is even or odd by basically a bunch of if/else statements. So if number is one, it's odd; if number is two, that's even; if number is three, that's odd. I'm gonna get messed up already. Most people would say, 'Oh, you just use a modulus operator.' If you know what a modulus operator is, or 'Oh, you use, you know, bitwise and with the least significant bit just being one, and then that'll tell you right there.' There's like a number of ways to kind of do this check. And most people would be expected as like a test of your language knowledge to know what the modulus operator is at minimum or bit masking depending on the context. But this person decided to in the blog decided to take it on themselves to for 32-bit numbers, which is four billion different numbers, write if statements. And the interesting part and the reason I bring it up is first just the absurdity of it. They decided to write it in C with what sounds like some tongue-in-cheek justification initially. But what they did, which is an actual interesting thing to learn to do and you know on topic a bit for us today, is to do code generation. And so they wrote a simple 0:24:22.760 --> 0:25:31.240 program—far simpler than a four billion line program to generate all of the if statements automatically. And that way they could produce that output. And what does that take which is a useful skill to be able to know how to do because there are times when doing not that, but other code generation, writing code to make code, is very useful. That didn't work—the compiler C compiler didn't like that many lines, so then it decided to output raw assembly and actually did get it to work. And I guess said it was very performant. If you think about it, it's almost just a big lookup table, and you can just there are jump tables, so you can just basically jump to a set offset. So if you know each if/else statement is two bytes, and you had a very clever assembler or whatever, you could just jump based on the number that many memory offsets times two or whatever. So I could imagine it being very, very efficient, and then just having a table where it either outputs zero or one. But yeah, so they actually did manage to make assembly that they say works. Of course, it's a—I guess we could figure out four gigabyte file roughly. So it's a gigantic executable. 0:25:32.820 --> 0:25:53.830 Gigabyte. Ridiculous in that like there this is of course the I don't want to say this is a very silly way to solve whether a number is even or odd, but actually leads to some kind of interesting computer science exercises in doing this. And then of course people responded, 'Can you do it for 64-bit numbers?' And answer is no. 0:25:56.952 --> 0:26:01.373 Yeah, I love how this—this is the only post in this person's blog that's okay. 0:26:02.250 --> 0:26:06.992 Are you gonna post something up when you have a cool idea and you want to show people? Like, I don't fault them. 0:26:07.904 --> 0:26:24.019 No. Yeah, no. I mean, I didn't mean that sarcastically. I literally love that this person—you know, it's not like ten posts about like 'Hire me,' or like self-aggrandizing, or any of that. It's just literally like, 'Here's this cool thing that I did.' Boom, blog. I 0:26:24.509 --> 0:26:27.833 will say though that's like a tough act to follow, so I don't know what you make post. 0:26:28.424 --> 0:27:49.660 Number two, that's true. Yeah, I mean, you really can't just start a new blog; it's been a month now and he hasn't made a post. Um that is awesome. All right, my next news is SeamlessM4T. Have you seen this? Patrick? No, uh—this thing is freaking wild. So this is what I use to translate the episode. So if you haven't done this yet, go to programmingthrowdown.com. Depending on a bunch of factors, the very latest episode might not have this. Just go back an episode and you can listen to us in Mandarin Chinese. So it's totally freaking wild. So basically the way this works is it does the speech-to-text now. Like you have to have a separate file for each person. They have to have a way of separating or—or in our case, you know, we have a separate MP3 for each person on the show. But you know, it converts that to text and then it translates the text to other languages, and then it goes from that language's text to one of several voices that they have for each language. And so 0:27:51.972 --> 0:29:49.995 the reason why they're calling it Seamless is I don't think that it's end-to-end learned, but it might be. But you know, it is separate sort of modules, but the API and all of that is extremely simple. I mean, you basically just say, 'Here's audio, and I want to get back text,' or 'I want to get back text and audio in another language.' I tried at first translating English to English, so it went from US to robot US. That was really entertaining. And then I started picking other languages. So this is amazing. I mean, it's amazing that it's free. It's totally open source. Anyone can use it. I actually bought for Black Friday a few months ago a 16-A GPU at 16 gigs of VRAM—that's the bare minimum you need to run this model. But if you don't have that, it's totally fine. I also got this to run on Google Cloud Platform, and you pay about 50 cents an hour, I think. Oh, rent one of those? Yeah, very reasonable. You can even now they have something where if it's idle for so many hours, it shuts itself off, so you don't have to worry about forgetting about it and getting a $500 bill or something at the end of the month. So so yeah, all of that is super convenient. So you can run it on Google Cloud. There's also Google Colab. You can run it there. There's some 16 gig VRAM GPUs there. I don't know if the free plan would work, but the paid plan is only $10 a month. So yeah, check this out. I mean, it's amazing. I will say that there's two modes: there's a batch mode and a streaming mode. The batch mode loads everything into memory, so 0:29:50.265 --> 0:30:47.033 you know, you can really only use it for like a 10-second file or something like that. You know, the streaming mode is really where it's at. The documentation is still pretty nascent. What I found really useful was they gave a tutorial at NeurIPS, which is an AI conference. They did a whole tutorial on SeamlessM4T, and you can watch the video and follow along as if you were at the conference for free. I was able to find that. So check it out. Check out this tutorial. I feel like this is really on the vanguard of something super exciting. I'm looking into, you know, can we clone our voices? Because if we could actually—if it could be Mandarin Chinese but it sounds like us, that would be even better. It's like a T, but boy your 0:30:47.033 --> 0:30:47.820 earrings match. 0:30:51.269 --> 0:30:55.167 It's gonna be like the DeepFake Jason, but yeah, speaking Mandarin. Yeah. 0:30:55.639 --> 0:31:04.465 So if in a few months you might hear Patrick and I in Mandarin, maybe we'll swap out an episode just for fun and see if people notice. 0:31:05.460 --> 0:32:27.979 I hope people notice. No, no, an old, an old episode. Oh, I see. I see. I see. You know, people focus like the machine learning stuff on the, you know, OpenAI, and I understand why. And the sort of approach to AGI, I guess, like general intelligence, but you're right, there's a lot of stuff happening out in the periphery that feels like we're really moving forward. Especially once we start combining some of these things. So there's been a long work separating back speakers. People say they can't tell the difference between us. So I don't know, maybe that'll have to be like Gen Two of that, but you know having a single file separating it back and then running it through the system and combining it. But also, you know, we've not—I don't know that we've talked about it—but the sort of NeRFs and Gaussian Splatting around, like making 3D models from pictures. And right now they need kind of a lot of pictures of a specific kind, but you know that'll get progress as well. And then you can imagine the sort of Stable Diffusion feeding into those things, like there's a lot of interesting stuff like one or two iterations away, and you know probably requiring some leaps there because it's not sort of like gluing normal software together, but yeah, there's a lot of exciting stuff happening that I think gets missed for the focus on LLMs. Yeah. 0:32:28.418 --> 0:34:03.200 I actually am more excited about Diffusion Models and LLMs. We should do it. We should do a whole show on Diffusion Models. All right. Well, we know a person who probably speaks to that; it's not me. Oh, I'm just thinking. Yeah, I was wondering if we knew a third person. But yeah, you know, have you seen the Diffusion Model with Bomber Man? No. Uh-uh. Yeah. So basically, okay, let me just do a really quick one and then we'll dedicate a show to this. The way Diffusion Models work is you have a bunch of references. So in the case of Mario or Bomber Man or any of these games, you have levels that were created by humans, right? That's your reference. And you want to create more of those, right? So what you do is you corrupt the references—in this case, the Bomber Man levels. You corrupt them, but you're the way you corrupt them is reversible. So whatever you do, you know the exact way to undo it. And then you train a model to reverse the corruption. All right. And then once you have that model trained, you give that model like random noise, like levels that are complete garbage, and you just tell it to reverse the corruption. And you tell it to do it again and again and again and again, and you tell it to reverse the corruption until it gets stuck in a cycle. When it gets stuck in a cycle, you say it's done. And that's your Bomber Man level. 0:34:03.200 --> 0:34:13.077 It does, and it does. It should have guarantees on like playability, like not having something you can't reach or so it... 0:34:15.507 --> 0:34:31.133 Doesn't? You know, it's all just AI, so there's no guarantees of anything. You'd have to do some fine-tuning afterwards, but it's like scarily true to form, you know. Like—yeah, I can't remember if they if they um I can't remember. 0:34:32.840 --> 0:34:37.659 If they wrote something to throw out the levels, you know, like they wrote basically a validator and... 0:34:37.967 --> 0:34:38.676 Throw out all. 0:34:38.676 --> 0:34:59.960 The bad levels? Or if they just like you know they didn't actually make a real game with it. It was a research paper, so they might have not bothered to do that. But but you see the levels, it's like yeah, that looks pretty cool. Like I could see how that could be kind of a fun Bomber Man level. We used to take it for Diffusion, but I guess like there's like some... 0:34:59.960 --> 0:35:18.248 Randomness that goes into how you select each operation. So I guess almost like backtracking. You could just naively get towards the end and sort of try to go back and assign some attempts or whatever, and you know, oh this, you know do a validation at the end, like did I get a good level? Did I get a validatable level? 0:35:19.480 --> 0:35:40.220 Yep, yep, yeah. Exactly. Oh man, it's gonna be cool. The future is scary. We're all gonna be out of jobs. The future is super scary. I mean, I definitely wouldn't want to be in the games industry. I mean, as an engineer, you would be fine, but if you are a concept artist or you know a voice actor or something like that, I mean it's—it's gonna be... 0:35:40.220 --> 0:37:35.240 There are some yeah, heads of studios and stuff coming out and basically forecasting like reduced need for a lot of those things. People talking about doing like cold calling and sales calls and informational sessions, you know, maybe reduced need for folks there. And then, you know, also some places are getting flooded. There's always been a sort of like low bar Kindle book, ebook, you know, coloring book drawing, just like, you know, low effort sort of written word kind of stuff, and all of that too I think is—this AI stuff is sort of competing with not the, you know, top-tier stuff, but the sort of like low-tier, low-effort people churning those out hoping one or two people buy them or whatever, which has proven successful. I think there's a threat to those, not not that that's going to be a big problem, but once they start targeting people who are higher up the food chain, there that's that's going to be a struggle. And I will weirdly segue that into our book of the show from high-quality books that are definitely not AI generated. That's right. So my first one I tried—I might have recommended it before if I did—well, it's worth a second recommendation. I tried looking, but after 171 shows, it's a bit of a process to try to search for it. And that is Foundation by Isaac Asimov, which is a actually many book series, and the first one is you know great place to start. And recently this became a TV show, which I think renewed some interest. The TV show doesn't follow the book super closely, but if you've never read some of the classic science fiction, it's really like a fascinating trip to see how close some of the stuff got from people writing long before, you know, computers were a thing, much less mobile phones. Some stuff is just laughably... 0:37:35.240 --> 0:39:15.224 Wrong, and some stuff is just like wow, that's eerily the same issues we're dealing with today. And so reading classic science fiction is something worth doing if you've never tried it before. And The Foundation series is a I feel like a pretty formative series of science fiction. And just to give people flavor since I've not said anything actually about the book, is the idea is there's an empire ruling the galaxy, and there's concern that this these people raise that the Empire is going to collapse and it's going to cause humanity basically to be set back and really be a struggle, and everyone's going to be bad off. And so they've created a sort of way to help shortcut that to make it better. But of course, the Empire itself is very unhappy with this. And the whole prediction that this was going to happen is—this of course fake—but it sounds very convincing: psychohistory, which is this idea that you cannot predict individuals, but that you know sort of like I think the analogy they use is like a gas. You can predict a lot of statistics about gases even though an individual molecule in a gas is very difficult to predict. And so through that they can kind of predict the course of human history and how things will go and make updates and help keep it on a plan that will benefit the most humans. And it's just a very interesting read in the dynamics. So if you've never read it before, and if that sounds at all interesting—although I probably did a poor pitch of it—or if you've watched a TV show and maybe didn't realize it was also a book, I'll pitch Foundation by Isaac Asimov. Yeah, did you? 0:39:15.966 --> 0:39:19.746 Read it recently? Or is that when you read? I read it actually as a teenager. 0:39:19.746 --> 0:39:22.109 So, I yeah, I read it a long time ago. 0:39:22.547 --> 0:39:28.335 Yeah, same here. I have to confess I don't remember that much of it, but I do remember having a big impression on me when I read it. 0:39:30.293 --> 0:40:03.351 I think I also had kind of forgotten. I read it and then I like looked up the sort of summary, and it was like, oh, I remember. And then I actually have watched—I think there's a couple seasons now of the TV show—and I will say, you know, it's a pretty dark thing, so not watch with kids, but pursuing the same kinds of ideas, and I thought I thought I liked it even though it doesn't follow the book that closely. It uses some of the concepts and the sort of setup to kind of examine some different ideas, but in a similar sort of similar vein. 0:40:05.579 --> 0:40:24.816 cool yeah definitely folks haven't read that definitely give it a read you know it's a great series um my book of this show is propaganda by edward bernays so this uh this is a fascinating book um it's it's so wild that i almost you know i really i i wouldn't 0:40:26.339 --> 0:42:15.972 Really? I mean, a book about propaganda might be propaganda. You know, it's kind of like meta, but basically the premise which they get right into in the book is that democracy can't work, and so you really need to have just a few different ideas to follow that it can't just be a total free-for-all. I mean, a good example is the App Store that Patrick was talking about where there's like a million knockoffs. Like you could search pretty much any word in the dictionary and you're going to find a video game based on that word, and they're all kind of cookie-cutter, and there's no—it's very hard to get noticed in the App Store or to find good content. And so you're really relying on the editor's picks, and so in a way it's like you have this really democratic platform, but what it really degenerates to is just like a handful of editor's picks and folks who have a big audience already, and all of that. And so the claim by the book, *The Propaganda* book, is that there's just so much content, and this was written 100 years ago—1928. Yeah, there's just so many clubs that you could be a part of, there's just so much, so many choices, so many brands. So this I mean that propaganda is the way by which like 90 percent of your options get filtered out and you can just focus on the top 10. So it almost becomes like a— 0:42:18.233 --> 0:42:33.049 Way to ante in. It's like okay, you know, you have to build this propaganda machine to be part of this system, and everyone who doesn't do that ends up just getting kind of lost, right? 0:42:34.787 --> 0:42:42.921 You know, and it dives into how propaganda works as a bunch of examples. Some of them were pretty mind-blowing. Yeah. 0:42:45.520 --> 0:44:25.032 Won't spoil them, but maybe I'll spoil one. There's this cigarette company that they were under a lot of pressure. There were a lot of moms who didn't want their sons and daughters smoking because they had heard about all these issues—I don't know if lung cancer was a really known entity in the 1920s, but you know they thought there were health issues. And so they were able to like start up this like Women's Liberation March sponsored by Marlboro, and they basically tapped into the women's liberation movement of the 1920s as sort of like a back pressure against like this kind of like mothers against smoking group. And the whole thing is just blowing my mind. I mean, it goes without saying that—I think that's fair to say—Patrick and I are like what would you say is task-oriented in the sense like we don't—we're not definitely not master manipulators. We're not like these kind of people. You know, Patrick doesn't even have a social media account. I barely use my social media account. And so for me this is like a whole different dimension to humanity really. I mean, it's something that I never would have really considered or thought of if I hadn't been recommended this book. So I found it absolutely fascinating. Highly recommend it. Very interesting read. Yeah, I think. 0:44:27.120 --> 0:45:32.280 Sometimes words get artificially intense negative things. So I think like some of these things you're just saying—the other one I'll kind of by analogy or by similarity talk about is politics. So like people have, 'I don't want politics in the office.' Like, well, you need to be aware regardless of whether you want to quote unquote play politics. Politics is a human way of how things happen and decisions get made, and know back door, back room decisions. You know these kinds of—you have to be aware of them if you want to kind of at least have some modicum of control over what happens. And I think by extension propaganda is similar. It's very negative tone to it, but it—I mean, it's a way of lots of people doing stuff. They may not see it as propaganda, right? They just see it as, 'Hey, I'm trying to get my ideas across,' or find a way to make people more open to this.' But sort of calling it out and putting a label on it saying how common it is is a way of sort of engaging with. 0:45:32.280 --> 0:47:26.100 It. Yeah, and it turns out actually the negative connotation to the word propaganda is also propaganda. So the word—they actually talk about this—the root of the word propaganda comes from the Catholic Church in like the 1600s, had a part of seminary where they were kind of teaching priests and bishops and cardinals about how to kind of spread Catholicism. And I think that's where the word actually comes from. And then you know, and then I think—and so at that time it had a positive connotation. And I think the connotation flipped over time as a society we got kind of like not happy with proselytizing, and then and then now it's like really commercial propaganda and all of that. So yeah, like *Propaganda*, I think collided headfirst into the Enlightenment and lost when it comes to reputation as a word. But yeah, it's totally mind-blowing. Highly recommend people read. And to Patrick's point, we talked about this in the marketing episode. We had an episode with a gentleman from marketing. You know, like you have to—I'll give you a really concrete example. I made that game *AI Hero*, right? I made it totally free. You know, there's no ads, there's no in-app purchases or anything like that. You know, I told my friends and family about it all that, and it has—I didn't check recently—but it has I think like a hundred downloads or something like that. And now you know, I made it to be frank, I kind of made it for myself. I mean, so I'm not out there trying to get a bunch of downloads. 0:47:26.100 --> 0:48:44.805 But I was even still surprised that it got such little notice. And it really just doubles down on that point that you know you have to get a propaganda cycle going, a marketing cycle going for anything that you want to have mass appeal. Otherwise there's just too much content out there. So yeah, it's not a propaganda marketing. You know, these aren't words that you should be afraid of. I'll say that it's not something that at least I don't know about Patrick, but it's not something that I personally am very good at. It's definitely something where I try to partner with other people who are much better at it than I am. But it's something that you know I think we all should really appreciate. It is very important. If I did want *AI Hero* to get a ton of people, I would have to go about things very differently. I think you know I'd have to be posting regularly on Reddit. I'd have to figure out some way to maybe gin up some controversy. Maybe I'd make a fake post saying that it's generative AI, pretending to be somebody else attacking me for using generative AI. 0:48:44.805 --> 0:48:46.965 Know, but this is just from reading. 0:48:46.965 --> 0:49:08.245 The book. There's a million things you could do to amp up your thing, whatever it is, but if you don't do that, probably no one's going to pay attention. And that's true in politics. You know, if you make a political party, nobody's really going to care. It's true in most things. So yeah, it's just a fact of life. Yeah, I think there's 0:49:09.089 --> 0:50:59.180 A lot to say here, but probably just move on for now. As they can all right? Yeah, moving forward, so time for the tool of the show. All right, what's your tool? Not a tool, a game. My cop-out is I've been getting into these recently, but this is the one that I sort of encountered first, which is The Room. This is pretty old at this point, so most people probably played it before. If not, you know, I would encourage you to check one of them out. Often they're very cheap or free on Sort of Android, iOS. I think there's a Steam version, probably a web version somewhere. But the idea is basically kind of like an escape room in an app. You know, just like without the time limit. You just like a series of puzzles and you interact with them. For me, I've actually never done an escape room. Why? I just like it just hasn't been a thing that I've been able to do. But I'm sort of fascinated by the concept, and these games are just like an easy way to spend a few minutes sort of playing with something and trying to figure it out. I will say in the genre some games do it better or worse, and I have my own particular sort of like—I don't like things that you're just supposed to try random stuff until they happen. There should be some clue that you can look for, and it can be obscure, or it could be that you have to pay attention or whatever. But sometimes it falls into like, 'How was I supposed to know that?' Like, you know, there's no indication that you're supposed to touch the top of the windowsill and like it's going to be movable. It's like, okay. But sometimes you fall into this. Anyways, I guess a pitch for these kinds of puzzle escape room games in general. There are many in the genre, but for me, the first one that I sort of bumped into was The Room series. I think there's like now four or five in the series, but you know, shout out to the original one. And if you've never played one before, you know check 0:50:59.180 --> 0:52:38.254 it out? Yeah, totally. I've played The Room. I think it's great, a lot of fun. It taps into your common sense. So yeah, you do have some intuition into what the answer of the puzzle—the answer to the puzzle is. My tool, The Show, is also somewhat of a cop-out because we've had them on the show before. We did in fact had a whole episode dedicated to Incredibuild, but you know, I was building a new version of NameHub on my on my house today, I guess, because it's like using all the desktops in the house. It's kind of the point of the tool. And I just thought to myself, 'Man, this is such an amazing tool! I've used it for so many years for free.' You know, if you're using it for your home use, it's not for professional use; it's totally free. That I feel like—I felt like they deserve another shout out after so many years. We've gotten tons of emails from folks who have, you know, got their company to buy Incredibuild and all of that. So we have been able to drive some business to them, which I'm really proud of. But yeah, it's Incredibuild. If it basically the way it works is it hooks into your build system. It even has plugins for Visual Studio. So if you're building on Visual Studio, it works. If not, it has, you know, it hooks into GCC and all that stuff. And you just instead of, you know, choosing 'Build Solution' for Visual Studio, you choose, you know, start Incredibuild. And I have the Incredibuild agent running on all the desktops here in the house, and they all just start the fans—all start roaring—and they all start working together to build stuff. So NameHub takes, I think like two and a half hours to build, but 0:52:39.756 --> 0:52:51.265 with Incredibuild, you know, I can get it done in like 10 minutes or something. So it's super nice and highly recommend it. Very nice, very nice. 0:52:52.227 --> 0:52:57.200 All right. Well, I think it's time for our topic, which is compilers and interpreters. 0:52:57.200 --> 0:53:06.419 Yeah, this is a request by Jessica W. I won't say your last name. I know some people might sense that, but thank you so much, Jessica, for this idea. It's a great 0:53:08.174 --> 0:55:07.514 one. All right. I didn't know, you know, Jason sort of bowed out of like—well, that's not my background. So it kind of fell to me, and I struggled a bit with the tact to take here because there's a lot to be said. This is a big request. So I decided to take a stab at what I'll say is a sampler, you know, kind of going through a bunch of different topics in the area, um with some sort of high-level overview. But, you know, choosing an interpreted versus compiled language—I guess you could say choosing a compiler versus interpreter—but most of the time if you choose a language and the pros and cons of that language, you're sort of choosing one of these before that. But then in each of these there's just an incredible depth you can go into or not. You know, most of the time I guess we're big believers in—and you know, the right tool for the job. So despite the fact that I may throw stones at Python, you know, a lot of times you're only going to run an analysis once, or you know, it doesn't really even really matter how long it takes, and so you can get this just really big worry about speed or performance or, you know, selecting exactly the best tool. But I will say most of the programs I write are probably run less than a dozen times ever. Even to check them in and put them at work, you know, and you—I may run it a few times. Someone else may use it. We may copy and paste the code around, but any given instance is just not that. And then, and then a very few are, you know, run hundreds of times every day, and you know, or thousands of times in inner loops and calls. And those ones—and so there's a really wide range. And so I find a few things that you're sort of proficient in, and I'll say way of disclaimer before we dive in here is just sort of like you'll see a lot of internet flame wars about, 'You know, oh, that's slow because it's a scripting,' or, 'You know, don't if you're going to build something in Godot, don't use as a GDScript because it's, you know, it's going to be slow.' 0:55:08.138 --> 0:55:38.142 And it's—I don't—I mean, if you're finding yourself in a problem and maybe you want to listen to some of that or being aware is good, but I feel a lot of people hesitate to take the first step and get what you know I'll call analysis paralysis. You know, from making a decision, and actually better to just build something and let it suck and be slow and introspect why at the end of getting somewhere with it and then make a follow-up choice than to never take the first step. Yep, totally. There's a balance there. But okay, off of my soapbox. All right, so I 0:55:39.779 --> 0:57:37.820 guess anytime we run a program, we're talking about, you know, a processor, and we were talking in the Christmas episode, holiday episode, Jason was mentioning—oh, oh, and I already forgot if it was RISC-V or RISC-5. I think RISC-V. Yeah, RISC. Okay, good. RISC-V, and RISC-V is a instruction set architecture, a sort of way of having computers run, and it has, I guess what you would call machine code. So out of, you know, wherever it's loading the program—probably RAM—and then it sort of loads a set of bits, right? If it's a 64-bit processor, it loads, you know, 64 bits and can kind of look through there, and part of that is the operation to perform and some instructions about to load it from a register or to load it from a place in memory or or whatever. And then the processor sort of handles that. And that is machine code. If you ever do like Nand2Tetris, or there's another game where you build logic gates out of NAND on your web browser, or you take a class in university where they're sort of teaching you about sort of building up from like VHDL, you know, how to kind of implement a processor, you'll end up realizing it really is just, you know, hex numbers on a screen that you end up getting executed in there. That's your program. And every kind of family of processors has a different set of instructions, a different way of organizing those operations, and to write a program in it, you would have to sit there and really write—right? You know, hex, you know, or you can write binary too, but you know most people would write it in hex, and that's the way we sort of think about it because it gives you the nice byte delineation. And you would not do that after about maybe one time of doing that; you would not want to do that anymore. And so as an example, like when we did Motorola 68K processors, which is just, you know, one of these families when I was in university and we had to do this as a task, like, you know, write some machine code, and I did that 0:57:37.820 --> 0:58:30.351 once before sort of realizing, 'Wait a minute. I'm just going to write a program to be like a crappy assembler.' And you know, do that. And so then that moves us to assembly language. And so assembly language is just like a very light skin over this machine code. So you're pretty much having one-to-one correspondence, but you're writing English symbols. So instead of hex code 47 being the move operator where you move, you know, one set of bytes to another location, you would say 'MOV,' 'Move.' Right? That's a lot easier. And then the registers each have names rather than just numbers like, you know, R8 or whatever for register 8. And so this is the first time we encounter what would you would call a compiler, and the compiler takes those English letters or whatever language you're writing in letters and maps them directly and emits the bytecode. So 0:58:31.364 --> 0:58:36.443 Does that mean that it's reversible? Like can you go from machine code to assembly code? 0:58:37.017 --> 1:00:34.660 Yes, great question. So you can—it's called disassembly, and for assembly, and I'll talk about a caveat here in a second. It's pretty straightforward. So if you take a program that lives and you haven't gone—we had oh, that was way, that was many years ago, but we had the folks from I think Intel in here talking about obfuscating your program code so people can't do that. Um, and they can't sort of like patch it or do what would in video games would be a hacking or you know, and more critical software trying to get your keys or something. But yeah, so you can do disassembly. So you can take the bytes flowing through your processor and sort of convert them back to assembly because yeah, it's a very direct mapping. Got it? Okay. Um, so the caveat there that I was already alluding to, and you can't help here, it's not too bad in assembly is what I've described so far. You kind of do in a single pass, but that also turns out to be easier to do better than, and you know, it's frustrating. So you wouldn't want to stay there very long, which is you may want—you may want to have an example where you have a loop, and so when you're on a loop, you want to go back in your program to somewhere previous where to get forward if you have an if statement. Um, and if you were doing what you know I described initially, and that's kind of like a single-pass compiler, you would have to calculate, you know, go down, figure it out. It's eight instructions for my loop, so I need to go back to that branch statement and say jump eight instructions down. That'd be really annoying. So what people do is implement the ability to add labels so you can put a label, you know, end of loop, and you can say, you know, jump jmp, and then you can put end of loop. The issue is when you're doing a single-pass compiler, you're scanning down. It says jump end of loop, but it's never seen the symbol end of loop. So it doesn't know where to jump to, right? And so now you get what we call sort of a two-pass compiler. So now you start to get what you would kind of think about variable names and labels and jump statements. It's not really that much extra, but your productivity goes way higher, and for many years 1:00:34.660 --> 1:00:54.244 people coded in assembly with two-pass compilers. And the two pass just means first scan through to find all of the symbols that are names and their, you know, locations, and then on the second pass does the actual emit the machine code. And now it knows oh, this offset is, you know, this many instructions down, and it's able to produce the machine code. But that 1:00:54.497 --> 1:01:04.842 does it—does it literally do it in two passes? Because I could also imagine like anytime you see a jump forward, you just keep track of it and then you go back and fill it in. 1:01:05.871 --> 1:03:04.200 I mean, but that sure—but that's still you can't do it in a single. It's not one pass because you have to go back up, right? And right, so I guess like the Big O notation kind of thing, like how do you want to call it? I'm sure, I'm sure they get pretty sophisticated there. There's probably, you know, it's just calling out that the progression, I guess, in here, but still these assembly languages you're writing in, you know, there's kind of some asterisk here, but for the most part, you're writing it per, you know, kind of processor. So if you want to write it on a, you know, if you're going to use an Arduino, which is I think based on a PIC chip, then you need to use the PIC flavor of assembly. If you want to do this in, uh, you know, Motorola 68K, x86 processor, ARM processor, if you're, you know, going to run on a phone, every one of these has a different set of instructions and therefore a different assembly language. And so again that gets really crappy if you want to ever write for more than one thing or you want to go faster because you're pretty much still basically writing one line of code per instruction, which is going to make you very easily able to sort of intuit about sort of like runtime and complexity of things, um, a little easier than my high-level language. Um, but but again it gets kind of frustrating, and there are concepts for still everything that's in a high-level language sort of has to be capable of being done in assembly language. So you still see things like doing function calls or jumping to a function and pushing stuff onto the stack for some processors. Those things are strictly implemented by a higher language compiler, but some processors actually have support for that even at the assembly language level. Does the new sort of start to call those things Complex Instruction Sets (CISC) versus Reduced Instruction Set (RISC) computers? And so there's a whole separate topic we'll sort of leave this here because we'll go forever, and and move up—move up the family tree, I guess, to higher, higher level 1:03:04.200 --> 1:03:18.964 languages. Yeah, that makes sense. So so uh um so every programming language like that you read about C, C++, Java, Python—they're all considered high-level. 1:03:19.656 --> 1:05:13.799 Yes, yeah, yeah, yeah. I mean, there's degrees to that, I guess. You know, you but at least when I sort of started learning about this and you know they shift over time, right? There was early on it was you writing an assembly or you're writing in, you know, something that wasn't assembly, and anything that wasn't assembly was, you know, high-level. And the idea there being in my mind at least the delineation um and it may be it's moved over time, but if you write a line of code and it produces sort of an arbitrary number of machine instructions, then you're now writing in a high-level language. Got it? Okay. So in a line of C code just as an example, or Java or whatever, you can write, you know, A plus B divided by C times D, you know, to the power two, whatever, right? That's that's one line, but it's many, many, many, you know, assembly operations to do that depending on architecture. Um and the compiler now is able to target different backends. So if you write, you know, and we'll talk about sort of C, C++ here for a second, if you if you're writing in C or C++, you can compile against many different processors. So you could write one program that runs on ARM or on x86 because the compiler is picking up your code, and and we won't go into a bit but building sort of finding the symbols, building a syntax tree, and it knows how to emit the proper machine code for, you know, executing that program. And it does it can do that with many passes depending on the compilers, but by many compilers also emit a sort of intermediate representation. So even different front ends, so you could say like an LLVM—LLVM—you'll see things that can target, you know, the LLVM IR intermediate representation, which is something which means the sort of machine code assembly and the high-level 1:05:13.799 --> 1:05:37.829 language that lots and lots of front ends can compile into. And then there's a specific set of operations and libraries and team working on taking that intermediate representation and compiling it down to your specific final machine. And so then that's called the back end. So you have the front end compiling it to intermediate representation, and in the back end taking it from intermediate representation to a specific class of processor. Got it? 1:05:38.942 --> 1:05:55.446 See? So if you wanted to write a new language, you could—you could create a language that compiles to the LLVM intermediate representation, and then you'd be guaranteed that it would run on all these different processors. 1:05:55.902 --> 1:06:17.680 Not only that, you're also going to benefit from a bunch of optimizations that the sort of like imagine loop unrolling. So you write a loop in this intermediate representation, and now the sophisticated folks across all these languages economies of scale kind of thing can implement loop unrolling or some stuff we're going to talk about later, but you know it can implement it at the IR level, and then you can benefit from it. 1:06:18.464 --> 1:06:19.932 Got it. Cool, that makes sense. 1:06:20.657 --> 1:08:15.980 So now you mentioned Java as well. Java just does, you know, gets a special shout out here, which is that when Java is compiled and this starts to say, you know, these are words we use, but you're no requirement that it has to map cleanly. So Java also targets sort of an intermediate representation called the JVM—the Java Virtual Machine bytecode. And so it does get compiled, and it has all this, you know, optimization and stuff that can happen, and then it produces basically a machine code, but the machine it targets is the Java Virtual Machine, not an individual processor. And when it was first developed, this was a huge boon because now you talk about portability—you can run on anything that there was a JVM for. The JVM for, and LLVM wasn't really a big time. So, you know, again times change, but at the time this was like seen as like a really big thing. And Um Sun Microsystems at the time, you know, we're sort of sort of pushing that hey, Java really, you know, runs on—I forget they had an ad, you know, some obscene number of devices and, you know, has this ability. And so there you're compiling it down, and then the Java Virtual Machine has an implementation per, you know, kind of architecture. So it has an ARM implementation and it has an x86 implementation. And the implementation of that virtual machine is to take that bytecode and pretend it emulates—right? Emulates this sort of conceptual computer that is like a almost like a lowest common denominator, but on some processors it may be able to be very efficient with a certain operation and others not. But you don't have to worry about that because, you know, it's sort of just handled for you. And um all of the standard libraries can have sort of special treatment or efficiencies added. And so um this was a very interesting approach, but it 1:08:15.980 --> 1:08:40.095 sort of threads the line here where you have—it's not a scripting language. We'll talk about that in a second, but it's not a true compile and that there's still an emulator, a virtual machine as it were, that sits in the middle here. And and sort of that's the program that's actually running. So that program is running your program. Um and so the compiler is not targeting a machine. The compiler is targeting—well, it is targeting machine, but it's not targeting a physical machine. 1:08:41.192 --> 1:09:12.779 So if I understand correctly, right? So the difference is with LLVM, you're going to this intermediate representation, but then you're instantly going to machine code. And so now you have machine code that can only run on that architecture. But with Java, you're still going to this intermediate format, but you're not doing that second step to go to machine code. At compile time, you're doing it at runtime. 1:09:12.779 --> 1:11:01.289 Yeah, so the intermediate representation is not really executed right there. There's no program—well, again, you could write one—but the intention isn't that there's a program that runs the intermediate representation versus in Java that there is a program, the JVM, that runs your, you know, JVM bytecode. Got it? Okay. So then now we get to sort of scripting and scripting and interpreters. The idea is sort of similar. It's very rhyming, which—you know, BASIC was this way, other things are this way and that. Instead of a compiler, you're running a program that needs to work per architecture, but that program is doing all of the work at runtime of executing your script. So it's converting the symbols you wrote into or the letters you wrote into symbols. It's building whatever it needs to be able to execute it. So if you're, you know, setting a variable name like Jason was mentioning, it's keeping track of that, but it's doing it sort of fresh every run. And so it's opening it up. And if you have as an example, you know, a typo halfway down, you're going to find out about it out about it a down your program's execution or whatever. And so it's not attempting to in this sort of level zero. It's not attempting to do anything other than just sort of execute your script, right? So if you've ever written Python and you have a, you know, typo at the bottom of your program and your whole program runs and it doesn't output, you know, the answer because it crashes at the end. This is happening. It's not trying to compile it. It's not trying to look ahead. It's just there is the program that's running on per architecture is parsing the script input at runtime. Yeah, I mean. 1:11:02.284 --> 1:11:45.839 You can—this is starting to really boggle my mind now that I'm thinking about it. Like in Python, you can open the interactive interpreter and you can actually have, you can write if statements and for loops and anything. You can write in Python, you can write in there, and so you can actually like write a for loop in that for loop have a break if a certain condition is met and then keep going. And so like Python has to kind of remember that like you put that break there, but it doesn't really know what's going to happen after the break because like you literally haven't written it yet. And so that's to kind of keep track of all of these things while you're while it's interpreting what you're writing. 1:11:46.952 --> 1:13:45.320 It's also not that it's impossible other places, but it becomes much more straightforward to things like Python loves to do, which is modify definitions on the fly, right? So like changing variable names to another type or to even modifying functions that are on a class because all those things are sort of held in memory, right? Like the sort of processing, the compilation goes into memory and is just held and is done sort of in real time, like at execution. And so modifying that stuff is more obviously, you know, straightforward to implement. More it's not obvious, but it's you can kind of envision how you would do that if it's like a thing sitting in RAM versus if you've compiled it into something that you admit to the disk, right? And you're, you know, sort of executing it out of read-only memory. And again, there's lots of things that blur these lines, but yeah. So scripting in it when people sort of think about it, it can be slower because at runtime you have to do more, right? You have to convert the mapping. You don't need to know that your variable name is some complex, you know, set of underscores when you're doing the compile; you just map it to this is the first variable I've seen, so it's variable lookup table entry zero, right? And it's literally just a pointer or just an index zero. And then every time you see that, you're keeping track of it. But in the final program, it all gets taken away, right? Because it's just everything points to the right spot and all those things are resolved when you're doing scripting though, when you're marching down the program, that table has to live in memory, and you have to go do that lookup, right? Oh, Jason's most complicated very long named but not meaningful variable that has to get parsed, which is a lot of characters, and has to get that mapping to take place every time it's seen sort of like at runtime. And so this gets to why people kind of say scripting would be slower, but as Jason mentioned, the advantage is because execution is happening at like, you know, kind of while you're doing it, and all these steps are you. 1:13:45.320 --> 1:13:52.097 can also modify the steps, and it becomes very easy to do and to interactive and to kind of see what's happening. And, you know, very straightforward. Yeah, that. 1:13:53.245 --> 1:13:56.700 Makes sense. I believe—I believe if you 1:13:58.864 --> 1:14:19.975 your machine code is loaded into the process memory that like you can't touch. Like if you start writing in the chunk of memory where your machine code went, it'll—the, you know, like they'll assume that you're trying to do something nefarious or something or a mistake. But but in Python, there's probably no way to catch. 1:14:20.650 --> 1:14:41.845 That a lot of early viruses did this, like self-modifying code. So they, you know, people screen them; they look normal, but then they get modified to do something that looks that is nefarious, you know, at execution. Um yeah, a lot of people—a lot of systems will try to stop or prevent that and assume that you were making a mistake and at best preventing you from doing something you really weren't supposed to do. Yeah. 1:14:42.925 --> 1:14:43.448 That makes sense. 1:14:43.515 --> 1:15:51.083 But yeah, that's it. Yeah, that's mind-bendy stuff when you start talking about that. Okay, two couple other things I wanted to cover briefly. You'll hear about something is a JIT, which is just in time for scripting languages. All is not lost about these runtimes. So in Python, as an example, or other things, you can have a JIT which says, 'Hey, I've noticed that this piece of script has been executed, this function has been called many, many, many times.' So I'm going to rather than every time I get to it do all the parsing and all the work again, it can effectively do a compilation step, right? Can go ahead and resolve all of those things if you think it could even go—could do, you know, implementing them all the way out to machine code or the operations or whatever, right? Um Java has this as well for bytecode, but basically at runtime sort of observing that you're about to do something or that you've done it before and pre-computing and storing those results so that the next time you do it, time you do it, you get your execution. That—that's sort of the concept. I'm probably doing a bad job but of just in time. Right? It's just in the nick of time. I've provided you a, you know, optimized thing to run. Um right? And you 1:15:51.083 --> 1:16:01.242 could do the loop unrolling and all of that stuff. Like if you know that this for loop is only going to execute it four times or you don't know it, but you're confident, then you could unroll it. Yep. Um and 1:16:02.845 --> 1:17:57.719 you could even do code analysis to do it, right? Again that, you know, extend these all have like research directions or implementations and various, you know, features. Um and then the last thing I want to talk about that—that also blurs the line though—is something that's actually interesting if you've ever seen it, which is profile-guided optimization. So some of these drag races, I guess you call them between sort of scripting languages and compiled languages will show, 'Hey, actually this Python code is a lot faster because in this case, the, you know, during runtime Python the interpreter observed these behaviors and performed this optimization that you couldn't have known a priori. It had to be known sort of like at runtime.' And it gains this advantage. It's true that can happen. That is a thing. Um and it can be really cool in compiled languages. Um we're starting to see some work around profile-guided optimization, which is run your program and you record a bunch of statistics about what—what how many times various loops by default was this if statement skipped 99% of the time. And even compilers do some of this. Uh sorry, even processors do some of this to help with optimization, but sort of recording statistics about what was executed, how much, and and various sort of things. And that the sort of call it happy path—you make a guess. I'm just going to assume this is going to be true, and I'll add a little check if it's true. I'm going to gain this boost. And if it's not true, then I'll have to execute something a little slower, right? No free lunch. Um but it does this via a profiling. So you've profiled your code and now this optimization uses that profile of the code as a sort of almost second compilation. So you run the compilation, you insert all these sort of like, you know, extra hooks to measure stuff. You look at the output, and then you run compiling again that uses that as input, and it's able to sort of be more thoughtful about how to perform certain optimization steps of compiling. 1:17:57.720 --> 1:18:20.562 Cool. So if people are really excited about compilers and interpreters and they want to work on the Python VM or the C++ like GCC or something, where does most of that work take place? My guess is probably like government programs and giant companies and stuff. But what would be your career advice for them? 1:18:20.680 --> 1:19:29.952 So there are folks working at you know big company, big tech companies that you sort of hear, right? If we if we sort of name some—um, you know Dart, you know at Google, or Swift at Apple, or um I think Java is still at Oracle. Um, you know obviously those places have teams that work on these things, and you can go work on them. Um but I think a lot of it also happens as hobbyists do stuff, um you know in university. This is a compilers and interpreters is a course you can take. Um if you want to do it as a job, what I don't know is like how you as a job without working at a big company that has a vested interest in it, you sort of get paid to make a very cool LLVM and intermediate representation optimization? Um definitely something you can do. And if you have a company that will sponsor their work, which is you know maybe a hard thing to find—I don't know that it's not like writing a game where you just do it and then people pay you for good outfit. It's very difficult to write like a new language that someone would pay you just to consult on. Um my opinion? Maybe I'm wrong. Yeah, I think. 1:19:31.234 --> 1:21:02.680 You're right. I think even if you set out to create a new language, I think even then I would probably start by working for one of these big companies. I think like I want to say one of the military labs is like the chief person on GCC, like Sandia or one of these labs—I don't remember—but basically, you know, for all of these things—LLVM, Swift, GCC, you know—look at who their biggest sponsors are, right? Look at who um the domain name of the email addresses that are most prevalent on the mailing list. Right? It's not hard to figure out who is actually putting in most, you know, of the effort for that particular compiler or interpreter. And then once you know that, then it's a matter of creating changes, you know, basically doing that, doing that job kind of pro bono for a little while, improving yourself to those folks, you know, becoming ingratiated by those folks, and then reaching out to them and saying, 'Hey, I want to do this full time.' And so if that's something that interests you, that's the path. For to do that, you don't need to be a master propagandist or anything like that. It's a very simple formula: just put some sweat equity into that, you know, talk to the folks who are also putting in that sweat equity, and that's. 1:21:03.979 --> 1:23:00.400 How you can do it? I'll give a two two shout-outs quickly. Um, you know for—I don't think this is the first thing you do, right? Write programs for a while before attempting to write your own programming language, right? Probably probably a good move. But two two resources that I think are really useful here. Many more, but you know people would have recommended The Dragon Compiler Book before, but yeah, I never made it through it. Anyways, um the first one is by—and I might not say the last name correctly—but Robert Nystrom. Nystrom, and he wrote a Game Design Patterns, like design patterns for games that's very useful. Um definitely check that out. I think you can read it free online. But also Read Free Online Crafting Interpreters, so craftinginterpreters.com. And I think there's like, you know, a web version, but I think there's like actually now a physical book as well, and I haven't sort of gone through it and implemented myself. It's on my to-do list. Um but definitely a way of talking about how would you add an interpreter? Implement polymorphism? How would you, you know, handle variables and sort of building up your own programming language and sort of, you know, not going to be state-of-the-art, you know, pushing the envelope, but but definitely giving you a flavor for this, which I think is really useful to know how how some of this stuff works. And then in the sort of opposite side of that, I mentioned it, but nandgame.com. There's also Nand to Tetris, but nandgame.com is really easy to jump into and start showing you like starts—oh, like one touch too low—maybe for what we talked about today where how do you build a NAND gate? How do you make an OR gate from a NAND gate? But quickly jumps in basically how do you build um, you know, the actual processor components, um the parts that do math? How do the bits get added and why is it that a certain machine code byte evokes a certain behavior from from the pieces of the computer, the actual silicon. Anyways, and so you can just like play it in your web browser and, you know, just sort of go at it. 1:23:00.400 --> 1:23:11.504 Might have to Google some help. I did. Sorry. But but you know how do you implement registers? How to address buses work anyway? So that's nandgame.com. We'll put them both in the show notes, but two yeah, there's. 1:23:11.504 --> 1:23:15.520 Useful resources there. There's a modern take on this game too called Turing Complete. 1:23:16.195 --> 1:23:18.760 Oh, I've seen this but I didn't want to pitch it because I haven't tried it. 1:23:18.962 --> 1:23:36.833 I haven't tried it either. So I haven't tried Nand to Tetris either, but um but it has amazing reviews. I just looked it up. It's overwhelmingly positive—Steve Reviews, 2200 reviews. So um that is a really good endorsement by the community. Yeah. 1:23:37.474 --> 1:23:40.780 I've heard a lot of people say it's another good resource, so I'll put it in the show. 1:23:40.780 --> 1:24:32.280 Notes too. Cool. All right. Um thank you Patrick for going into into a ton of detail here. Um it's an awesome episode. Thank you Jessica for recommending the topic. Phenomenal topic. Um and uh thank you to all of our patrons who are kind of supporting the show, helping us reach new people. Um I was looking through my email the other day about something kind of unrelated and I stumbled upon a email from a listener um a while back who we were able to help them you know find their first job. I mean that stuff's really inspiring. Continue to send that to us. We read all of them. Sometimes we read them twice. Sometimes we find them years later and read them again. Um so thank you everybody out there for listening, for supporting the show, and we will see you next time. 1:24:44.873 --> 1:24:47.016 Music by Eric Barndorler Programming. 1:24:48.501 --> 1:25:06.600 Throwdown is distributed under a Creative Commons Attribution-ShareAlike 2.0 license. You're free to share, copy, distribute, transmit the work to remix, adapt the work, but you must provide attribution to Patrick and I and ShareAlike in kind.