WEBVTT 0:00:00.000 --> 0:00:23.080 Programming Throwdown Episode 179: Project Planning. Take it away, Jason! Hey everybody. 0:00:23.080 --> 0:00:42.066 It is so fun to actually do these duo shows. I have to say, I think the interviews were super lucky that we could get such a multidisciplinary set of talented people to do interviews, and I appreciate everybody you know reaching out. We get many, many requests to interview. Yeah, we do. 0:00:42.066 --> 0:00:42.657 We get a... 0:00:43.247 --> 0:00:47.449 Lot. Yeah, we get—we get I get a ton in my personal email too that you don't even see. Oh. 0:00:47.449 --> 0:00:49.019 We get a lot in... 0:00:49.339 --> 0:01:02.780 Your personal email? Nope. Yeah, see mine's too available. So we really appreciate it. I mean, we're honored that so many people want to be on the show, but it's just really fun, just the two of us talking about tech stuff, so we're gonna keep it like this for a... 0:01:04.847 --> 0:02:04.011 While. And I will kick us off with a store that we opened. Or I say by—we now, I'm talking about my wife and I, and really by—we mean my wife opened. I did a lot of the manual labor, but but we opened a—it's called Bricks and Minifigs. It's a franchise, and it's basically an event space and a retail space. So on the retail side, you know they sell LEGO, you know? They sell like bricks and minifigs, as it suggested, like sets and all that. They also have like a thing where you can build your own minifigure, and then it's also kind of like a clearing house for LEGO. I didn't realize this was a thing, but people will want to sell their LEGO, right? Which makes sense after they built it and played with it. The thing that surprised me was there's a huge audience of people who want to buy already built LEGO. Whoa! Yeah, that I did not know. 0:02:04.011 --> 0:02:04.838 That not expecting. 0:02:04.838 --> 0:02:56.442 And you know now the store is up. You see it every day like I'll be in the store building some furniture or something, and someone will say like, 'Oh, that would look really good on our coffee table.' And they just buy it as if they're at like an art gallery or something. And a lot of these people actually don't want to build it; they actually just want to have it. So really surprised at that. Now the other part of the business, which is you know honestly the part that really I like the most, is the event space. So we do like birthday parties, all different other kind of like corporate events and stuff, and people build LEGO. We have actually this cool racetrack where there's like a metal car, like a chassis—a metal chassis of a car that has the LEGO studs on it, and so you build a car and try and race it down the track, kind of Pinewood Derby style. Nice. 0:02:58.163 --> 0:04:11.260 And I guess you know, I mean beyond just shelling the store for the one percent of listeners who happen to be in Texas, I want to talk a little bit about career breaks. So you know, I basically took a career break, and I actually you know did a lot of side projects and things like that, but I also helped my wife kind of get the store off the ground. And it was awesome. I mean, I know that this is like a thing that might not be accessible to everybody, but you know if you could take a short break, and especially if you've been in the industry for 10, 15, 20 years, it's really nice to just instead of—I was always a type of person where if I decided I wanted to change jobs, I would still keep like working really hard at my current job until I found the next thing, and then I would leave like a one-day gap or something between the first job and the second job. And you know this time like having a longer gap is really nice, especially if you've been in the industry for a long time. I highly recommend it. 0:04:11.260 --> 0:04:18.758 People talk about—I guess maybe it used to be more common? Maybe it was never more common? I don't really know, but sabbaticals. 0:04:20.614 --> 0:04:36.477 Yeah. That's—I don't really know like what goes into a sabbatical, but first of all, congratulations. Tough thing to do. So good luck. Thanks! I won't quote all the statistics about new businesses to you, but I'm just teasing. It helps. That it helps. 0:04:36.940 --> 0:04:59.039 That there's not like a default, like what is it? Who's a person who Paul Graham says you know startups are default dead until they're profitable. But in our case, like we don't really have to be profitable. It's just really a fun thing for the community, so I think that helps our survival rate. There you go. Non-intentional non-profit. Um no. 0:04:59.039 --> 0:05:00.135 I'm sorry, I... 0:05:02.177 --> 0:06:58.940 Just teased. No, no. Best wishes, best wishes. But yeah, I mean, I think like taking a career break, change of things, is actually like good advice. And there's a lot of new ones not to derail into this for very long, but just if you're planning to leave a position, just look at what the rules are around. Most places your insurance coverage is a concern, so people don't want gaps because of insurance. But in a lot of places, once the first of the month hits, your insurance is basically paid for the month by your company and your premium. So if you stop on the second of the month or whatever, you're actually covered for the rest of the month. And so you could take, you know, a four-week break, basically, and you know have insurance coverage without any gap. Now again, you should please like read that—that's not something you really ask your manager, right? Because then you're kind of admitting you're leaving. But for something that if it's helpful to plan for and plan around, it can ease some of the stuff that make it a little more accessible. I would say to take a little bit of a gap. And then you can also in many positions in Tech, you can negotiate a signing bonus to help cover that. And that's not necessarily always obvious or offered, but you can always ask for, hey, you know, I—you know, I'm gonna, you know, take this. I'm convincing my spouse or whatever. I need a little bit of help to take a vacation in a, you know, in between, or you say whatever excuse, you know, helps it in your head. If negotiating is hard for you, try to ask for a little bit more of a signing bonus. And and your negotiating power, you know, starting the new job. But I do agree taking time to do something off. People talk about online. I've never done it. I kind of envy it, but it's like an early retirement, like a mini-retirement. We kind of wait till the very end of our careers and then we retire and never work again. That's at least the—they're sort of stereotypical thing. There's no reason why you couldn't save money, take a year, then return to work. 0:06:58.940 --> 0:07:32.500 And for many people returning isn't that hard after a year? You know, it is very situational and person-dependent. And compounding of money you know does make there a slight problem to the hand wavy. It doesn't make a difference, but if it allows you to come back to work with new freshness, new eyes, and new vigor and energy, I would argue there's compounding returns from that as well in your job performance and how people are happy. So yeah, yeah, I think taking a break is great. 0:07:33.310 --> 0:08:59.102 Advice? Yeah, totally. I would say a couple of things based on what you said. One is there's this thing called COBRA, which is probably just an American thing, but there's probably equivalents in other parts of the world where you get your employer's insurance for an extra year after you've left. Now you have to pay for it, but at least you don't have to like change all your doctors and all of that stuff. So so that is an option. It's also kind of interesting. You have the option of retroactively getting COBRA. So for example, you could say like, 'Oh, you know, I'm just going to go without insurance for six months,' and then oh, you break your arm. You can actually sign up for COBRA like after they've fixed up your arm and everything, and they'll retroactively like recalculate all of your medical expenses, which is pretty wild. I mean, it's actually kind of remarkable just from a Tech standpoint that they can do that. And then the other thing I was going to say—oh yeah, you know, I think it's important to have some kind of a goal. You know, I actually told people when I was leaving my last job, they said, 'Oh, what are you doing next?' And I just—I had it ready to go. I was like, 'Here's what I'm doing: I'm gonna help my wife build a retail store.' And so it's like I think having a plan helped a lot. I feel like if you just veg out for a while, that's fine. That's a different kind of career break than saying I'm... 0:08:59.102 --> 0:08:59.760 going to like... 0:08:59.819 --> 0:09:19.352 hike a mountain or travel the world or whatever. And if you're gonna have some kind of focus, then you should plan it all beforehand because you don't want to waste your time not getting paid planning. You know, that's—that's time where you could be doing that on the nights and weekends while you work your job. You know, you're... 0:09:22.154 --> 0:09:35.316 not working the nights and weekends. You know, do you? I almost never work weekends unless things are really wrong. I will work a lot during the week, but I will not work a weekend unless I absolutely have to. 0:09:35.637 --> 0:09:57.119 Yeah, unless it's just like a really quick statement or whatever. By default, if someone is saying I anticipating that I need to do it, then they've needed incorrectly, I guess. Is yeah, I try and I try to do that for you know the people that I help on the teams and stuff as well. It's like we shouldn't plan that the work needs more than the normal working hours. Yeah, I don't know if you... 0:09:57.642 --> 0:10:34.460 feel this way, but I get more and more energy as I work. So like by 5 p.m., like I'm really excited about work, and then by, you know, let's say 8 a.m. the next day, it's like I have to kind of start the train again. And I've been this way my whole life. Like it's not an age thing. Even I was 20 years old working, I still had this mentality. And so it's like I would rather work a ton of hours on Friday than like come in on Saturday and work for four hours and get pretty much nothing done. So you know, it's like it's like you're that at—I feel like that works a lot better. No, I don't share that trait. 0:10:34.460 --> 0:10:47.800 Yeah, but I know, I think everyone's different, and I think yeah, we can get into that when we talk about planning for projects, but I think different people have different ways of reacting to things. 0:10:47.862 --> 0:10:52.199 All right. Well, I think it's time for the news. Oh, sorry. All right. What is your news, Patrick? My first... 0:10:53.079 --> 0:11:42.419 The topic was interesting. I don't even know; these show up, you know, on social media like sometimes you see them on X or on Reddit or wherever, and someone just posts this, and a conversation spins up, and everyone's talking about it. Anyways, this one was someone who had posted a GitHub Gist where they put some numbers down, and I can't speak to the veracity—the sort of correctness—of these numbers. But they were saying that they were a big contributor on Stack Overflow, and that they were posting these numbers to say that, you know, Stack Overflow number of new questions has been going down. It of course sparked an interesting online discussion around it, and so I just bring it up here because I think it is interesting. On the one hand, I don't know, have you ever posted a question on Stack Overflow? 0:11:44.039 --> 0:11:55.817 Oh man, that is itself a great question. I've literally never done that. I've answered questions, but I'm definitely not a top answer or anything. I've literally never posted a question. 0:11:55.986 --> 0:12:27.480 Haven't either, but I have read enough stories—like one of these things, I guess, like in my head I've built it up to be a thing where you know, I don't think there's a lot of low-effort questions; they get closed. And I think over time Stack Overflow has gone through curation to try to get good quality questions asked. So I think part of it—and you hear the same thing about Wikipedia—I've never edited a Wikipedia article, largely for what I would say is kind of the same reason. You read stories about people who talk about the process and the context around what's expected of you if you're going to, and I just... 0:12:27.480 --> 0:12:59.540 Yeah, quick side note: I tried to create a Wikipedia page for the show, and it got rejected. This was like 12 years ago or something because we weren't noteworthy, or what was? Yeah, they didn't. At the time, we didn't have enough news stories about us. At the time, we had zero. I probably could make one now, but to your point, it's like once you get rejected—like I put in a lot of time into that—yeah, and then it's just gone. They think they literally delete it to where like you just can't even get it anymore. So it's like who wants to work on Wikipedia after that? 0:12:59.540 --> 0:14:57.960 Yeah, a bit of an aside, and I'm thankful that there are people that do, and they become committed to the idea of, 'I'm going to do this.' It's like an iterative thing, right? You get shut down; they submit again. They try something different. They're persistent. And once you get over that hurdle, I do believe it can be very valuable for you to continue sort of writing articles, submitting things. But if you're just seeing it as like a very transactional—'I want to do this one time'—I agree. I think the bar in my head at least gets to where I don't attempt it. But I will. But that's not where the conversation went. Although, I think we should be aware of that; the conversation immediately jumped on, not surprisingly, to everyone just asking their questions to AI instead. And so there were some points and counterpoints, which is a very interesting discussion about Stack Overflow. Profit has actually stayed up in part because they've been able to license all of these questions and answers to AI training. And so in some ways, the sort of questions have gone on to those platforms because more than just a search; they've sort of been able to aggregate and combine all of this information there into the LLMs that you can then interact with and sort of be more responsive and ask your questions. And then not to get sort of conspiracy, but you get kind of the 'dead internet theory.' Then the problem is, okay, but all of the questions about new languages and this—like how do we, how do we get that information out onto Stack Overflow? So just a very interesting discussion. I don't have a strong-held opinion or belief here, but just as the internet sort of rolls forward, I've benefited many times from going on Stack Overflow and sort of seeing question and response style; it fits very naturally with the kind of work that I do, and so it's very useful. I personally haven't shifted a ton of that over to artificial intelligence, but maybe I will in the future. But then what about the questions about the next libraries and all? 0:14:57.960 --> 0:15:12.681 Of that? That never get posted on Stack Overflow. You know, it yeah, over time does it naturally sort of fade into the sunset, and we you know just go other places. I mean, probably it's a natural course, but an interesting discussion nonetheless. Yeah. 0:15:13.002 --> 0:15:40.839 I mean, I think to your 'dead internet theory' point, it's like when you ask an AI a question, it might be a question that the AI's never thought of before, seen before, but it can piece together an answer from all the other questions it's seen. The problem is really that that answer is not in Stack Overflow, and so like you're not growing the collective consciousness if the AI just tells you the answer, and then it's... 0:15:40.840 --> 0:15:56.910 Deleted? So maybe the AI's need a blog where like they when they have these novel thoughts, they're sort of aggregated, collected, and then published, and then the AI's can help the other AI's grow. And it can be no, that's not gonna happen. Yeah, no. 0:15:56.910 --> 0:16:23.337 You know what's funny is actually when you were saying that, I was like, 'Yes, yes, that's right.' I think that actually is where it all goes. I think the AI will—you could do some kind of novelty detection and say, 'Oh, this is a novel question,' then we'll post the answer, the question, and the answer maybe on Stack Overflow, so Stack Overflow have a bunch of questions and answers from AI, and then people can go and correct it and all that. 0:16:25.902 --> 0:16:26.897 Yeah, it'll be. 0:16:26.897 --> 0:16:33.360 I use AI a ton actually the other day. I wanted to do this; it's a little weird. 0:16:34.880 --> 0:17:33.440 Well, hard to explain, but I wanted to do like 2,000 Gaussian Mixture Models in parallel, and so I wanted to vectorize it so that I would without like a for loop because one thing about machine learning stuff or just generally these kinds of things—like for loops kill your performance, right? So if you do like four x from 0 to 1,000, you know, A plus equals X, or A plus equals IX, right? So so you do that to sum an array, or you just use some operator on the whole array. The second one's going to be like hundreds of times faster, right? And so I needed a totally vectorized way to do a bunch of Gaussian Mixture Models, and I asked AI, and I'm fairly confident nobody's asked AI to do this, and it produced an answer, and it was actually totally correct. And so that's the kind of thing that like probably should have somehow made its way into Stack Overflow. 0:17:33.440 --> 0:18:32.228 Yeah, I had an opposite experience. I asked a question that it gave me an answer which was obviously wrong. Wrote some sample code, wrote all this stuff, and then I was just like, and I'd even just told, I was just, I don't know what, like that's not right. And it's like, 'You're correct. I did get it wrong actually.' And then it proceeded to write something else so it was actually still incorrect but at least like closer and less obviously wrong. And I found this, like you know sort of, you know diverging off your thing, you know. I—it was a very weird mental thing for me to be like by simply telling it it was wrong. It's like if you had thought longer, would you have given me the second response? Like couldn't you just tell yourself like, 'Are you sure?' It's just this very strange. So it's sort of very different than, you know, going on to Google and searching for a blog post or a Stack Overflow. Um, and so yeah, I still not long all of the, you know, asking AI the questions, but I do think it's 0:18:32.970 --> 0:20:29.560 Getting better. Well, that is a great segue to my news story which is DeepSeek claims its reasoning model beats OpenAI's on certain benchmarks. And so I have an article linked to TechCrunch here, but you know what you said is exactly right. And so you can imagine like imagine a system where every time you gave it a question, it gave you an answer, and then you said, 'Are you sure? That answer looks kind of wrong.' And it went through and gives another answer. Um now let's assume that, you know, if you kind of like tell it, okay, so let's assume that creating an answer out of the ether versus creating an answer based off a past answer—let's assume that the latter is better than the former on aggregate. So like if you have the benefit of this earlier answer and all the computation that went into it, let's assume that you could write a better answer the second time, right? So now that's really interesting because you can actually create training data for yourself as an AI. So an AI could spend twice as long answering a question and then go back and say, 'Hey, that answer—that's like twice as long. Like I want that answer right away.' And so it can fix itself, like self-healing kind of stuff. And and so this is not a new idea. Like you see this with games. So for example, look at AlphaGo, right? So AlphaGo will use tree search and like simulate future boards, right? And so it might think, 'Oh, this move is good,' and 'this move is good.' But then it does a bunch of simulations and it finds, 'Oh, only that first move is good. The other one's trash.' Well, like that's feedback to the first system that has to make that like snap judgment, right? 0:20:29.560 --> 0:22:17.560 And so this is taking that idea and bringing it into the Large Language Model space. And so um a really interesting idea now the challenge becomes like credit assignment. So you know there are times where you say like, 'Hey, think about this more.' And it does, and the answer is worse, or it's like more—it's like hallucinates even more and stuff. So you have to craft—you still need to guide the system as long you know as long as you're making this assumption that just like every second answer is better. You know if you want to relax that or push back on that, now you have to guide the system using reinforcement learning or other tricks. But um but so OpenAI has been working on this for a while. Um the thing that makes this really interesting is the DeepSeek model comes from a quant finance research lab in China that I think they were—if I get this story correctly, you know they were like doing quant stuff. And I'm not a finance guy. Patrick, you're probably gonna rip on me for this, but I guess it's like you're buying and selling things at certain times or I don't know what they're doing. So so then all of a sudden they like had to take a break from that. Um there was some like issue that they—there was some crackdown or something, and so they started working on some of these other models, and they produced this amazing reasoning model that's completely free, like source code available. The only thing that's not available is the data they use to train the model, which is which is not uncommon. I think most of these models have that closed data. Um but the model itself, the weights are free, the source code's free. You can fine-tune it. You can do all these things, and so uh it's really kind of revolutionary. We'll have to see where it goes. 0:22:17.560 --> 0:24:11.840 This speaking of credit, I've lost whose idea this was—not mine, but someone was pointing this out, and I've been intrigued. So you mentioned sort of like AlphaGo, and I think one of the things that was really interesting is initially AlphaGo was trained with sort of, you know, all of the Go games that were online, basically like these expert games, whatever. Then eventually they said, 'Hey, because there's rules of the game,' I think this is the guiding you're talking about. There's a framework for winning and losing better and worse, which is sort of different from all human knowledge, I guess. But the—and they kind of later said, 'Oh, actually we've removed all of those games, started again, and bootstrapped with just playing each other and just like figuring it all out from scratch without that anchoring.' And the I don't know thought experiment the idea people were talking about as like you know, I think we've sort of moved the goalposts. Given up the sort of like how do we determine if an AI is is sort of like AGI or if it's if it's able to fool a human? Like everything sort of shifts because things kind of became in a way that is different than we kind of thought they would. Um and so this discussion has been lost, but one of the things someone was saying as like a sort of marker would be right now we feed tons and tons of data—all of Wikipedia, all of Stack Overflow, all of the internet—as this training data. If instead you sort of went back and said let's take all of the books and newspapers and research up to, you know, sort of 1900, and then sort of ask questions like the phenomena were often known for sort of quantum physics or for general relativity, like everything was written where you could via sort of thought experiment come up with a lot of these ideas and theories and show that the math work, and later we sort of back explained things that were happening that either were misascribed or that we didn't know. So could you give one of these models like everything up to in the date, you know, we kind of argue with, but where you have some of these 0:24:11.972 --> 0:24:13.255 experimental results, you know, kind 0:24:14.082 --> 0:25:31.757 of lock it and ask it to thought experiment for sort of physics frameworks. And would you get quantum mechanics? Would you get Einstein's Theory of General Relativity? Would they be able to sort of self-reinforce, right? And you kind of need almost like what you're talking about where they would write their own papers, then they would sort of incorporate those learnings into other learnings—maybe an agentic AI where people are taking different things so they can kind of bounce off each other. Whatever the solution is. But I don't think we're there today for two reasons. Like it seems still all of these models require just gobs and gobs of training data, not very small corpuses. All of the written recorded data up to 1900 is a very tiny fraction of what goes into these today. And then second of all, I don't think there's this ability, and maybe we're getting there with what you're saying, but to kind of roll forward to let these things sort of have their own simulated society and like roll it forward and discover these things and learn them and understand them. Um And so of course, it's even further to say they could run their own experiments. You know, maybe that's—that's a lot harder and out of reach today. But what I'm describing just sort of via thought experiments and sort of deep introspective thinking, you know, and then later it gets proven out by experiment, right? But it feels like you could do that if you had the right architecture system training, but we're not there today. 0:25:32.584 --> 0:25:34.680 Yeah, yeah. I think you're right. 0:25:37.208 --> 0:27:35.060 Yeah, I—I think that with a lot of these things it'll end up like it really multiplies the hallucination problem. You know, it's like, 'Oh, um I see there's a kernel of truth here that one plus one equals three, but we got to the wrong conclusion.' Let's let's use that kernel of truth and like go a different route. You know. So so I think you know in general when you multiply errors—right? When you have a chain of things, you have to multiply the errors. And so um what we're missing, I think the big foundational thing we're missing is like we have kind of really strong models in our brain of physics that's how people can like hit a baseball out flying 100 miles an hour, right? It's like we have these like at a really low level. It's like primary visual cortex. We have something similar, I'm sure for audio and for physics—all these things, and they are evolved so the training data goes back millions of years, right? And so that's I think those kind of simulators, you know, we don't really know how to plug that into an AI. Like we're just starting with very basic things like, 'I will decide oh what's left here is like an algebra problem. I can use Wolfram Alpha.' You know, like but that's very early days and like using tools, and um you know and having sort of like simulators that aren't kind of uh—here's a very simple experiment. You can run to prove this to you if you take a picture of a maze, even a very simple maze that like any child could solve, and you tell an LLM to draw like an ASCII art version of the maze, like it won't be able to do that. It will completely fail. Um and and I think the reason for that is um is that you need this kind of simulator where you kind of as you're drawing you're then kind of going back and like you have this like concept of spatial that's kind of independent of 0:27:35.060 --> 0:27:41.280 the language model. Um now these are all things that are coming, but they're not there yet. Oh yeah, okay. 0:27:41.280 --> 0:29:39.899 That's really interesting. I have a—yeah, a couple follow-up thoughts, but we won't get stuck here forever or we'll never cover anything else. Um but the first one is, I think again not original to me. I think Jan Lokun talks about this in some of his interviews, but the sort of work model and the evolution of that. But I sort of have that thing in back of my head is the video generation. If you watch some of the video generation stuff rather than the LLM stuff, they end up needing an understanding like, you know, 'I have a picture of a guy holding a ball, and I ask you to generate a video of him dropping it.' Right? Well, how far should the ball drop each frame requires some understanding about physics, about where I'm wanting. What if I throw it up? Like hey, a pitcher is throwing a baseball. Are you going to draw in sort of a parabola, right? Like this up and down that physics is well described, but it doesn't actually—it has gobs of data about people throwing baseballs, but it's never been given the equation for throwing a baseball, you know, that it needs to follow a parabola half spin. You know, this kind of stuff is really interesting. Um but then more directly into to the DeepSeek thing, I guess, is I did see people posting some of the, you know, interactions with DeepSeek and what you're talking about—the infamous 'How many R's are in Strawberry?' So somebody asked DeepSeek, you know, how many how many R's are in Strawberry? And it gets stuck in this arguing with itself loop. Like it's like, 'Well, let's list out the letters: S, T, R—oh there's one. A, W.' It gets to the second R. 'R gets to the third R.' It's like, 'Uh-oh, there's a mistake. I know they're supposed to be two R's, but I just counted three. Uh wait a minute. Let's try it again.' And so like it goes again and it tries to go again and again. So like it keeps getting to this like you said, 'A kernel,' like it's doing the correct thing. It knows how to count, knows how is arguable, but it like it's doing counting which is correct, but then it also somehow like has this belief that the answer is two, which you know several LLMs seem to have gotten stuck with this for whatever reason—source grounded that Strawberry has two R's. And because 0:29:39.899 --> 0:30:02.314 of that, like it sort of goes, 'Wait a minute, I must have miscounted. I must have done an algorithm I wasn't supposed to do.' So then it tries, you know, variants on counting. Basically keeps coming up with the answer three and going, but that's wrong. And so it's just this very fascinating, like clearly the answer is three. You could ask anyone, but like it—it kind of can't do it. It's just it's just it's like oh, it just killed over and died. 0:30:03.124 --> 0:30:18.020 Yeah, yeah, totally. Yeah. I mean this is like it needs like a roving eye that can like rove like over all the different letters um and and it needs to be able to do that kind of explicitly instead of implicitly, and we're just not there yet. 0:30:18.020 --> 0:32:14.899 All right. Well, my next news article is not directly in this, but it is a bit on topic, I guess, and that is a list of computer science papers every developer should read. I'm apologizing; I won't attempt to pronounce the author's name, Dr. Well, I guess I'll try to say the last name: Do you are you able to say this Jason, or do you want me to go Milan? Milanovic? Okay, Doctor. Yeah, I was gonna say yeah, same thing, Dr. Milanovic. And he wrote this blog post, the links in the show notes. It's a lengthy link, and the papers here listed—I don't know—have strong opinions about these specific papers. But he kind of goes into a defense of why developers should read academic papers now. I guess this is going to expose a bit of a difference between Jason's practice of computer science and my practice of computer science, but in the kind of work that I do with largely embedded folks, you know, application programming, reading academic papers is—it doesn't happen. Most folks probably didn't do it in university; they don't do it now. It's just a foreign thing, and we've talked about this on the show from time to time, but I guess I will say that I think it can be a bit people overuse it, but a bit of a superpower if you're in a role where no one reads academic papers and you're willing to at least try to read academic papers when the need arises, going around and saying, 'Oh, there's this new research paper that says, you know, this,' and we're going to do it—probably not the right answer. But saying, 'Oh hey, people must have had this problem before. People must have, you know, worked on this issue,' and just doing a literature search is kind of what I would call it. It often turns up via academic papers or whatever, and you would be floored at the number of people who won't just search for something, 'Hey, I'm stuck on this hard problem; I need an 0:32:14.899 --> 0:33:41.571 optimization of this nature and they just won't sort of what we're just talking about ai maybe that'll be the future but they won't just shove it into google and try to like spend a few minutes sifting through this it's not going to give you a stack overflow answer but you might find a paper where someone has looked at that hey optimizations uh cash efficiency of various hash map styles whatever it might be and so i will agree with with the doctor here saying i guess we don't say it that way because that makes it sound like a medical doctor anyways but the author of this blog post uh and sort of say that i think you should try to read a few papers um i will be honest whenever i read a paper 90 of it is is it just like one year out the other like i either can't decipher what they're saying or can't understand it but even just reading a few of it looking at the pictures start to pick it up if you do one or two of those you will start to glean the gist and then either going and giving you the next thing to go search or writing it it really is for certain problems just such a like 10x ability to to kind of like execute on it and i'm not talking about papers that were written this year or last year in this journey i'm talking about like 1980s you know the you know 2000s that there were these papers written and you'll find them and it's just like wow that's yeah this is this is this is great this is exactly what i needed or gives somebody a you know like hey there's backup that i'm not crazy doing this like this is a well thought through thing and in some corners of the universe yeah totally um so 0:33:41.571 --> 0:33:53.140 yeah i mean i've read probably two to three papers a week for 25 years so i've read a lot of papers um uh i've actually read most of the papers you posted which i feel like is 0:33:53.502 --> 0:33:54.514 hey good for 0:33:54.514 --> 0:35:47.880 you considering like it's not these aren't really my fields per se uh actually this out of the tar pit sounds really interesting um uh folks can check all the show notes on the whatever app you're using to listen to the show the notes should be there or you can go to programmingthrowdown.com but yeah these papers are amazing like map produce big table kofka um i will say to read a paper well typically it's good to you know you read the abstract because it's the first thing um and then i go straight to um uh well i'll read the introduction if i'm not already motivated you know the introduction is there to kind of motivate the idea um depending on kind of like the first few lines introduction i may or may not skip it but i'll go to the background um and see like okay are is this standing on a foundation that i know like if i start reading the background it's like we all know about uyo saka method and i'm like uh-oh like okay i guess i have to go read that one um if it seems like i can like more or less understand the background i'll actually skip the main like contribution of the paper and go straight to the results and typically in the results there'll be some other method that i know and that will really help even just knowing what they compared with like i'll go straight to the table of the results it's like okay you know they compared with like several different actor critic policies so this must be another actor critic policy you know these kind of things um and then i'll look at the results and they'll also give me an idea like if this is like a 16 page paper and it's like 0.02 percent better than like some other method well you know it kind of tempers my expectations right um and then i'll go back and read the method so you kind of have to like go in that order i think 0:35:50.125 --> 0:36:58.220 it is a learned skill i mean i think like what jason's saying i mean to be fair i've read maybe three to five papers a year um that's i guess maybe i'm gonna shame myself but whatever uh it is it does take time and and it is not uh at least for me not just like i'll pick it up and do it there's a there's a technique to it and you do learn that they're written in a certain way and you know it sometimes does take two or three in the same sort of corner um to to kind of get get and then you go back and i think that's the other thing is it's not like a blog post you can kind of read it once and get it i think in my in my thing at least you you kind of skim it look at the pictures kind of get where you're going and you might go back and then as you sort of figured out you might go back again um and as silly as it sounds even in this day like printing it out and for whatever reason like actually having it in front of me this is literally the only thing i ever print out anymore but i will print out a paper if i'm really trying to get through it and like holding it and sort of like being able to see two pages at once or you know or mark on something or like you know circle something i want to come back to i never do that for anything else but when i do papers it it you know it helps me 0:36:58.220 --> 0:37:07.463 yeah i used to print it out and now i use my the e-ink tablet um and then you could just draw with the with the pencil um man i mean 0:37:08.965 --> 0:37:38.648 this is okay we're going into into a bunch of side talks here but it'd be amazing if someone made why hasn't someone just made a e-ink tablet that's the size of a u.s sheet of paper like why is that so hard money i i will pay for that uh you'll have an you have one person who will buy that um maybe there's some technical thing too i don't know like maybe like making an e-ink tablet that's that big is probably yeah i 0:37:39.593 --> 0:38:08.972 would also i mean i guess like once you add in the other stuff maybe it's like difficult to fit into your backpack or something like i you know and so therefore people i would assume most people buying them are reading fiction you know novels and those fit really well on on a relatively small screen and just right data just flow the words just flow around right i would assume that's not whatever 95 of the the hours spent on e-ink tablets are spent doing that and so i think today they sort of 0:38:08.972 --> 0:38:36.850 cluster around the size of a paperback yeah i think that makes a lot of sense there's an amazing thing uh if folks i don't know if this is on iphone it probably is but if you have adobe acrobat on android they have this thing called liquid mode where it takes like the paper and turns it into kind of like that format where you can scale the text and read as you know what i mean it basically takes the pdf and turns it into a format that's awesome for mobile and it usually doesn't mess up the diagrams and all that that 0:38:38.318 --> 0:38:48.460 feels like a low-hanging ai target right there which is like take this pdf and make it like you know whatever i don't know what they call that yeah flowable liquid like yeah yeah yeah 0:38:48.460 --> 0:38:49.152 but like good 0:38:49.320 --> 0:38:55.969 I've seen so many bad ones where people it just like it's broken and doesn't work right. Yeah. 0:38:55.969 --> 0:40:45.032 Yeah, same. Um all right, my article is my second is NVIDIA Cosmos, an AI platform to change the future of robots and cars, wins Best of CES. So um that's a pretty bold subtitle there. Um but what is that anyways? The thing where like you kind of put a title in the middle of the title? But um Cosmos is really interesting, and it ties into what Patrick was saying. It is a world foundation model, and so the idea is people have—people at NVIDIA have taken, you know, like videos and tried to predict the future of the video, and they've done other things like that. What you end up with is something that the world foundation model is trying to be, kind of like a physics simulator um to try to give you like an idea of what's going to happen in the future. And then what you can do is take embeddings, which are like, you know, compressed vector representations of images. You can take like the embeddings of the future and feed them into a model. And so the idea is if you have a robot, the robot might know that like, hey, you know, this future looks pretty weird, like this future looks like it's the kind of future of a robot that's falling down the stairs. And so I'm just going to stop because I don't like where this is going right. Um and so the idea is you can combine these world models with like some like inertial data and all of that to like make robotics a lot easier. Um, you know, nothing practical has come out of it. Um but I think the—I think it's something really interesting, and I think that it's—it's going, it's skating where the puck is going. I just don't know what's going. 0:40:45.032 --> 0:40:45.859 To happen there? There? 0:40:48.154 --> 0:41:19.542 There were a lot of really interesting ones we didn't put them in here, but NVIDIA announcements. So I mean, I think for sake of time, we'll just skip them, but if you haven't gone and looked at the various things that NVIDIA is doing and introduced, and you're vaguely interested in AI and how the underlying hardware is, is, you know, shifting, I would definitely go there. There's some like what do they call those? Like Supercuts where they cut up the sort of CEO's speech and sort of like show you the highlights, you know? I've ever heard of that. I think so instead of, you know, watching an hour keynote or whatever, you watch, you know, five minutes. I did. 0:41:19.609 --> 0:41:26.545 That on the one from Google last year, and it was just Sundar Pichai saying 'Generative AI' over and over again. 0:41:29.464 --> 0:41:34.037 But then you realized it was a generative AI video. It wasn't actually him. 0:41:36.349 --> 0:41:37.969 All right, time for book of. 0:41:39.235 --> 0:41:41.783 The show? Book of the show, Patrick. What's your? 0:41:42.306 --> 0:43:38.980 Book? Okay, completely out of character for me, but whatever. And it ties back a bit to what I was saying, trying to kind of read sort of more academic approach and broaden my horizons. I'll call it, but this is Alice's Adventures in a Differentiable Wonderland, which I haven't made it all the way through yet. It's a little slow going for me. Um, but it is—I think so far it has been like a little bit of an in-between machine learning book. So you can get the sort of like elementary takes, which I feel like I kind of get—I think a lot of people are out there sort of understand at a, you know, high conceptual 30,000-foot level what's going on with, you know, the machine learning and more than just, you know, 'Oh, it's an LLM.' Like, okay. Well, there's, you know, convolutional layers, and there's, you know, there's this—you maybe watched, you know, The Three Blue One Brown always mess up his name video about LLMs and Transformers and how they work, right? Like, you sort of understand some of the nuts and bolts from I'll call it like the top down to about the middle. Um, and you know, you may even could run Python scripts to train your own little little models stuff stuff I've done. Um but then if you flip it around and go sort of like bottom up, you know Jason was describing this earlier, like how do you vectorize this thing I want to do and build models? Like, how do you start to piece together these things? Why do they work? What is the expectation? Um And then the thing that for a long time has eluded me and is maybe clued in by the title here is these, you know, ways of combining the math operation so that they're differentiable. And what does that mean? And that's basically how you get the training—how you take an effectively random set of numbers and guide them into weights that actually evoke a certain, you know, output. I'm probably using the wrong words here, but that's because I'm not trained. And so this book seems to be a pretty middle of the road just as I'm trying to get through it. It doesn't—not use any formulas, but it's also trying to explain what they're doing and where they're going if you just jump into one of the papers, you know, Transformers are all you. 0:43:38.980 --> 0:44:29.909 Need or whatever? Oh no, that's not the name anyways. Um the—you know, you get this sort of like immediate dumping into like way too much context that I don't have. Um and so this book, it's a book, but it's free. It's, you know, published in a PDF. I think you can go buy a physical copy as well, but I have a link in the show notes to where you can find it. Sort of kind of takes you through the various things—I'll say sort of bottoms up in this sort of middle of the road, probably someone who has a math background who knows computer science but isn't a machine learning person and is trying to learn more than just 'Here's the PyTorch commands you run,' but 'Here's what the PyTorch commands are doing or why they're doing it.' What is the motivation here? Which is a big gap in where I am. So, I—I'll take that maybe right or wrong assumption that if I'm struggling with that, maybe other people are as well. And since someone wrote a book, clearly, I guess yeah, this is awesome motivation there. So yeah. 0:44:31.630 --> 0:44:34.971 This is so cool. I mean, this is a what? A 300-page book totally? 0:44:37.840 --> 0:44:41.502 For free? Yeah, yeah, yeah. This is amazing. Yeah, highly recommend it. Um very cool. Okay, you 0:44:43.054 --> 0:44:46.092 got me hooked on this. I'm gonna have to read this now. He's gonna 0:44:46.500 --> 0:44:49.973 Ask the AI can you read it to me please? Yeah, I'm gonna have an AI read this to 0:44:50.597 --> 0:45:05.751 me now. Um, okay, My Book of the Show is a movie. Um, this is also totally out of left field. I mean, you thought yours was out of left field? Wait till you hear this: It's a Beautiful Day in the Neighborhood, the movie about Mr. Rogers. And uh um yeah, it's uh 0:45:07.574 --> 0:46:27.038 it's a really deep movie. I mean, I actually thought it was gonna be a biography, but it's not. It's a fiction movie that's based on a true story of a reporter. The reporter—the reporter was basically trying to write a smear piece about Mr. Rogers, you know? One of these like muckraker kind of reporters, and he follows Mr. Rogers around. Mr. Rogers like actually like uh um gave him permission to like, you know, follow him around and interview him and all this. And uh and he ends up like uh, you know, like changing his life and everything. It's like it's a really, really touching movie. Um and uh yeah, I highly recommend it. I think is pretty wild, actually totally blew my mind. I'm generally more into like uh, you know, documentaries. Like I saw the um the one about Queen, the documentary about the band Queen. Um and I saw another one so so oh, the one about Theranos. Uh I think it's called The Dropout. Um I saw that, and so I went to this one just expecting another documentary, but my mind was like completely blown by uh by this movie. So it's on it's on Hulu, Netflix, all that good stuff. Um highly recommend you check it out. 0:46:28.810 --> 0:46:51.456 I never watched documentary. I guess I should. If it says 'documentary,' I'm like—I'm basically like instant out. Like I just for whatever reason, I don't know, I should probably watch them more instead of just like whatever. I I don't know. It's just like for me, I always feel like I'm gonna have to do a lot of follow-up work after this documentary to like like understand the other side or whatever because it always is sort of a hot take, right? And so um yeah. 0:46:51.659 --> 0:47:23.535 I mean, you know, I think the ones that I've watched have been maybe not too controversial. I mean, like the Queen one, for example. It's just really interesting like how they came up with the songs they did. And so it's really like I think it's a way for you to have been somewhere that you weren't, you know? Like I wasn't in the studio when they were recording their songs, but somebody was, and they provided insight, and uh you get you get that kind of experience, I guess, like through through that. So I will say 0:47:24.413 --> 0:47:29.999 The most impressive thing is that you said the title without any hint of sing-song in your voice. Oh. 0:47:32.969 --> 0:47:39.837 Man, it's uh it's definitely a throwback to watch because did you watch Mr. Rogers growing up? Yeah, me too. It was great. Um okay. 0:47:41.777 --> 0:47:53.978 Yeah, all right. What is your Tool of the Show? Tool of the Show? It's not a tool; it is a game, so it will waste your time. I would not recommend you to play this game. Um you wasted so much of my time on. 0:47:53.978 --> 0:47:59.614 Bellachro, by the way. I finally—I beat uh I think all the chips or I'm on the last ship? Oh, you're better. 0:47:59.614 --> 0:48:03.242 than me. I got so good and then I the red 0:48:03.732 --> 0:48:10.532 and black deck is so stacked. I mean, you just have flushes whenever you want. Yeah? Okay, people are like what? 0:48:10.634 --> 0:48:17.974 Um, don't download that. Definitely don't. Yeah, save your time. Yeah, I have I have I have made it so that I'm only allowed to. 0:48:17.974 --> 0:48:20.775 Play that game when I'm on an airplane. That's smart. So otherwise it's 0:48:21.332 --> 0:49:03.030 It's bad for me. Okay, this one's not quite so bad. This is actually a relatively really short game. I guess they call these like incremental games. But someone threw this out. The other one that really killed a ton of my time was Vampire Survivors. There's like it must be a whole thing. I don't want to know. Please don't send me emails, please. But like three-dollar Steam games or whatever. So this is like a three-dollar Steam game supposed to be short, just goofy. But it's Digsium, and on a computer, you know, use a mouse, keyboard. On my Steam Deck, it actually works really great. A lot of the, you know, sort of like clicker games kind of don't work great on your on your, you know, Steam Deck if you have one. But this one, I did this, I've been playing it, actually works really great. Do. 0:49:03.030 --> 0:49:04.397 You have a touchscreen on the 0:49:04.397 --> 0:50:27.439 Steam Deck. Yes. Okay, and that's what makes this game particularly so problematic for me, which is the idea is you're kind of like I don't even know the story. It's like you're running a museum, and you're going out to these various fields and digging up relics to put in your museum, and you get money from the visitors who come to [the] museum. It's all in not even 8-bit graphics like Commodore 64 kind of like like graphics. But you have the various places you go to harvest relics, increase in size and increase in the difficulty of mining up the dirt, and you'll have only so much energy to kind of mine it up. So if you're not able to sort of mine up all the blocks, you could not get all the relics that were buried in that particular round. But everything is like very quick. You know, it's maybe five seconds to sort of tap, tap, tap, tap or click, click, click, you know, clear the field whatever run out of energy, you go again. You go again, and then you're also building up money in the background, and the classic I guess like incremental thing. You can buy upgrades, and then you can reboot yourself. But some of the upgrades are permanent, and you get for how high you get, you get sort of permanent upgrade points that you can use before you start again. And it's just like I know it's such a distillation of some of these incremental games either for me they just like have this plateau. You get too and oh, that's the that's the grind, and it's amazing. Yeah, I hate the grind. Yes, I got. 0:50:27.439 --> 0:50:33.970 That at work. I just wanted to be like, uh, I was gonna say something. It's not. I guess I won't say what I was gonna. 0:50:33.970 --> 0:50:45.310 It's just very like I just want to be light and very, you know, just a soda junk food kind of like get me through it. I don't need it to be long-lasting. I don't need it to be thoughtful. I just want to go full steam ahead and feel like I'm powerful. Yeah, totally. 0:50:47.031 --> 0:51:28.240 Yeah, this sounds awesome. This is right up my alley. So yeah, Digsium. I don't know if we said it, but yeah, it's available on Steam, and I don't know where else. I tried looking. It didn't look like there was other places, but definitely it's available there. It's like three bucks maybe. It goes on sale sometimes. Don't know. I just paid three dollars, and don't do that. It's just real. You'll regret it. You'll spend it. It's actually not that long, but you know, you'll just burn a bunch of time, and you'll be like what is it? There is an ending, right? I believe so. I think I'm pretty close. People told me how many hours like when I—that's how I found it, you know, via some random link I saw somewhere. And I think I've about played that hours, and so I do get the feeling that I'm towards the end, but I don't actually 0:51:28.240 --> 0:52:42.102 Know. So got it. Cool. Um my tool of the show is a Python library called SQLiteDict, which is a weird name. It's, you know, based on the dictionary object in Python. So everyone doing Python has thought about this where you have Python, you know, it's everything's a dictionary. Classes are dictionaries, you know, all the objects are dictionaries, data classes are dictionaries. So, dictionary is like a really foundational object, right? It's equivalent to like Object in JavaScript, right? And so you're constantly passing these around. You're setting keys, you're sorting them, etc., etc. And then you always ask yourself, well I have this dictionary and maybe it takes like three hours to compute all the keys for this dictionary, and when I'm done, I want it like on disk, right? Now if it's the kind of thing where it's a batch job, like you process this dictionary and then now you just need it on disk. You can turn any dictionary into a JSON object, a JSON string, and then write that string to disk pretty easy. And there's a similar function for going the other direction. 0:52:44.430 --> 0:53:28.491 There's a library called Pickle where you can pickle any Python object to disk and then unpickle it later. That's great. The challenge becomes what happens when you need kind of like a backup of a dictionary on disk but it's like incremental, you know? It's not practical anytime this dictionary changes for you to have to save the whole thing to disk, right? So what you really need is you start getting into the realm of like needing a database, right? Like at some point, like you want to use Redis for this or something, right? But that's pretty heavyweight. You need to stand up a database and you still have on a port, use TCP to connect to it. It's just like a lot of work, right? You know, you could do it with Docker and stuff, but it's time 0:53:30.735 --> 0:55:30.059 Consuming. So ideally you'd have just like a file on your computer that has the dictionary on it, and you could just access that file the same way you access a regular Python dictionary. Like even it would be even better if like you didn't have to change your code. You could just or change a lot of your code. You could just replace like if I have a dictionary and I'm stuffing a bunch of data into it, I could just replace that dictionary object with some other object and get a dictionary that's persisted on disk, right? And so there's something built into Python called Shelve which does this, but Shelve sucks. So there's a bunch of reasons why Shelve is terrible. The biggest reason is if you're on Mac or Windows—actually it might just be Windows now—but there's definitely a time where if you're on Mac or Windows, you got one database format called DBM. It's literally called DBM. And if you're on Linux, you get this other database format called DBM. So you can be on Linux using Shelve having a great time. Things are super fast. You run the same code on Mac or Windows and it's like 300 times slower, and then you're like, well what happened here? If you could take the time to dive into it, you find out oh, it's because instead of using DBM I'm using DBM, right? So Shelve is a mess. There's also data corruption. So it's not that often, but sometimes if you use Shelve when you go to open or when you when you go to access a key, you'll get this error and I don't remember what it is off the top of my head, but effectively the database is corrupt and your data is gone, right? So not a big fan of Shelve. Now SQLite is amazing. SQLite is freaking awesome. Everyone uses it. Your iPhone probably has like 30 SQLite databases running on it. You know that's at the OS level. Everyone's 0:55:30.666 --> 0:56:41.600 Using SQLite. So SQLiteDict basically gives you the API of Shelve but under the hood is SQLite, and you know it's not going to be—it's one of these things where like it's probably a little slower than Shelve, maybe it's even faster than Shelve. I don't know, probably a little slower than Shelve on Unix, but on Mac and Windows it's the same speed as as the Unix Shelve. So so you don't have to worry about that. You can access the database with SQLite, you know, in another language or stuff. It's going to be whatever you put in. So if you're if you're pickling things and putting them in, then you're going to get just random binary garbage, right? But if you put in let's say a number into SQLiteDict, then you'll see a number in the database, which means you can access it from other languages and stuff. So um so it's an awesome library. If you're using Shelve, stop using it. Use this thing instead. If you're not using any of these, it's another sort of tool in the toolbox that I go to pretty often. 0:56:44.528 --> 0:57:03.090 Yeah, I've heard so many issues with Pickle. So yeah, I think putting it—even if it's tedious to get the stuff back out—I guess like having something that's SQLite that other, you know, programming languages whatever could access other programs like feels like a win. Yeah, I 0:57:04.964 --> 0:57:43.489 Mean by default, Shelve and SQLite will both what they'll do is they'll look at the data type of what you're trying to put into the dictionary, and if it's a primitive data type, it'll just store it. If you try to put like a Python class into the dictionary, both of them will pickle it. And I totally agree. I think instead of trying to throw Python classes into either of these, you should like convert to JSON or something like that. Um, Pickle is good if you're in a hurry, but yeah, there are security issues. You really have to trust the person who pickled the object. And then also there's weirdness around like if you change the class and all of that. Very cool. Well, guess 0:57:48.923 --> 0:57:52.787 What time it is? It is Project Planning and Management Time. I was gonna 0:57:53.580 --> 0:57:55.454 Say thank the Patreon times, but yes, yes. 0:57:56.145 --> 0:57:57.580 Oh, thank you. That's all right. 0:58:00.010 --> 0:59:49.849 Thank, thank you to all our patrons, but yes, I was shout out it is actually time to talk about Project Planning and then we realized maybe a little bit of Management. I'll give the disclosure: uh, this is not advice; this is just entertainment. No, I'm just kidding. Um, we are—we are software engineers. Both, we are not Product Managers. We are not Project Managers. We are not like this is just from a sweet perspective. It's a very casual conversation. So um trying to give people this sort of vibe, I guess this is what the cool kids seem to say the vibe of what what this Project Planning is all about, and there are things to think about and the you know stuff to go on. And uh yeah, we're gonna jump right in to start motivating the reason why. Although, you know, maybe if you took a second, you'd probably get most of these, but the reason to plan your projects I guess is to think about the alternative if you don't have a plan. Um, but no, if you work in a company or on a medium or large size project—anything that's going to take more than just a couple hours—really having a plan is important. And you know, I think there's a bunch of reasons why, but some of it is that having an explicit process where you kind of go and plan what you're doing allows other people to give feedback on the plan, right? Make sure your co-workers are aware. Make sure everybody's sort of gathered together um and to figure out hey, what parts of the plan are going to be risky? You know, is this—are we desert? You know, hey, we're going to do this by this date, but there's a holiday break here. So if everybody goes on holiday break, then you know we're not going to get our plan done, or we're going to have to try this new technology or this new algorithm we don't know yet how to do. And making sure that sometimes you have blind spots and that people identify, hey, what are you sort of not counting seriously enough. Yeah. 0:59:51.047 --> 1:00:25.624 Yeah, totally agree. Um I think um another really important part of Project Planning is is headcount planning. And this is something where I've seen this go horribly wrong even at big companies. I—I definitely am not going to give any personal stories although I'm thinking a bunch in my head, but I've seen situations where people who asked for, you know, managers who asked for more headcount just got it just for asking. And it's like if you didn't ask, you didn't get it. Or I've seen cases where um it 1:00:27.717 --> 1:02:20.339 was pretty subjective. It's like, oh, your project is kind of cool. Let's like put more people on it, right? But but ultimately, like the principled objective way to do this is you should plan like what is the expected amount of work that our team is going to accomplish and when you see a team—this can be thousands of people. It could be four people that still works. Um and then come up with like a distribution um and maybe like, you know, this is like science brain of me, but come up with like a distribution where you know if you're like one sigma below the distribution, then like that was slightly below expectations. You know, if I'm one sigma above the distribution, that's above expectations. And then that's really important for so many reasons. One is, you know, at the end of the year, you need to figure out did my team perform at my expectations or not. And you can't do that if you don't set expectations right. But but the second thing is um is you get to see what you didn't do at the end of the day. Like adding more people allows you to do things you couldn't do, and you have to know what that is before you can make a case for more people. So you at the end of the—at the end of the year we could say, you know, here's a list of ideas we had. Look, this idea we didn't do it, and someone else did it, and they published a paper, and that paper was amazing. So like that's a miss, you know. Um or you know, here we had a list of ideas we didn't get to any of them, but it turns out they were all kind of like maybe not that good. Like like it was all below the fold, and we're actually not really missing out on not hiring people. Um so so I think you know headcount planning is really kind of like dovetails nicely 1:02:22.399 --> 1:04:20.120 into this thinking about, you know, expanding from headcount—not just new hires or, you know, allocation of people, but also making sure if there's other teams upstream or downstream of your work and that there's a dependency. Either like hey, we are going to use this thing that isn't being used today, or we're going to assume this thing another team is creating is going to be available. Is very important to communicate so much. I mean, so obvious, but in life how many things could be solved by just, you know, communication? And so making sure that you communicate expectations around headcount, expectations around—we're going to talk about schedule—but also, you know, thinking through what are the dependencies between modules and sort of the APIs that I want to call or depend on downstream upstream if you are going to use a third-party service pricing, right? Like am I going to be sensitive to if the prices go up on these function calls down for me or API calls? You know, is that going to be a problem? And sort of making sure you have that. And then when we start to talk about schedule and thinking about, you know, depending on stuff that that's not yet available or how much it's going to take for you to get your sort of modules or work, you know, implemented, work items implemented, there's also identifying the sort of critical path. What both from terms of the data flow but also in terms of from your schedule standpoint? What is the sort of thing that if you walk through the schedule that is these are the items and some items are happening in parallel, but these are items we expect to take longest. And if we sort of line them all up, this is the sort of length, and it's the set of things that are we are most sensitive to in the final schedule, right? These are the set of things where if we get behind or we aren't able to handle the risks or you know something slips, then our total schedule slip. And that's very important to—and maybe everything is on that path, that's entirely possible, but if there are things off of that path, I would argue actually 1:04:20.120 --> 1:05:14.237 knowing that they're off of the path is as important as knowing what's on the path because when it's off the path you have a little bit more flexibility. You have a little bit more sort of dynamicism. You know, that Jason was mentioning thinking about things as a distribution. If you start to get involved and you find out you're needing more resources from, you know, then you the sort of median was, then you know, you can kind of pull them from things that were less important, maybe, right? And if you sort of, you know, kind of understand which things have which priority, there's that classic I guess kind of joke: everything is the most important. But if everything's the most important, nothing's important, right? And every company seems to fail at this. This is fine. It's just like a difficult thing, but you can't say everything is equally important because you really do need to know if something's going to slip where is it going to come from? Where are we going to pull things? And having that thought through and communicated is very vital. Yeah. Yeah. 1:05:15.402 --> 1:06:26.159 Totally. One thing that I, you know, if I could impart any wisdom at all, it would be I always underestimate the ripple effects from decisions that I make in the workplace, and maybe just universally. Um I think it's just really hard to measure multi-order effects because usually what you're trying to do is you're you have your own plan like I want to try and build this, and then I want to try and write this paper, and then that paper is like a stepping stone to this other project. And then I have this idea—maybe it works, maybe it doesn't work. And so you're like you're doing what DeepSeek is doing, right? And it's hard to do that and think about like all the ripple effects of like okay, if we're late on this project, that causes other team to do this, which causes other team to do that. Um Or even like an even better example is like given that we're late on this project, if you communicated that now versus next week, that actually can result in totally different outcomes, even though it was the same—that you know, the same effect. The same initial effect could have two totally different outcomes depending on when it was communicated. And 1:06:28.268 --> 1:06:30.310 These are really difficult things to measure. 1:06:31.187 --> 1:07:00.972 Yeah, I think the interconnectivity—we can talk about the mythical man-month. We can talk about the classic, you know, hey, if I just give you another person, can you go twice as fast? Um, but I mean a lot of that comes down to what you're saying is like it is very difficult for human software engineers to understand just how much impact one decision, one choice, one slip—one whatever—can have. And the sort of compounding you get from that. Yeah, yeah, totally. Um, it's definitely like a bullwhip. 1:07:00.972 --> 1:07:24.023 effect where, you know, especially if you're at like the—if you're doing research, you're kind of like at the beginning of the bullwhip. And so if you kind of um make a bad decision or you don't try to advance something or whatever, then that causes like these massive swings downstream. So that's actually great use of that term. Can can you maybe—or again—but 1:07:24.023 --> 1:07:25.795 Can you kind of explain what the 1:07:25.795 --> 1:08:36.180 bullwhip effect is? Yeah. So I believe this came out of like supply chain management, but the idea is—you know, let's say you end up with slightly less grain this season. Well, then people will start—um, you know, people downstream of that will start to experience shortages and they'll start to, you know, buy in volume and do different things to like handle those shortages. And so then some of those processes that they start implementing, like buying extra surplus, doing all these extra things, they'll continue next year even when you don't have a shortage. So they're kind of like because they're downstream of you, they're reacting to you, and their reactions aren't very quick. And so they end up like buying more than they need. And then next year you have a surplus, and they're still buying more than they need. And so now they have to like double correct. Um, maybe, you know, yeah. And so and so so it just has this compounding effect, right? And that's just one step. So imagine if there's a supply chain with like 20 steps, the person 20 steps down just experiencing like wild volatility because of all of these kind of like mispredictions. Did I get that right? Yeah, yeah. 1:08:37.581 --> 1:10:37.460 Yeah. So if you think about like software that is finally delivered or a product that ends up on a shelf in a store and you think about all the unique steps that it takes to get there, and especially when they're sort of like a continuous—continuous process, which we could kind of talk about software being viewed as a sort of one-shot or continuous. But yeah, it's just sort of like as you add these dependency things, small perturbations in the beginning, if you sort of like if everybody sort of stacks it up, they just get compounding. And so yeah, I think that can definitely be there. And when you talk about—and I think we were gonna maybe talk about the end, but you know, I think it makes sense now is if everybody sort of tries to and you were sort of saying like overbuy or have a buffer or allocate a little extra, right? And so if you take all of the teams and all of the steps and everybody sort of buffers up so that, you know, they're not the ones called out, you end up with something that's really like four or five times longer or more expensive than it needs to be because you end up with all this hidden sort of like padding that occurs all over the place. And so you'll hear that that like upper management will sometimes say, 'You know, I whatever they tell me, I cut it in half,' or 'I cut it by 20 percent.' Or and this is what they're talking about. And it happens with budgets. It happens with headcount. It happens with schedule where everybody sort of pads, and then the people over them pad, and then the people over them pad, and you end up with like ridiculous amounts of padding. And so there is this element when you're doing project planning of—of trust, right? That you need to be actually honest. And there's already difficult enough to even estimate how long somebody's going to take if you're not if you're sort of also playing this game of politicizing it. Now does it actually happen? No. I think everybody politicizes it to some extent, sure, but you know the less of it that you can have, the better. Which is, hey, you're brutally honest with your manager, they're brutally honest with their manager, and you have an actual view up and down the 1:10:37.460 --> 1:11:13.439 chain of what's happening. And just like we talked about the bullwhip effect, unfortunately it only takes one of those links or one of those layers to be—I don't want to say like a bad actor, but to just try to make sure like I'm definitely not going to be the one who is called out. And and they sort of add something how at the top are you supposed to know, you know, how honest all of the estimates, all of the work is? And so yeah, the bullwhip effect can be unintentional, but I will say it also works kind of in this sort of like intentional safety that people add sort of aggravates the effect even more. Yeah. 1:11:14.181 --> 1:13:05.742 Yeah, totally right. I think that—you know, that's a good point. I mean, sort of a meta point here is like, you know, I notice I see people who transition from an individual contributor to a manager and they start off with like, 'Okay, how can everything be perfect? Like, how do I make everybody's task done? What do I have to do to kind of like rewrite history so that like everything we set out to do in March we now have accomplished next year?' And everyone on my team is perfect. They all need to get promoted all the time. And then over time you start—you start like calibrating and you say, 'Okay, like the goal actually isn't to like play the organization system as if it's Monopoly or something, right? The goal is to like have the most accurate assessment of the situation.' Um, and so so I mean, you know, the earlier example of like put things on your plan that you feel unlikely that you're going to accomplish. Um, and then when you don't accomplish those, that's—that's normal. Like, you should you should set your team up so that there's like an upper and a lower bound that that they can achieve. And you know statistically, you will eventually have a quarter where your team massively underperforms. Like that's that's how it's going to work statistically. You're going to draw that short straw, you know, so many times in your life, right? And so and so—I think project planning also helps keep people from panicking. It's like, 'You know, yeah, we missed this quarter, but you know, we didn't miss the past three quarters,' and so this is natural, and so we pick it up next. 1:13:06.737 --> 1:14:51.140 quarter. Yeah. I mean, I think that's a great transition to goal setting, but I think it becomes almost a parody, a joke when you see people who set goals and then they always get exactly the goal or like exactly the goal plus some little fudge factor. And it's always slightly more. Like, 'I always accomplished 1.03 of whatever my metric was.' And it's like if you never come under your goal to your point, you actually don't have the distribution set correctly. Like it should be 50/50—sometimes you're over and sometimes you're under. And that's life, and that actually means you have the sort of estimates well set. And if you don't, you're just sort of like goofing around, gaming the system under whatever. You know, like at least that's sort of my opinion. I mean, maybe you're in some, you know, hockey stick growth thing and it actually is just really, really difficult to predict that can happen. But setting goals is very critical, and I think also being flexible and not saying which we've—can sometimes happen. The plan isn't there to say, 'Oh, we've ruined the beautiful plan.' You know, plans never encounter the first, you know, never survive the first encounter with, you know, real life or the enemy, or you know, use whatever your favorite, you know, quote there. I'm butchering, but I think the plan is there to help give people the feeling that like, 'No, it's okay. We we have these things written down. We have them documented. We know what to go adjust, who to go tell, what is going to be impacted.' Not, 'Oh my gosh, you've ruined all of our work!' And I very dislike working in situations where people are like, 'We made the plan and now the plan is ruined.' And it's like, 'Well then there's probably not a good plan.' Like the purpose of the plan is to help us, not hurt us. But on setting goals, you know, there's a mnemonic that—that you'll hear which is set your goals to be 1:14:51.140 --> 1:15:04.204 SMART. Yeah, totally. Um my favorite um my favorite my favorite quote is the Mike Tyson one where he says, 'Everyone has a plan until they get punched in the' 1:15:04.204 --> 1:15:05.166 face.' Oh there. 1:15:05.250 --> 1:15:24.820 You go. Yeah. Um so SMART goals stands for that—the the acronym SMART is Specific, Measurable, Achievable, Relevant, and Time-bound. And so if you've ever taken any kind of project management course in any of your work, you'll you'll hear this come up. 1:15:24.820 --> 1:15:53.091 Yeah, I think it's one of those things where again maybe it feels a bit obvious. It's actually really useful if you have set goals and you actually ask yourself personal goals, project plan goals, but you know, have you done all of these things? I often find that—oh yeah, despite the fact that when you say it this way, it's pretty obvious—yeah, I often screw up at least two of them. I'll say it's very rare that without very intentional work that you meet all of these. Yep. And I 1:15:53.462 --> 1:17:49.520 I think the two most important are the measurable and the time-bound. So time-bound is almost a no-brainer, right? I mean that one—I think everyone, everyone's like, 'Well, when's Task X going to be done?' But the measurable one, that is the one where I often catch people and have to iterate. So someone will say, for example, like build the Foo widget, right? And so we all have to agree that just building the Foo widget is a good thing, you know, and that happens sometimes, right? But generally, I'll push back and say, 'Okay, like well, what what is the point of the Foo widget?' You know, like here's a company, you know KPIs, here's a company, you know metrics. Do we need to create a new metric? Like is this a new dimension that we need to be looking at? And and you know it might be that, you know, we're going to create the Foo thing—the Foo widget—and then we're going to have like a list of things we're going to measure on the Foo widget. And so and so, then you can like distill down to the goal. So if the if you're creating the Foo widget because you have a new sort of segment of customers, right? What are those customers going to do? Well, they're going to buy the Foo widget. How much does a Foo widget cost? It costs twenty dollars. Okay. So so we have a goal that, you know, we're going to we're going to make like a hundred thousand dollars in the first three months selling this Foo widget that we haven't built yet. Like like there should be some way to get it down to a measurable thing where you're not measuring like, 'Yeah, the thing is built.' That's—that's one. Or the thing is not built. That's zero.' You know, like getting a proper measurement that like 1:17:49.520 --> 1:19:04.380 connects to the company's objective. And all of that I think is really important. Um so you know sometimes you might build things, you know, you see this a lot of research, like you build things that are trying to kind of push the industry in a new direction. And even in these cases there should be some way to measure, even if it's an internal benchmark or something like that. Maybe instead of building the Foo widget what you need to build is the benchmark that you're expecting the Foo widget to outperform in. And we need to actually build that thing because otherwise someone will build the Foo widget, they'll be super proud of it and then they're wondering, 'Like why their performance review wasn't good?' You know? So it's like well, you have to build the thing. Sometimes you have to build the thing that motivates or explains the efficacy of the actual thing. And so so the measurable one is super key. You know, the other ones I feel are, you know, like specific—it's kind of like okay, everyone should always be specific, achievable. It's like how could you have a non-achievable goal? But but the measurable and the time-bound seem the ones that really stick with me. 1:19:04.380 --> 1:19:07.160 You've never had your manager set a non-achievable goal? 1:19:11.507 --> 1:19:21.953 Um that's oh man. So if so, you've been incredibly lucky. So I feel like I have to be diplomatic. I've definitely had managers set non-achievable goals, but not once. 1:19:23.624 --> 1:20:09.962 Have I ever said, 'This goal is unachievable,' and they've said, 'Okay, we'll change it?' So so it seems almost like—I do think that achievable goals are more achievable, but I don't think that knowing that is going to help you get an impossible goal and make it achievable. Like I've seen one time they wanted this metric to go up by some ungodly amount, and I basically was saying, 'Like, you know, it's not—it's not really going to happen.' And what ended up happening was the metric went up by a nominal amount, and then everyone was kind of meh about it. So the fact that the goal is unachievable, I don't know if it really changed anything. Well. 1:20:12.173 --> 1:20:42.520 Then I would, but then I think that's the point of the sort of letters working together. Like if if it being unachievable didn't matter—I mean, I think that's because it wasn't a good like you could have had a better goal that would have helped people rally around it or would have done something slightly different and maybe it could have been at least 2x of whatever the nominal increase is. Maybe not a thousand x, sure, but like it being more achievable could have been potentially more motivating to people because there there could have been a path to it. 1:20:42.520 --> 1:21:51.128 Yeah, I mean this is tough, right? So if if someone sets a really ambitious goal that gets people excited—you know, it gets their managers excited, it gets your team really excited. And then if you are directionally accurate and you do make an improvement but it wasn't like the enormous outstanding improvement, it's really tough. I mean, like your instinct is to say, 'Well, I expected let's say a 2 to 4x improvement.' My director promised a 400x improvement. We got a 3x improvement. Like your instinct is to say, 'Like yeah, that was like a giant mess up.' Like the head honcho should have just predicted a 4x improvement. But then it's like when I look back on it, it's like that that like fake number caused so much excitement. It's like it's kind of like a counter to it if they get actually worked out. Um I don't know, this gets really complicated, but but I guess if it's measurable then maybe that just makes it at least somewhat achievable. I don't know. All right. 1:21:52.326 --> 1:23:49.160 Let's keep going. I think you know talking about other things about that—that work here we're talking about identifying the critical path. You'll hear this, and I think the concept is really solid. I think people can often, and you know as we're kind of saying here, I think people can take things for unintended reasons or take them too far. But one of the things is Gantt charts. So if you sort of represent each of the tasks as a sort of—I guess like a rectangle, a bubble—and you sort of schedule them over time and you sort of draw interdependencies, it kind of sounds stupid, but you often will discover like we have work scheduled to begin before the thing that it is needing at input finishes, right? Oh well, that that is not going to work. And if you just sort of say, 'Hey, you know, to do B, we need A is going to finish in Q1. We're going to start B in, you know, February.' It's like, 'Well, wait a minute. Like are you going to be able to do work?' And then you would say, 'Well, you know, we can start it.' Well, then it's really probably two tasks: like getting ready and then executing, right? And so it helps you to decompose them, which again then helps with what we talked about earlier, you know, headcount, slack, buffering, understanding what's critical and not critical. And so putting up a Gantt chart is a way to sort of say—and in some ways, you know, not the right use—but work backwards and say, 'Hey, we really need this to be done by this date.' If we really put all the processes in place and say, 'You know, when do we think we need them?' And sometimes you realize, oh, actually we need to start this a lot sooner than we thought. And then you say, 'Oh, we don't have the right personnel.' And that's always a very delicate thing to balance, which is, 'Hey, you may need 10 people in the beginning and only two people at the end.' Is there a way for us to be thoughtful and make it so that, you know, we have a more level headcount throughout because it's very difficult to need 10 people and then just get rid of them. Ramping up is generally easy; ramping down 1:23:49.160 --> 1:24:07.664 can be a bit more difficult. Um and so is there tasks that aren't in this dependency chain that we can reschedule or reallocate and and you know sort of make it a little bit more palatable? And so Gantt charts are a very specific way of sort of identifying the interconnectivity between the tasks and and the schedule. Yep. 1:24:08.963 --> 1:25:38.029 Yep, that makes sense. A couple of like you'll hear these methodologies get thrown around, like Scrum, Agile, Waterfall. And I have to confess, like it's they—they all occupy some like really nebulous part of my brain where like it all just kind of flows together, like a like one of these like really weird text-to-image things. If you put in like 'you know, a theory of mind' into a text image generator, you just get garbage. So I don't quite understand the different nuances and differences there. Um I'll just say my piece on it, and then Patrick, you probably have a lot more experience. Um I generally feel like you should just consistently be replanning. So, you know, in my case, like right now for example, you know, I have a weekly planning meeting at my current job, and so we look at the last week's plan and, you know, the quarterly plan, and how are things trending? And I just think it's really important to stay kind of on top of your plan and just discuss it every week. And then also have like a very quick touch base every day of like, 'Hey, how are things going? What's the roadblocks? What are the barriers? What are the accomplishments?' And just you know kind of touch base. Um and beyond that, I haven't really found any of these like acronym or any of these words like Scrum or Agile to be particularly useful. 1:25:40.020 --> 1:27:36.180 I think it's one of those like people are surprised when I talk about it. Um in general has been my experience and talking to a lot of other people that I that I work with and around that the sort of Silicon Valley tech companies are—I don't know—like a bit unique in a couple of ways where they try to hire people that are very talented. They try to hire people who aren't hyper-specialized by default. They generally prefer people who are flexible more than specialized. Um and their customers are like end shipping products and not, you know, pay for accomplishments and milestones. So you'll hear about even internally we'll talk about like setting milestones if you worked where you were contracting for another company to deliver something and they weren't going to pay you all the money up front without knowing that it was done and you need to pay your engineers. So you can't take all the money at the end when you've finished. So you need milestones. Those milestones need to be evaluated. I think you get a lot more process heavy um, you know culture which is completely makes sense. Silicon Valley is a little different, and that people can kind of tackle many many of the different tasks that are within a project that be better or worse, but they try to be generally less specialized than maybe in other industries. And the fact that most of the stuff is all what what is internally funded—they're not doing it against a contract; they're just internally trying to do research and try to make the product better and ship metrics. So all the goal setting is internally motivated rather than tied to dollar amounts that occur for incremental progress. Um it's sort of the end outcomes that matter, and nothing along the way. It's just sort of like 'Get it done,' leads to a more flexible dynamic, less strict stricture around sort of following this methodology. But if you look at like defense contractors or people doing contract software for other companies, 1:27:36.180 --> 1:28:35.920 you end up with a lot more of these, and there are good ways of doing them and bad ways and pros and cons of each of them. But in the end, a lot of it is dictated by how the money is going to flow from the person paying for the work to the people doing the work. And so I also have very little experience um I can't talk about you know pros and cons of various ways of doing these things because in general I would say we are it like like Jason is sort of saying it's a sort of continuous process where we're doing this, but it's not very formalized. It's a very informal process, and you rely heavily on everyone generally being aligned towards the same goal. And acutely we'll see when people on the team aren't aligned to the same goal, it's actually problematic that there's none of these processes in place. When everyone is aligned and has the right incentives, it's actually amazing because you don't end up with any of the overhead that these processes would have. And so it's sort of a I don't pick your poison kind of thing. I'm not saying it's better; I'm not saying it's worse. It's just very 1:28:35.920 --> 1:28:48.683 Different. Yeah, that makes sense. And I guess maybe we could distill the few sort of like key components to all of these methods. You know generally, you know when you're 1:28:50.489 --> 1:30:47.700 doing any type of project planning, you have a backlog. So the backlog are your ideas that you've thought up but you're not ready to work on. Typically, you'll get a lot of tasks assigned to you from other teams or other members of your team. And then you have your, you know, current things that you're working on, and the Gantt chart that Patrick mentioned, and all of that associated with that. And then you have things that are you've basically committed to—like these are things that we're definitely going to do, and maybe you even have dates on those, but they're not in progress. So effectively what you're doing on a weekly basis is you're looking at your backlog. You're saying like, are there things that we should add to this backlog? Like, have people thought of new things? Is there derivative work? Is there a task that needs to be split, and that second part needs to go in the backlog? And then you're also assigning sort of like a rough amount of work to the things that are in the backlog—like is this a really large task, a tiny task? And then what's the priority? So you're trying to prioritize that backlog even before you start pulling things up. And then you're depending on the job. I have worked different jobs where sometimes like in research things are really staggered, right? Because you have all these different conferences, and they all have different deadlines. So like, so the person who wants to submit to ICML is on a totally different schedule than a person who wants to submit to NeurIPS because those conferences have different deadlines. If you're not dealing with kind of that kind of world, then you can maybe have something more regular where, you know, at the end of the quarter the entire team does a retrospective of the whole quarter. You know, that's something that can work if you're in a less 1:30:47.700 --> 1:31:56.440 heterogeneous environment. But I think, you know, so two big pieces of advice for project planning, and this is important whether you are a Project Manager, an individual contributor, a manager, etc., is look backwards just as often as you look forward so you can kind of calibrate yourself. You know, have a weekly meeting where you kind of go through those different states of your tasks, and then also make sure that you're putting new ideas down. I've seen a trap where people will just get into bug fixing mode and kind of forget to innovate. And so sometimes you have to put a bug on hold so you can innovate. And again, that's if people are upset about that—that's when you go back and say, well, this is it gets back to headcount planning, right? But I've seen a lot in research where, you know, research idea becomes production server, and then all of a sudden like it becomes total project planning chaos. And these are all steps that will help mitigate that. 1:31:56.440 --> 1:32:55.615 Quickly just cover some of the tools that you might hear that are associated with this. My favorite is whiteboard, and then I had to clarify because it sounded like it was a product name, but just like an actual whiteboard and people in a room standing around and quickly like it. We talked about work from home from last time, but I will say one of the things that most—I think can be difficult without just being there and like on a piece of paper or on a whiteboard is just that real iterative dynamicism that happens very early in the process, specifically, or you know when you're really under a lot of change. The digital tools for me at least are just they're not as easy to kind of just like put up something, draw it, redraw the arrow, move it here, move it there. You know, you're trying to—the tool forces a certain behavior versus you kind of just want it to be at least for stuff I do, you know, a little more free-flowing in that early project planning stage. 1:32:56.475 --> 1:33:02.145 If only someone made a really, really large E-Ink tablet like four by six feet. 1:33:02.145 --> 1:33:06.840 Like the size of a whiteboard, but it's E-Ink. Yeah. 1:33:07.967 --> 1:33:11.714 This is all I want. Please some factory in what you 1:33:12.625 --> 1:33:21.299 need is like modular E-Ink panels like with magnets on the side. You can just like build your own, put a bezel on it, stitch them together. Whatever that would be amazing. 1:33:22.210 --> 1:33:30.782 Yeah, whiteboarding really important, really useful. Do you how do you use a whiteboard since you're remote? Patrick. Yeah, I don't 1:33:31.170 --> 1:34:02.559 have a good answer for it. Occasional like on-site visits is a good way to try to accomplish that and planning when done right—I mean, doesn't that kind of stuff doesn't need to be done super frequently? I mean, updates to a plan sure, but you know that sort of initial stuff doesn't happen quite as often. But I mean there are, you know, digital whiteboards that you can erase and draw on and scribble for some reason they're just never quite as clean as whiteboard. Oh, here's the bonus in real life: one Post-it notes. 1:34:02.811 --> 1:34:05.579 Oh yeah, Post-it Notes plus whiteboard. Yeah, okay, those are my 1:34:06.135 --> 1:34:10.034 Two. Okay, you can move on to the I guess more serious. 1:34:10.371 --> 1:35:03.595 I mean I've used all of these. They're okay, you know Jira, Asana, OpenProject. These are all names that you'll hear. I'm sure there are other ones too, I can't remember, but um, uh, you know all of them more or less do the same thing. I mean if I had to recommend one, I would recommend Jira because it's free. You can have—I think one or two projects for free. So if you're a student out there, I know the majority of our listeners are students, just use Jira. It's free. But, uh, you know the interface is a little clunky. You know Asana, I would say is a slightly better interface, but is it worth the price if you're a student? Probably not. If you're a professional, then that matters less—the cost of it. Um, they're all great projects. They'll all help you, and they all have all sorts of integrations so you can get like a Slack message or an email whenever there's a new task and all that good stuff. 1:35:07.459 --> 1:35:16.179 So we talked a little bit about dealing with uncertainty. Oh, uh, there was one more tool I don't know anything about—that OpenProject. Do you know? Is that... that's, yeah? 1:35:16.180 --> 1:35:50.912 OpenProject is it open source, kind of like Jira equivalent? And so if you, you know, there's times where like going open source just feels kind of cool or you might just be feeling like I really don't want to depend on any service or anything. And I did run OpenProject sometime in the past, and it was pretty good. I mean, it had a lot of the same features, and your code you can totally edit it and do whatever you want with it if you are like really into project planning to where you want to sort of like hack on the Project Planner itself, then you could use OpenProject. Oh, we— 1:35:52.212 --> 1:37:50.180 Also didn't cover but one you may hear a term is Kanban, and it's really sort of like in all of this stuff, but but you know there are many, many tools for doing for doing Kanban boards. Um, but those are more or less having tasks on little boxes or Post-it notes and columns and sort of moving them for just trying to get into that low overhead understanding that situations are dynamic and a sort of visual representation of it. But the final thing to just talk about is, you know, I guess to continue Jason's quote: what happens when you get punched in the face, somebody gets sick, the goal posts get moved, whatever, you know, sort of happens, and your plan is no longer sort of working. And we talked about making sure that you do have some padding, some buffering specifically around things that are harder to schedule like—you know Jason keeps talking about research, and I think that's very difficult because it's sometimes very hard to put a specific time on when you're going to have that innovation or when it's going to occur. And so making sure you have buffering in the schedule can prevent it. But again, once you get punched in the face, it isn't really there. But making sure that everyone's very communicative and on the same space and that you know being really honest with management that there needs to be slack in the in the schedule—to say, hey, we're going to have issues. We don't want to, you know, have everything booked end-to-end, no no downtime, no slack. Like that's just a way to have the, you know, expectations get missed. And then once they are missed, being honest—like for me, I'm not everyone does it, but I try to be very clear when we're no longer tracking to schedule. A lot of people try to hide it and we'll make it up. We'll make it up. I'm a big believer of let's move the schedule, and if we make it up, we'll move it back. Um, and that doesn't make me always popular with people, but stuff generally gets done. And as you get closer and closer, the certainty of that final date is is more nailed down, and you're not always all situations aren't flexible. 1:37:50.180 --> 1:38:01.559 Like that, but for me that's a huge thing is just saying, listen, that end date is going to move. It will move the closer we get, but we should move it to be accurately reflecting of what's actually happening. Yeah. 1:38:02.470 --> 1:39:59.620 I totally agree. I think that um we covered this already, but there's this—there's a short-term gain. I won't lie. I think there's literally there there is a short-term gain in fudging the numbers, making your plan in hindsight look like incredibly prophetic and you know trying to like sell some Cinderella story about your team. Like you could probably do that especially if like your manager is also pretty junior, like you could get away with that. Um but it will fall apart. It is not a long-term strategy that is going to work. Um eventually the what is it? The chickens will come home to roost after you get punched in the face and all the other analogies. Um um but you know just being up front, and you could end up at an impasse where you know you feel like this set of goals is nominal is expected and your team disagrees or maybe a single person on your team says, you know, this is way more than I can accomplish. And those are very difficult conversations which makes the project planning even more important so that you at the end, you know, and and this is not something that people like to talk about, but you know if you do have a person who's just not performing and you have to have some really hard conversations, you can say, look, like here's the plan we set out. There was disagreement about the plan, but you know the rest of the team, you know feels like this was a very reasonable plan that somebody could do in a reasonable amount of time and that person didn't do it, and so we have to, you know, have a performance discussion. So so being able to do all of that planning up front and and have those, you know, hard conversations up front is super important. Um um and and it will help give you sort of like a long-term sustainable team. 1:39:59.620 --> 1:40:10.620 All right. Um I think uh that about wraps it up. Do you have any parting thoughts before or we hit the road, Patrick? 1:40:11.580 --> 1:41:36.310 Uh just reiterating my disclaimer—nothing we said. No, I'm just kidding. Uh yeah. I mean everybody does it whether you want to or not, informal formal like somewhere on the sort of gradient from nothing to overdoing it, but you know project planning, it happens. I mean if you, you know, are gonna have a have a kid, if you're going to get married, if you're going to like projects and planning happens. And um you know you can either sort of blunder through it or you can try to pick the best pieces that work for your situation. Um and and really try to have it be something that you know maybe it's not your expertise but isn't a weakness. And so I think there's a lot of self-learning that can occur by planning things and then looking back and saying, like, hey, they didn't go according to plan. That's okay. But like, you know this was my plan, and sometimes the plan can be a little bit more audacious and you know be inspirational, but you know maybe you got to be real that it's that. But you know it's sometimes sometimes you're right. Like I think, you know in our in my personal life, I just try to say like hey, I'm gonna have a plan for the year and at the end of the year look back and say, hey, did I hit or miss? Right? And so not to get deeply philosophical, I guess, but you know I think planning and goal setting and projects is something that really goes across sort of all the things we do. Yeah. 1:41:36.732 --> 1:42:03.023 You know, the other thing is there's a other side of this which is celebrate when your team does awesome. You know, I've seen a lot of managers forget to do that. Um they're sort of like almost afraid of the first thing I said—they're afraid of like people think you know we're sandbagging or whatever. But like look back and say, like wow, have we set out to do five things? We did six things. They're all awesome, and and we should throw a party and get sushi for everyone. 1:42:03.833 --> 1:42:06.380 something oh oh man i'm hungry oh man now i want 1:42:06.380 --> 1:42:30.260 sushi all right we will let everyone go super uh uh shout out thank you to our patrons we really appreciate y'all support and um and uh now we should all go get some sushi see you next time catch you later so 1:42:30.260 --> 1:43:16.580 and uh share a liking kind share a liking kind instance