WEBVTT 0:00:15.200 --> 0:00:21.600 Programming throwdown episode 185 workflow orchestrators. Take it away. Patrick. 0:00:22.240 --> 0:00:26.720 Welcome to another guaranteed to be fantastic episode. 0:00:27.370 --> 0:00:31.210 That's right. Yeah, it's money back. Yeah, I nailed it. 0:00:31.370 --> 0:00:39.930 We've been doing this too long. Okay, I'm going to try something a little different. I was thinking about this. I don't have a good formulation of it, so you can sort of help me. 0:00:40.170 --> 0:00:40.730 All right. 0:00:40.970 --> 0:04:49.080 But it's sort of been coming up in a number of things and the sort of most obvious one is when people talk about venture capital, which is, you know, investors, mostly tech investors, and they invest in, you know, a hundred companies, but really they're expecting sort of 10 to do really well. They're hoping the rest don't, you know, they just assume they basically go to zero. They kind of don't care. And then they hope, you know, one or two 1000x or whatever and pay for the risk, they take over the entire pool. And you know, part of that is this sort of like asymmetric returns. And then I began thinking it, it pops up in sort of other things. You know, people were talking, you know, most recently I was listening to someone talking about, about health recommendations, like how much protein should you consume? And people have this, you know, oh, maybe you did this much, maybe that much, maybe. And they were sort of said, you know, up to a certain point it kind of, you know, is very, very important. And then past another point, you know, it's you, you kind of get this like non linear relationship. Um, and then I began to think about it and, and they kind of said something that led me to thinking about it in terms of, in terms of work, which is if you set a certain target goal for how much you learn, you know, in a given day, or continual learning, or, or you know, how much output you want at work. I think internally we have this, you know, sort of everything gets kind of normally distributed that maybe some days you're going to be a little bit above, some days a little bit below, some days, you know, very rarely a lot above, you know, very rarely a lot below. And in my head at least, you know, I think we do this thing that pops up intuitively over and over again where things kind of clump together and become sort of normally. And we think that the returns there end up being, you know, kind of not super impactful, like they even out basically that the unders and the overs. Overs, you know, are okay. But what I began to realize and sorry, trying to bring it full circle, it's a little cloudy in my mind. But back to the VC thing, this, this health thing that they were sort of saying, you kind of need to set your targets a little higher because actually depending on where you are on this sort of like return curve, right, that is not flat. It's not, you know, just one single slope that if you are at one of these inflection points or lower, then the good days can be sort of 10x and the bad days are, you know, negative some amount. And if you are really low, what ends up happening is those negative amounts are, are sort of larger than the big amounts and you can sort of lose ground or, you know, not, not move forward versus if you're higher up on the curve and you set your target, then even the low days, you know, you're still sort of moving forward. And so when I talk about like learning or output at work, if you, you know, sort of set your target as like my median is, I get a little bit of good stuff done and some days I get a lot of, some days I don't. Some days I, I make more work for myself by doing something stupid or saying something dumb at work. Right? You, you kind of think about this as like evening out to like slow small steps. But in reality, setting yourself up for at least some of those home runs. So setting yourself up positionally, but also just where your target is, like in how you expose yourself, it sort of drifts away. Sorry, my analogy is breaking. But like you sort of shift yourself into a position where you're, you still have days that are below average, but your above average days give you this exposure opportunity for just outsized growth. Right. You're taking those really, really hard home runs at work. You're, you know, going for just, you know, really stretch opportunities. And thinking about this, you know, it's just an interesting thing I've been juggling with, which is, I think a lot of myself included, a lot of the time I don't give myself this opportunity to have this, this sort of outperformance, this exposure, this VC1000X. Right. You know, we talk about investment. You do the boring. Everyone always says just put all your money in the s and P500 or the Russell 5000. You know, just put all your money in there. It's very average out return. But maybe some, some amount of it, maybe some small amount. You gotta take those like really big swings and you gotta limit it. But if it, you know, 1000x, it could carry the whole portfolio. 0:04:49.720 --> 0:05:03.940 Yeah, so I can, I can riff on this. I mean, so I don't know if I mentioned this in a past episode, but there's an awesome book from Nassim, Nicholas Taleb, the Black Swan. Have you read the Black Swan? Yes. 0:05:03.940 --> 0:05:04.300 Yep. 0:05:04.300 --> 0:05:12.460 Yeah. So, yeah, I'm a big proponent that there are black swans and there are white swans. In other words, like, there are. 0:05:12.620 --> 0:05:14.860 Wait, is white swan the opposite of black swan? 0:05:15.340 --> 0:05:27.300 Well, white swans. Okay, first of all, white swan is a term I literally just made up five seconds ago. But, but the way I would define it is like, you know, it's a unexpectedly high return. 0:05:27.540 --> 0:05:28.100 Okay. 0:05:28.420 --> 0:08:33.970 Because by definition, using your VC example, you know, the VC funds, I've heard things like they do a hundred companies and they expect one company to make up for the other 99. So it's even more extreme. But so what that means is if put yourself in the chair of a vc, when you talk to one of these hundred companies, you know, you're kind of expecting them to fail because, you know, statistically, they're going to be one of the 99. Right. And at some point, you know, you figure out which one the one is, I'm sure. But I would consider that kind of like, like you're betting on a opposite of a black swan. Like you're betting on an unexpected, like, huge boom. Right. And so, yeah, I think that, you know, otherwise you wouldn't, you wouldn't really take the risk, Right. If there wasn't the opposite of a. Let's call it the white swan, the white swan wasn't there. The VC wouldn't do what they do, right. If, if, if one company, you know, 5x and the other 99 went to 0, that wouldn't work. So, so, so they're kind of counting on that. And I think that is kind of a, a good way to look at things. I think that there's sort of certain moments where you have kind of eureka moments or you have kind of a burst, or you just get lucky. And, and so kind of budgeting for those kind of moments, I think makes a lot of sense and being, being kind of ready to really capitalize. That's the other thing is the vc hopefully is set up to where, you know, when they do find the one that they can then, you know, 10x triple, triple down on that one. So, so same kind of thing. Like you, you're constantly kind of exploring, you see something pretty exciting, and then you kind of bet big on it. I do think that most returns are asymmetric. Oh, the other part I was going to say is, you know, your example with investing, you Know, I think that at, at a large enough scale, things start to become normally distributed. Right. So, so in other words, like if Broadcom has like a killer year, well, it doesn't really matter if you've invested in, I don't know if they're in the S&P 500, but if you, if you invested in all S&P 500, then their killer year gets sort of diluted. Right? So like at scale, you don't really have black swans or white swans, at least you hope. Which is, which is the reason why we can all be sort of comfortable. We're not just anxiety ridden as, as, as we get older. Right. So, so I think that, that, that at scale, when I think about investing as someone who's not a very savvy investor at all, you know, I, I just think, well, this, their returns are going to be normally distributed around 5% or 8% or whatever it is. But then individually or when I look at things at a smaller scale, I do think the returns are often asymmetric. 0:08:33.970 --> 0:09:38.820 Yeah, I think I might have mixed up sort of two, two things I might not. So, so the one you riffed on is, is good. So I think there's like taking it to career growth. Not that we always have to talk about that, but taking. I, I actually believe I was bringing this up the other day. Most places we overestimate the impact of a failure. Like if you really, if your boss came and asked you, hey, do you want to try this like really hard thing? And you're like, I'm not sure. Yeah, I like, what's really the downside of, of failing? Like, I think most of the time, in most cases, in all cases, you know, this is not, this is not universal advice. I'm not a lawyer, I guess, but like most cases, the risk of failing is pretty low. So it's actually really capped to the downside, you know, and it's, I guess maybe similar to the VC example. The chance of the VC having money taken away from them is, is non existent. So the worst case is capped at zero, but the top case is uncapped. So if they do really bad, if 100 out of 100 fail, they lose all their money that they invested, but it doesn't mean they invested all their money. 0:09:39.300 --> 0:09:39.860 Correct. 0:09:40.340 --> 0:10:56.590 But if one goes to Facebook scale or whatever, you know, then all of a sudden, you know, they're filthy rich. And so I think it's, that's probably a good mapping. I think there's another thing. I was trying to slide in there, but I Maybe now, thinking about it, I. I muddied the water. I think it's. It's separate, which is just like individual performance day to day. So, like, taking opportunities and getting that exposure, but then day to day, trying to set yourself up so that like, like you're getting the reps in, you're getting practice in. Like, you're. You're getting yourself, like, training and skill and like setting your expectations for yourself, maybe higher. Knowing that sometimes you're not going to do as well, and then realizing when you practice being at a higher level that you're gonna get similarly. You're. It's not exactly the same though, I guess, but you're gonna get those out performance days and those outperformance days may shock you with just like, I've had that before. You call it kind of like eureka moment. You're, like, doing some work and being up at that higher level. You're going to potentially see more of them where it's just like, what was I thinking? That, like, it was amazing. Like, how do I get that in a bottle? Like, I would love to be like that every day. And it doesn't happen often, but I. I do feel like it happens more often when I'm operating at a high, not burning out, but when I'm. When I'm expecting more out of myself and performing at a high level. 0:10:56.830 --> 0:11:31.410 Yeah, I think that makes a lot of sense. I think that similar to the vc, you know, having that money to invest in the next round, like, you also need to balance your work and life so that when you do have like that eureka moment, know, you can kind of really triple down on that and then. And then go back to a. A more healthy state. So I think. I don't know if we have a term for this in my family, but. But there is kind of like this notion like, oh, you know, dad's going into some work hole, you know, he's going to come back in like a couple of days or something. 0:11:31.410 --> 0:11:32.890 I call it the fugue, but. 0:11:32.890 --> 0:11:34.530 Yeah, the fugue. 0:11:34.850 --> 0:11:38.330 Yeah. Isn't that like an organ thing where, like, the organ music gets really loud. 0:11:38.330 --> 0:11:41.930 And, like, you know, oh, look at you. I've never. 0:11:42.330 --> 0:11:42.690 Never. 0:11:42.690 --> 0:11:44.850 That's an SAT word I've never heard. 0:11:44.850 --> 0:11:46.050 Now I have to look it up maybe. 0:11:46.050 --> 0:11:46.370 Yeah. 0:11:46.370 --> 0:12:02.410 Fugue state. Here we go. In music. Yeah, it's a fugue. A fugue state is a disassociative, psychological, unexpected. Oh, wait, this is bad. No, this is like. This is like you have amnesia. What? Are you completely. Okay, maybe I'M thinking, I guess I'm thinking of Patrick's. 0:12:02.410 --> 0:12:04.530 Patrick's going dementia again. Girls. 0:12:04.530 --> 0:12:15.870 Oh no. Oh no. Hang on. Uh, I don't know, maybe I'm just using this, this wrong. Oh, well, it's a word like you said. Maybe you just need to define a new word. It's like a flow state. That's why I think the like. 0:12:16.030 --> 0:12:16.510 Yeah. 0:12:16.830 --> 0:12:18.710 You know, anyways, okay. 0:12:18.710 --> 0:12:19.990 But yeah, I do think I just. 0:12:19.990 --> 0:12:26.190 Made up a new word for it apparently, because this doesn't match any existing definition. So if you use this, don't expect anyone to know what it is. 0:12:26.750 --> 0:15:37.250 You know, just. Okay, kind of. Maybe this is too much of a segue, but you, you made me think of another thing. I used to coach, you know, high level ICs at Facebook. So basically, I mean obviously like there's, I wasn't the highest level, so there's people above me and everything. But, but specifically people who are trying to go either from five to six or six to seven. I did a bunch of one on one coaching and then I ran this thing called the IC Circle where you know, we would just do more like one to many coaching. But it was more of like a community type thing. And one thing really struck me. This person said they didn't want. This was one on one. They said they didn't want to go to level six because if you look at the stats, you're something like 15 times more likely to get fired if you're level six than if you're level five and for performance. And they're like, well, you know, I kind of want to take this promotion. They also were, I think they were on a visa. I mean it's definitely part of it, but they just felt like, well, my expected value is zero if I get fired. And so getting, keeping that probability low is actually you're looking. Yeah, I guess it's not zero. You get a job somewhere else. But basically my point is your expected value at meta is 0 if they fire you. Right. But. And, and so the, the, the raise the, the level 6 compensation is higher, but not higher enough to justify that. Right. So anyway, what I told, what I told this person at the time was, was, was basically, you know, okay, I totally get it. There is. And there, I think there was kind of an issue with this. People at 6 and above just getting, getting fired for performance. I said, but you know, plan for the future, don't really plan for the present. So hey, the present situation is you're looking around and you're seeing the level sixes on your team getting, getting sent home. But that's, you can't kind of base the future on the present. And I think that, that, but I, I, I think that yeah, you have to kind of in like, like the asymmetric returns kind of kicks in where, you know, by taking that chance and getting, getting that promotion now, you know, you can always kind of figure out your steady state afterwards. What you can't do, or it's very hard to control is having that amazing, you know, product launch that causes you to get a promotion. Like, in other words, if you have an asymmetric return and you don't capitalize on it, it might be very, very hard to bounce back from that. But if you do capitalize on it and then let's say worst case, you know, you get, you get fired for performance at the next level. I still feel like that was, that was the right call. And just to finish the story out, the person didn't get fired. They're actually still at Facebook, happily at their level. 0:15:38.370 --> 0:16:28.750 So I mean, this got talked about not, not me, but by people in the field, financial people, when Facebook was doing the outsized bonuses for hiring AI people. I think they've turned it down now, but they were saying a little bit of a similar thing. There's an expectation of a continuing current state. So in other words, I don't have to take this right now because I could take it in 12 months or, you know, 24 months. And in reality, actually it turned out it was like a window of like 30 days. Yeah, it was like really short. But it, it's sort of to what you were saying, those opportunities, you think they're, they're going to come at some even spacing or they'll always, and it's, that's not right. It's not true. Your ability to get a promotion, your ability to be on that project, your ability to start something new, greenfield, make your mark, whatever, those things aren't evenly distributed in time. And so. 0:16:29.150 --> 0:16:30.830 Yeah, I, yeah, I agree with you. 0:16:31.790 --> 0:16:44.600 All right, so just to follow up, I, I did find the correct definition of Fuga. I wasn't that crazy. Although it apparently have a slightly negative commentate connotation. But it means to do something where you're completely aware and focused, but afterwards you can't remember what happened. 0:16:45.080 --> 0:16:51.840 And so that, oh, that sounds perfect. That's actually exactly what, so actually it. 0:16:51.840 --> 0:16:56.640 Took me a little bit of searching. Apparently it's a very deep dep. Like a, yeah, back catalog definition. 0:16:56.640 --> 0:17:00.960 But it's like what, what idiot Wrote this code and it was you a week ago? Yes. 0:17:00.960 --> 0:17:18.830 No, no. That's what I like. You get really focused and you churn out all this stuff. My new one is writing code and then, like, having other people sort of help me, like, finish it off or whatever, and then submit it under their names. And then I think they wrote the code and they're like, no, you wrote this code. Oops. 0:17:19.070 --> 0:17:22.910 All right, all right. Time to do news. You're up first. 0:17:22.990 --> 0:17:47.060 All right, I'm up first. So it won't be this weekend when. When this comes out, but that's okay. Just recently, Andre Karpathy, I believe is how you say his name. You know, he's. He's gained some renown recently for a little bit, being the people's AI person, I guess. Like, you know, he's not. He's not running a big AI company. He clearly knows what he's doing. He can explain concepts, but he explains them in depth. 0:17:47.460 --> 0:18:09.930 Let me. Let me. Let me add a little bit of color there. So. So Andre was the. The head of AI at Tesla, and he left. And he basically is like. I don't want to say retired. That's not the right word, because he's very productive. But he quit to do exactly this, like, to just build things for the people and teach the people. And so he's. He's in a way, like, doing some monastic work right now. 0:18:11.210 --> 0:18:13.890 What do you talk about? Sat words? Monastic. 0:18:13.890 --> 0:18:15.370 Oh, there we go. Look at this. 0:18:15.850 --> 0:18:16.810 Wow, man. You. 0:18:16.970 --> 0:18:20.650 You thought. People thought they were coming here to talk about programming. 0:18:23.850 --> 0:19:02.480 So I, you know, in some ways, actually, I think. To your. To your. Thank you for that. Actually, that's. Color commentary is very good. You know, I think our podcast tries to be, although I don't say were as effective as this, something in a similar vein, trying to break down stuff for folks. He released a new GitHub repo out sort of out of the blue, as far as I know. I don't think he was. I didn't know he was working on it. Called Nanochat. And Nanochat is. Previously, he had done some other sort of examples of training. I don't say like toy, but like toy Transformer networks and models and getting them up to sort of GPT2 level, sort of, you know, trying to be 2.2level. Like, they like, wow, okay. It's just better than. If you. If you remember. 0:19:02.800 --> 0:19:03.600 Oh, I don't. 0:19:03.680 --> 0:19:59.650 You know, while ago, a decade ago, people were doing hidden Markov models trained on, you know, Harry Potter, and it would produce Words that like if you didn't look too closely, they kind of look like maybe sentences but they didn't make any sense. And then you know, we kind of got into the chat GPTs. These GPTs were good. So he's sort of done some of that before but now sort of what he's done is done an end to end pre training, doing the chat ui, you know, constructing the models and everything end to end in a single repo so you can really kind of study it and really awesomely if you don't know training the frontier models. I don't even know what the estimates are now like $10 million to train, to train one. They're very, very, very expensive. So most training outside of the maybe 10 companies in the world is doing fine tuning, you know, other small things but not the sort of pre training that's reserved for people have much more money to burn than myself. 0:20:00.050 --> 0:20:10.690 Yeah. And 10 million I think just is one instance. So it's like if you miss a colon, I mean you're going to catch earlier than that. But like in theory if you have some data issue now, it's another 10 million. 0:20:10.930 --> 0:22:09.720 Yeah, yeah, yeah. You know, who knows how many times they have to do it. Yeah, that's right. And that was a big thing if you recall Deepsea getting really popular for they were had their moment. They've kind of, you know, taken a little step back, you know, in terms of, in the, you know, general awareness. But that was their big thing was, was doing some of this pre training cheaper. Okay, anyways, back to the story. So Andre's Carpathy's repo also made it so that you could do all of this for $100. Now a hundred dollars isn't nothing. But I'll, I'll make a couple observations. So first one is again compared to that other amount, people spend more than that. Go play a round of golf, you know, go do something, you know, it, it, it is a lot of money. I don't want to diminish it, but it is enough like if you're doing it for learning, if you're doing it for understanding, if you're you know, really getting into this, it, it's attainable, it's a, a hobbyist level amount that you could spend which is awesome on, on sort of relatively reasonable spec hardware and, and he has the scripts for doing all of it sort of set up, which is really awesome. Um, but then of course the second observation I'll make is two, twofold. One, the cost for his stuff will go down if you wait, you know, six months, 12 months, and I would expect it to be cheaper as new hardware and, and instances and, and supply come online. But the second thing is, as a general sort of observation, a lot of this stuff, this is where you're seeing the growth. You know, we're not seeing chat, GPT, whatever, 5 or any of these other things become 2x better, 3x better. It's not really happening. But the cost to train, the cost to run inference is all coming down dramatically. And here, you know, I think is, is a little bit of an example of, you know, sort of the rise in the hobbyist stuff and the decline of the cost. You know, you're coming to, you know, not exactly a meeting point, but, but coming and converging together to where we can start to not get to state of the art, but push to something that five years ago would have been like, oh my gosh, like you did all of this wrong. That would be crazy. So it's a reminder of the insane progress we're seeing. 0:22:09.720 --> 0:23:14.360 Yeah, totally. Yeah, I think it's a really good, good kind of recap. Yeah, I mean this stuff's super exciting. I think a lot of the innovation now is, and I think Andre actually talks about this, is around, you know, reverse engineering the best possible prompt. I don't know if I brought this up, but I don't think we have. I, I had a situation at my current job about a month or two ago. We changed models. So a model that we were using was deprecated. And so we moved on to the next version of the same model. And the performance went way, way down. Like really, really far down. Like it went from maybe 80% to 7% kind of thing. And after a lot of debugging and, and kind of, you know, looking at, at a lot of numbers, we realized that all we had to do is change the. So, so we had a prompt that was about three or four sentences in a paragraph. If we swapped the first two sentences, the performance went back to 80%. 0:23:16.040 --> 0:23:19.280 So, so basically like the, the, the. 0:23:19.280 --> 0:24:52.620 Ordering of the sentences, the way the thoughts are. And it's not like it wasn't illogical the first way. Right? It was two independent thoughts and they just, if you switch the order, massive improvement. And so what that says to me is that we humans shouldn't be writing these prompts. I mean, we write the first version but like, at, at the bare minimum, you know, we should have a system that just like reorders them a few different ways and picks the best ordering. Like Somehow breaks them into independent thoughts and then tries different permutations. But, but at the limit it would be, you know, some type of feedback mechanism that's constantly updating all of these prompts. But that I think is where I'm seeing kind of a lot of the innovation. And I think it's because of cost. Exactly. You know, as you said, when it costs 10 million to train a model, what we ultimately need, and I think nano Chat's a good way of getting there, is, you know, a set of benchmarks and a set of models that are small and trained for very cheap. And then you can, you know, learn, learn some insight. Someone can invent the next thing, right? Like maybe it's SSMs, maybe it's diffusion chat models, right? Someone invents the next thing and then it's, it blows everything else out of its weight class. But, but it's still nowhere near ChatGPT, but, but it totally dominates the very cheap Chat GPT. And so then you could say, okay, this is now worth spending 10 million. 0:24:52.940 --> 0:25:36.580 I mean, I guess that's partly where I, I think the running local models, you know, on your, whatever home, home smart device, your personal assistant on your phone, where we were just checking before briefly, there's a, you know, cloud outage this morning. So that happens from time to time. And so I think having some of these models run locally and I think when they hardware gets sophisticated enough and the models get distilled enough to be like absolutely actually useful in those things, I bet you see a lot more investment into, into that as like a focused product that you can have like what is state of the art for a capped size model? Not unbounded, right? 0:25:36.740 --> 0:26:04.770 Yeah, exactly. All right, My news article is pedantic AI. To be totally honest, I picked a random news article. I really just wanted to talk about pedantic AI. I could have made it a tool, but I had a different tool. So this is really cool. I used this the other day. I built a very simple test, like a prototype. So I downloaded. Remember PC Part Picker? 0:26:05.490 --> 0:26:05.890 Yes. 0:26:05.890 --> 0:28:19.270 Remember that website? Yeah. I don't know if they're still around, but someone posted basically a CSV files of all the inventory of PC Part picker. So there's a CSV file for heat sinks, a CSV file for processors, et cetera, et cetera. And I use pydantic AI, which is this way to build AI agents, which you should probably do an entire show on, but it's, it's taking off and I think it's very interesting And I only give it access to one tool. And the tool is, you know, DuckDB. We've probably talked about this. Yeah. So the tool literally is. You can it. You pass in which CSV file you're interested in, video card, CPU, et cetera and DuckDB SQL query. And this, this AI tool opens that CSV runs a query, returns back the results. And so that was it. And so this whole AI agent thing with pedantic AI, it was like 80 lines of code or something. And now I have this thing where you can ask it to build you computers. Like you can literally type in, hey, I want a computer with the Nvidia GPU, at least 16 gig of VRAM for under $2,000 and it will actually build a computer for you out of all those parts. And it runs SQL queries and you can actually see all the steps. Like first it runs like a describe select on each of the CSV files so it can get all the schemas of them all. And then it starts running queries. Sometimes it'll type in a query that doesn't compile. DuckDB will like send an error back and Pedantic can act, will actually catch errors and turn them into strings and send those strings back to the AI. So the AI like got an error and was like, oh, I need to change the query. And it did. And, and it eventually gets there and it generates like builds PCs that, that are good. So but, but I just found this amazing and I'm sure there are other tools out there, but I just, I couldn't believe how in just a matter of hours I was able to go from nothing to all that. 0:28:20.310 --> 0:28:28.180 That is really cool and it's really like commoditizing. Yeah, it's coming, coming to the fact that yeah, we can do that in a super cool. 0:28:28.340 --> 0:28:56.050 Yeah. So if you're doing agentic stuff, these. This is the same company that made Pydantic, which is a, which is kind of like a data class system that's very popular and really well done. Actually it's the same. The same people made SQL model and fast API and like a ton of amazing libraries. So. So this is also, you know, walking in good footsteps. 0:28:56.130 --> 0:32:00.950 My next news article is something that I actually botched and then it was amazing how it was amazing before. It was amazing. Okay, here we go. So I saw this news article and it was 1000th Starlink after a launch this again this weekend from SpaceX. So all the Starlink satellites up and there's been actually a huge thing like the amount of money they're making. It's a private company still, you know, owned by SpaceX. But the number of subscribers they have, like as far as like a tech story, it's, it's actually really astonishing. Would love to have, have invested. Would invest. It's private. So it's, it's a bit, bit of a thorny issue to, to try to, to get access, but this is a great idea, I'm really for it. But you know, saw this, I was talking about it that you know, there's a thousand thousand satellites and then I was like wait a minute, hang on, let me look, look at this again. And it's, you know, oh, it was a thousand Starlink satellites had been launched in 2025 so far. So yeah, what I thought was like, yeah, what I already thought was amazing is like, oh, there's a thousand of these satellites, you know, orbiting around. How cool is this? It's like no, that's incorrect. There has been a thousand launched to this year. So then I wait a minute, how many are there? And it's like 8, 400 and something in orbit. There are so many in orbit now that actually every single day one to two Starlink satellites incinerate back into the atmosphere from sort of end of life ink, which is expected. So they put them into, you know, relatively low Earth orbit. So their orbits, you know, there's a little bit of air friction, whatever they naturally decay. They're meant to do that. You don't want them staying up forever and you know, getting obsolete and then just drifting around a space junk. So they, I actually think they do a process for you know, intentionally sort of making it so they'll deorbit deliberately rather than just, you know, chaotically. But basically there have been so many gone up and going up that one to two every single day are just basically burning up. And it's still this like insanely lucrative business doing all this crazy stuff. And there are 8,000 found a visualizer, you can click and actually see all of them because there's you know, sort of orbit information is known. And it turns out orbital mechanics is like well understood. So you can actually just go and visualize the entire like mesh of all of the orbits which is very well planned to get nice coverage. You can see new ones that are being deployed are like clustered together and slowly drift to their, I guess, station. Not familiar with the term, but it's absolutely bonkers just to think about the scale of this. And I don't think I, maybe I've talked about on the Podcast before, but I was talking about it with. With some local friend group. You know, just sort of how if you had said this, you know, whatever 10 years ago, like, oh, yeah, there's like this darling thing. It's roughly competitive with broadband. It works great. You can play video games on it. Like, it's not super laggy. It's in this little dish, this, you know, flat dish thing. And people would be like, what? No, that. And then now people are like, yeah, of course. That's called Starlink. Like, yeah, I have that in my cabin. 0:32:01.030 --> 0:32:14.629 Yeah, it's. It's phenomenal. Do they have coverage in the ocean or like. Like do. Or how does that work? Is it the kind of thing where it's only in places where there that can be populated or is it literally circumnavigated? 0:32:14.629 --> 0:32:48.890 They. They have slightly different fee structure. I'm not clear if it's because they can or because there's a reason. But, you know, people have it on like, their ships, on their traveling RVs, on their boats. Yeah, there. And now the. This weekend or whatever they're demoing. And I guess maybe some people still don't know this, but there was like a couple weekends ago or whatever they were talking about one of the airlines, I want to say United or American here in, in the United States was going to be putting Starlink dishes on all of the planes, which they've used a different satellite provider. So if you've ever gotten satellite on an airplane, it's, you know, it's not very good. 0:32:48.970 --> 0:32:49.970 Yeah, yeah. 0:32:49.970 --> 0:33:23.370 But now they were like, flying this reporter up and sort of, they were able to take a FaceTime call and it felt like normal and they were like, from an airplane and. And they were just like, their mind was sort of like blown that, you know, flying through space. I think Elon Musk even ended up retweeting about it and he's like, well, if you're traveling whatever the orbital speed, I won't quote it because I'll get it wrong. Of a Starlink satellite is like, even airplanes look basically still. And so, you know, that makes sense. Flying through an airplane, like, oh, my gosh, it's moving so fast. Aren't there going to be all of this? And he's like, no, they basically look like they're standing still because we're. We're flying so fast. 0:33:24.170 --> 0:33:25.610 Wow, that's incredible. 0:33:26.650 --> 0:33:38.250 And so it's just this interesting sort of normalization of what is. Is just absolutely. It's, it's. It's rocket science. Right? Like just all of this stuff, you know, being up there and going around is. 0:33:39.210 --> 0:33:48.090 So when they, when the sat. When the Starlink satellites are, are, Are, you know, shuttled, is that shuttle reusable? Or like, how does that work? 0:33:48.320 --> 0:34:31.550 Yeah, so they did the Falcon, Falcon nine. So if comes back and lands normally on the barge, so they go up, launch it in a. The, the second stage, the top part of the rocket, as a fan go up and they don't relaunch that. That's the new thing, the really big one that they make a big deal out of Texas. The, the starship and the booster that goes with that. The full thing is supposed to be reusable. So that's like where they see the future. But for now, the first stage, which has most of the, you know, more of the rocket engines and everything basically goes up and then comes back and lands. And those are the videos you see. And so that's why they're able to put, well, part of why they're able to put so many satellites up in orbit, but they still. There is some portion of the launch vehicle that doesn't get reused. 0:34:32.270 --> 0:37:23.280 Got it. Makes sense. Cool. All right, my second news story is the introduction of the apps in ChatGPT and the ChatGPT apps SDK. I feel like this is really power. I feel like there's something here. So just to explain at least how I think of it, because it's still very early, basically, and I don't have a lot of details on it yet. I don't think they even kind of have a lot of details yet themselves. But the way I imagine this working is, you know, let's say you ask ChatGPT to play your favorite song or you ask it to buy eggs from. From. I guess if you say from, let's say buy eggs from walmart. Right? So right now, if you do that, you know, Chat GBT is going to say, hey, I can't do any of these things. Would you like me to do something else? It's always so polite. It's like, would you like me to just talk to you about eggs? But, you know, in the future with these apps, you know, you will install the Walmart app, and when you ask a question, you know, the, the OpenAI folks will figure out from your question whether they should route that question to an app or not. And if, if they decide they should, it gets routed. And now Walmart, you know, which probably is using ChatGPT anyways, but like, now Walmart gets your question and they're, they're Poised to, to handle it differently. So, for example, if Walmart gets your question, they could order the eggs. You know, they have your identity from you being logged into ChatGPT, and so they can order the eggs, and eggs show up at your house the next day, which is pretty cool. I do feel like this is kind of like a new kind of App store moment, which is pretty exciting. I do feel like we'll kind of look back on this and say, oh, like, I wish I'd built like the, you know, whatever the most obvious app is, maybe groceries is it. I don't know, but I feel like now is the time to think about, like, what is the app that obviously everybody wants? Because apps aren't even allowed on the store yet. I think they're going to announce a date when you can start submitting apps. But, but you have access to the SDK, so. So if you have something where you think chat would be a better experience than going on the web or whatever else you would do, now's the time to build that thing so that as soon as they announce the submission process, you could be first in line. I do think there's something just extraordinarily powerful here. 0:37:23.520 --> 0:38:06.330 When this was launched, I saw it was like a thousand startups are killed by one. One slide or something was like the headline. But I guess maybe I like the way you explain it. I guess if you develop your app, it's sort of like now you can host your own, like, server if you do something local and your chatgpt, your chat agent can query your, you know, local server that's answering something. So it's like sort of you do some training or upload some workflow or something, and then OpenAI will route ChatGPT queries to that internally. If you host something there, like, if is it state in their ecosystem, can you run arbitrary code? So not completely clear. 0:38:06.570 --> 0:38:43.650 Yeah, let me, let me see if I can explain it. So the idea is, you know, you, you, you submit an app, OpenAI approves it. Now you're on their App Store, right? So, so a person who uses ChatGPT installs your app right now when that person asks questions to OpenAI, OpenAI can route some of those questions to you, and you could then use OpenAI as part of helping answer the question, but they're now expecting you to respond with an answer. 0:38:44.690 --> 0:39:13.730 So in the case, like you said, so you're a grocery store, you, you do work to make sure that, like, your OpenAI app knows how to call your inventory and pricing, you know, rest API. And so you give all of that information or whatever to your sub component app and then like, okay, and then the user installs that one, which is why it doesn't have to disambiguate across every app on the App Store. Okay. 0:39:14.040 --> 0:39:59.890 Yeah, I think that's right. And so you can also build tools like I talked about with the Pydantic AI. So in addition to being sort of so Open ended that ChatGPT just calls you with a string of text. You can also have a tool. So for example, maybe you have a buy groceries tool. And that tool takes like some very specific structured data so like the name of the grocery and, and, and how, what quantity and all of that. I mean you still have to do all the validation yourself because you don't Never know what ChatGPT is going to call your tool with. But, but at least that might be more structured than, you know, you just have a string of, of text and have to figure out what to do with that. 0:40:00.130 --> 0:40:00.850 Very cool. 0:40:01.250 --> 0:40:09.170 Yeah, definitely. Check this out. I mean I feel like folks out there, you know, especially if you're a hobbyist, this is like something to get in on early, I feel. 0:40:10.360 --> 0:40:18.680 Yeah, I guess that's the what the dozen fart apps on the App Store that made, you know, tons of money in the first year of the Apple App Store. 0:40:19.480 --> 0:40:20.720 Yeah, yeah, exactly. 0:40:20.720 --> 0:40 You need the equivalent the Chart GPT chat. I tried to make it chart fart GPT. I don't know Fart GPT. Okay, nevermind. 0:40 --> 0:40:29.960 This is not Chart GPT. 0:40 --> 0:40:35.120 Oh boy. All right, let's just move the book at the show. 0:40:35.120 --> 0:40:38.860 We'll have to get rid of the friendly tag or whatever. The safe for work tag. 0:40:38.860 --> 0:42:25.520 I don't think we have a friendly. No. Anyways, my, my book of the show just hard pivoting segue. Forcing a segue is. I'm repeating the will of the many. I had this book, I believe a couple episodes ago. I had only just started it. I have finished it. It was, it was really good. I really like it. So I, I, I felt like I owed the people the like follow up that if I was going to pitch it as a, you know, hey, I'm starting this book. It seems kind of interesting. I did finish it. It is good. You know, and the second one comes out. Probably by the time this episode's out, it'll, it'll sort of be out. So if you haven't started this book and it sounded interesting from my last pitch, I guess I could re pitch it. But anyways, so you Know, it definitely has like an equivalent of kind of a magic system, a loose, loose magic system where people can sort of seed their. Something called like a will, like sort of part of their like energy, their life energy, I guess, isn't. Hasn't been you know, fully explained yet. And they can sort of give it to somebody else. And those are assembled into their government in a kind of form of a hierarchy, a pyramid. And it's all about sort of the dynamics, the politics of, of that. And so of course, you know, a story about a boy who's an orphan and ends up getting, you know, sucked up into intrigue and mystery and all of that good stuff. But by the author, James Islington, who I've recommended other books from before. Definitely something that I would recommend now having finished it and I'm looking forward to the next one. I guess I could have made it my book of the show, but the next one seems like it's going to be really good as well. And I'm looking forward to reading that. 0:42:26.030 --> 0:45:24.480 All right. Yeah, I'll definitely have to check that out. I need to get more into reading fiction. My book of the show is actually a podcast episode. I'm cannibalizing our show here. But basically I had a long drive. So long story short, it was just me and the boys, me and my two sons, and we were driving to Houston and back and you know, they're playing, you know, Nintendo Switch or whatever they want to do. So I'm like, I'm listening to a podcast and I listened to the whole six hours of this podcast with dhh, who's the person who made Ruby on Rails. I thought it was fascinating actually. I kind of, I kind of went in thinking, well, you know, this is somebody I have kind of heard about. A friend of mine told me we should have DHH on our show. That would be kind of cool. Dhh, if you happen to be listening, which I doubt but. Or somebody who knows him or something. Totally down for that. But you know, I'd heard of him and I remember specifically him posting a lot about getting off of the cloud and he ended up moving to like a colo. So, you know, I thought it'd be kind of interesting. But I had I honestly a little bit low expectations of just any podcast episode where it's, it's, it's just kind of a one sided where the person just basically talks for six hours. Um, but I was like blown away actually. I felt like the content was really interesting. Um, actually the, the whole. It really goes against a Lot of the sort of like common themes that you're seeing a lot of. So for example, dhh, like despises going into any office. And so he's like, you know, remote work is the only work. It's a really interesting perspective. You know, he talks about how to make remote work work. Well, he's been doing this for like decades, right? So he gives a lot of perspective on that. They, the, the company, which I think is called 27signals, they took $0 of funding. So they basically bootstrapped themselves by doing, by doing contract work. So they did contract work until they had kind of enough money. And I guess they maybe they structured the contract work in such a way where they could share the IP and then eventually open source the IP and it became Ruby on Rails and all of that. So $0 of venture, $0 of anyone investing. And he said something kind of interesting. He said, you know, people gave me a hard time because they say, well, you know, you. 0:45:25.679 --> 0:45 So. 0:45 --> 0:47:01.900 So their, their product, which I forgot the name. Oh, Basecamp. So their product is Basecamp, which is similar to Jira. And people say, look, you could have made Jira. You know, your product is, is actually like in many ways better than Jira. And Atlassian is like a giant company and you're this tiny company. And his response was interesting. It's basically like it's a tiny company, but, you know, I own half of it and like Jason, who's his co founder, owns the other half and, and actually like owning 50% of a tiny company is awesome and I have more than enough money and all of that. And it was like a real kind of like, I guess, like, it is also like, I keep wanting to think of the word monastic, but. But it is kind of like totally antithetical to everything you see on social media. Everything on social media is, look at this big round we raise. You know, no one ever says, look at this $30 million we raised. We only had to give up, you know, 70% of the company. Like no one ever says that part of it, right. And, and everyone seems to be just extolling all the virtues of being in person. And yeah, I feel like just to have a different perspective, it was pretty mind blowing. So I highly recommend folks listen to it also. You know, my dad was a race car driver, so he spends about an hour or two talking about race car driving, which, you know, hits close to home, which I loved hearing about. That's more of a personal thing, but I found the whole interview fascinating and I'd highly recommend Folks check it out and it's six hours, so I think it counts as a book. 0:47:04.220 --> 0:48:26.080 Fair enough. Yeah, I think 37signals is private. I've looked before. I was just looking again to see. But I guess one of those interesting things about private versus public and to your point, not only they have a lot more discretion, like just don't have to share a lot of information and get judged about decisions. And so because he doesn't have investors or all of this, like even people you know, here are some hacker news post and they're just saying like can we even estimate how many subscribers they have, how much money? And people are like throwing out numbers, but they're all, you know, just speculation or, or, or you know, based on some, you know, sort of swag at it. Like they're not accurate. And it's just kind of interesting to think about like the amount of stress reduction that would give to someone to say we just gotta make the decision, can we continue to pay the people who work for us? Can we continue to make money that we're happy with? And like there's always that question, like, what's enough? You know, and like if you don't have an answer to what's enough, then maybe you just end up in one of these quests to, you know, seek power and fame and fortune and, and that's okay, I guess. But if you can say this is enough and, and he is saying, you know, I can do these really cool things, drive race cars and have this business that doesn't make me hate my life, then you know, that, that is awesome. It's great to see like you said, sort of another way instead of it. Only that there's only one way. 0:48:27.120 --> 0:50:28.860 Yeah, I mean somebody said the, the, the host of the podcast said, you know, hey, you know, one thing about working remote is, you know, it's kind of lonely. How do you, how do you deal with that? How do you meet people? And, and the guy was like, dhh, is like get married, have a family. He's like, yeah, I have kids running around the house. I'm never lonely. You know, it's like these are the kind of things that like, just like, just a different perspective. Like I've never heard anyone give an answer like that. And you know, it's so different. I'm still honest. I mean I did. We just got back from Houston, so. So I haven't had a lot of time to really like sleep on this but, but it's such a different perspective that I have to really think about it. But I think there's definitely nuggets of wisdom there. Absolutely. And, um, I do feel like, you know, I've kind of ping ponged between, you know, I was in the office Covid hit. I worked remote for what, four years now. I've been in the office for the past year. And I do actually feel like. I feel like remote is better in many ways, but it has to be done correctly and you have to be in the right situation. So I think, I think remote, where there is an office and like 90% of the people are in it, I think that's hard. Of course you can make it work. But I think socially, like, it takes a lot of. You have to be very active on managing that. I think remote, where everyone's remote or in the office, you know, it's, it's. You're on zoom all the time anyways, because there's always. This is. Okay, this is, this is my perspective. Everyone's remote. Everyone's remote because you go into the office. But there's the Seattle office or there's Bob, who's remote. Like as soon as, as soon as you have more than one office or a single person in the company who's remote. Now you're remote because you're on zoom all day. 0:50:28.860 --> 0:50:57.300 Anyways, I guess my, my thought here is similar to what I was saying before. I. It's. I don't think there's one right way. I think, to your point, it's situational. It's company by company. It's person by person. Some people hate, like, I couldn't be at home, it's noisy. Other people, like, I can't stand the office, it's noisy. People dropping in on you. Like it's, you know, for almost any statement you can, someone will say the same statement in positive and negative light. 0:50:57.540 --> 0:50:58.020 Yeah. 0:50:58.180 --> 0:51:23.630 And so like anything, my. How do you want to say? My thought is you're going to have to try really hard to prove that there's a singular right answer and no nuance. Anytime someone comes and says remote work is the worst, it doesn't make sense. It never works. Or if someone says and the office is dumb, it never works. We should all be remote. Like, whenever you make a statement with no wiggle room, those are always the hardest to defend. 0:51:23.630 --> 0:51:23.950 Right. 0:51:24.030 --> 0:52:07.540 And it's like, come on, there's. There's gotta be a gradient here. And you know, and if there's a gradient, I think that, you know, I will say, even being people fully in the office, having the setup that we could do fully Remote means people have some flexibility to do if they need to take a week, a week. They may have taken a sick week or something away or they needed to, you know, I was pointing out, be home. There's so much random stuff where people expect you to be at your place of living for delivery or for some improvement or to come in or whatever and then they change the date. So having some flexibility to handle those situations has been a, A net benefit regardless of sort of what percentage allocation you spend, office or at home. 0:52:08.100 --> 0:52:10.660 Yeah, totally makes sense. All right. 0:52:11.380 --> 0:52:15.140 Tool of the show. Oh man, I think I'm losing my voice. It's not good. 0:52:17.220 --> 0:52:17.780 All right. 0:52:17.860 --> 0:53:49.900 Mine is a, is a game and it's a game. I am almost certain I didn't look because I didn't want to know, but I'm almost certain that I've recommended before. Everyone knows it. Everyone has it, but I don't care because I got sucked back in. I was, I was free. And then I am no longer free. I've been playing Factorio again and not even, although it looks amazing, not even the space age dlc. I'm just back to playing Vanilla Bass. No Mod Factoria. I've been playing it on my switch and it has been, it was. It's always like a little awkward. It's so much better on the PC with the keyboard, I will admit. But once you get into the groove and you're like on an airplane, which, which I was recently, a couple times, you know, can sit on the couch with the family, you know when them watching some show you just really don't want to watch for the fifth time in a row. Which, whatever. Anyways, like you, if, if you can quote the show and like you know what's going to happen from the kids show, you know, you've seen it too many, many times, which means they've seen it even more than you because you know you haven't watched it every time they've watched it anyways, I'm sucked back in Factorio. If you've not played it, you either are going to assume it is either going to be work or amazing or you won't understand it. I guess those are the three, three outcomes. Be like, this is just like work. It sucks. I don't want to work. Or you're going to go, I don't get it. This, this is like weird and, and just like this is not fun. Or you're going to be like, what? Yeah, fugue. You're gonna spend 10 hours and then you have no clue what happened. 0:53:51.340 --> 0:54:29.750 You know, I posted about a month ago, I. I had a friend, a close friend, who's trying to grow his Twitter presence and just actively trying to do this. And he posted saying, you know, I went from 1300 followers to 7000 followers in a month. Here's a list of things I did. And it was. It was a list of things. And I replied and I was like, how? I went from 1300 followers to 1300 followers in three years. And it was a screenshot of Factorio. But Factorio is amazing. I mean, it's. It's. It's so satisfying. It's like, gotta be one of the most satisfying feelings, like building a factory. 0:54:29.990 --> 0:54:42.040 I've. I have a confession. I have played probably. I don't want to look how many hours. I don't actually. Maybe the switch does track. I don't know how to look. It's not prominent like on Steam, so I'm not sure how many hours I play. I've never launched a rocket. 0:54:42.680 --> 0:54:44.520 What? Really? 0:54:44.760 --> 0:54:54.360 Is that bad? It's bad, right? I need to do it. It's become like a goal. Like, I. Before I stop this time, I have to do it. I just always do other stuff. I just. I don't know. 0:54:54.440 --> 0:55:06.360 Wow, that's amazing, though, actually. It's amazing that, like, you have built all these intrinsic rewards, you know, it's like, oh, can I. Well, like. Well, give us an example of something that took you a long time. That wasn't rocket. 0:55:06.670 --> 0:55:49.710 No, it's not. No, no, it's worse than that. I play for a while, I start to get to, like, a good. And it. And it reaches like, there. There's always some place and it's always a little different where it. You need to, like, do a bunch without, like, a good progression so you can, like, progress early game progress. And I'm getting better, I think, actually, each time. Then I go away for a while, I'm like, I don't have time to invest to, like, double my output here, build another one of these or, you know, get a blueprint working or get the bots going, going. So I, I just, like. I put it away for a while or I get busy and then I come back, I'm like, I don't remember where anything is. So I just start a new game and then I just, like, start over and I just, like, do with all the improvements. So it's just like, whatever you call it, like, greasing the groove. Like, I just keep playing the early game over and over again. 0:55:49.710 --> 0:55:52.230 I know, yeah, that makes sense. That Makes sense. 0:55:52.230 --> 0:55:58.130 It's bad. I'm a bad person. Like, I need to finish the game. There's so much the developer put in I haven't even experienced. 0:55:59.240 --> 0:56:12.120 My kids play with creative mode where you're not even a person. You could just build anything anywhere, and it's unlimited. And you kind of, like, teleport around the map, and they just build random things with it. 0:56:12.440 --> 0:56:16.720 I turn the biters. Like, the biters are there, but they won't attack you unless you attack them. 0:56:16.720 --> 0:56:18.920 Because I just, I don't know, it's. 0:56:18.920 --> 0:56:20.360 Too anxiety inducing, man. 0:56:20.680 --> 0:57:09.390 Well, you know, I mean, you probably know this, but the construction robots just totally change the profile of the game. Like, like, I, I, I have mine. Like, near the end game, mine will be set up where I'll have basically stamps that, like, stamp an entire screen's worth of content. And so it's like, like, four robo ports, a bunch of these, like, electric towers, and then a bunch of solar panels, and it just stamps down. And then, because the robo ports are part of the stamp, you know, it builds the, the robo ports, and then it can build the next stamp. And so I'll just walk across the screen stamping, like, hundreds of these. And then I'm like, done with power. It's like, okay, well, I don't have to think about power for the rest of the game. Nice. All right, my tool. 0:57:09.470 --> 0:57:10.270 Oh, go ahead. 0:57:10.590 --> 0:57:17.510 My tool to show is Nip IO, which is apparently a great way to get in trouble at work as part of. 0:57:17.510 --> 0:57:21.970 Yeah, I still have no clue what this is. So this is like. 0:57:23.810 --> 0:59:50.460 I could see why workplaces wouldn't like this, but I'll explain what the tool is. So the idea is there are many places where, you know, you can't use just a IP address. You have to use a host name. And so you end up in your code writing all these things like, oh, if I'm local, then use, like, this IP address. Otherwise use this host name. Or you end up in weird situations where, like, if you're running on kubernetes, you know, then the host you want to talk to is just called database. And you can, like, ping database, you know, curl database, like from the other nodes in, in that Kubernetes cluster. But then when you're not running in kubernetes now, it's like, oh, no, it's actually local host, and it gets all confusing, right? And so this Nip IO is like a cool way to turn any IP address into a DNS. So basically, you take the IP address, whatever it is, you know, 1.1.1.1 and then you add.nip.IO at the end and now you have a DNS, but it's going to resolve to exactly that IP address. So you know, you're kind of trusting. I will say that for, for the reason probably is blocked for Patrick. Patrick's job is that you're kind of trusting these people to convert that DNS to the correct IP address. So you know, you might put in like 2.2.2.2. Nip IO and it resolves to 2.2.2.2. Because the people behind that website have built like a DNS lookup that does that. But they could change that code tomorrow. They could make it so everything nip IO goes to Google or goes to a web based Bitcoin miner or whatever like that. So that is sort of a vulnerability that you're taking on. It is, you know, it's, it's powered by a company that seems like pretty reputable but, but that is a risk. But you know, when you're doing develop debugging stuff, especially locally on your laptop or you're getting started with a new project, it could be an easy kind of convenience. 0:59:50.460 --> 0:59:57.260 Yeah, that's not used it before, but I could see. Yeah, like you said, there are specific use cases where it would be really useful. 0:59:58.300 --> 1:00:12.540 Yeah, totally. There's a lot of code, I mean especially third party code that expects a domain name that just will not take an IP address. And this is a way to get around it. All right, so. Oh, go ahead. 1:00:12.540 --> 1:00:17.380 No, no, no, I just. Yeah. Anyways, Orchestrator. 1:00:17.380 --> 1:00:21.820 We're gonna orchestrate the show here or the workflow Orchestrator. 1:00:22.220 --> 1:00:25.180 I'm distracted today, dude. We're just like, I just like barrel forward. 1:00:25.340 --> 1:00:30.620 We're gonna just fuse. Fuse this fugue. We're gonna fugue it. 1:00:31.260 --> 1:00:32.780 Oh, I'm never living that down. 1:00:33.900 --> 1:06:09.350 It's all fugued up over here. Um, so, so okay, let's talk about why you need a workflow orchestrator. So here's a very simple example and actually kind of spoiler alert. I actually asked this as my interview question. So if, if you ever end up getting interviewed by me, you're going to probably get this question. Um, let's say you're building, you know, YouTube and specifically like worry about like the social networking or any of that, but just, you know, you upload a video, it turns transcodes your video a hundred different ways for mobile and you know, low bandwidth and all of that. And then when other people go to that URL, they get your video. So video is maybe not the best example of this because it takes a while to upload the video. But know you, you, you upload the video and you know, then it has to go through and do a whole bunch of work, right? That transcoding of that video to other formats, maybe you uploaded it as a quicktime video, which, you know, they don't want to serve that on the web, right? So that is a time intensive process. And so it's not practical to like just have someone wait on a web request, right? Because if that person just hits F5 or something and closes the connection, you know, how is that going to work? Right? There's kind of two problems here. Problem number one is sometimes you have to do things that take a long time and then problem number two is sometimes you have to do things that are kind of non interactive or someone will leave you need this thing to keep working and eventually report back in some way. And the last category of why you'd need this would be, you know, you have a lot of things, a huge batch of things that you need to do. So for example, actually this, this is a very real example for us. I, I went through and re o not OCR re transcribed all of the episodes of programming throwdown. So if you look at the transcriptions, they're, they're better than they were before. The old system I was using was massively out of date and I felt like, well since I upgraded it I might as well just re transcribe everything. And so you know, we have 180 some odd episodes, 184 episodes. And so that actually is expensive but even if it wasn't, it's you know, a batch job that I have to just submit a hundred of these and just tell me when they're all done. Right? So that's the basic reason why you'll want to use a workflow orchestrator under the hood. They all work more or less the same way. So there's you know, a set of message queues and so I think we've talked about pub sub and these things in the past, but basically you know, you need a way to submit jobs and the jobs need a way to read your submission. And so there's needs to be some kind of queue to handle that. If the job is multi step then when the first step is done now, you know, a new thing needs to be pushed onto this message queue saying okay, we're ready for the second step, right? A job can fail you know, like AWS could shut down that machine, the hard drive could fail on that machine, et cetera, et cetera. And so message queues are important for there's, there's a lot of logic around. Basically you send a keep alive to the queue while you're still alive. And if the queue doesn't get a keep alive for some amount of time that you specify, let's just say 10 minutes, it assumes that you're dead and it will rerun, like it will ask another machine to go and process that. Um, okay, so, so message queues a big part of it, and containerization another really big part of it. Because, you know, you need a way to define some task on your computer, but then ultimately send that task out to the cloud. And there might be hundreds, maybe thousands, tens of thousands of machines kind of running your task. And so it's not practical to like sudo apt, get, install some package all those times. So there needs to be some type of containerization and then also the workflow. So the thing that, you know, tells you know, the overall system what tasks need to be run and which ones depend on which other ones that also needs to be put in the cloud somewhere and containerized. And so that's the second ingredient. And the third and final ingredient is some type of worker pool. So some type of, you know, pool of machines that are ready to do work. And you typically pair this with some type of auto scaling. So if you have, especially if you have bursty work. So imagine, well, YouTube is so such a multinational thing that this wouldn't happen. But imagine if you were building a version of YouTube and you're just getting started where it's maybe way more popular in the U.S. you know, you'd want to scale those nodes down when it's the wee hours and our time zones, and then scale them up when it's peak hours. 1:06:09.590 --> 1:06:18.070 Yeah, I think that becomes important too for, you know, something going viral. So if your website goes viral, you don't. You want to capture the signups, you want to capture whatever. 1:06:18.070 --> 1:06:18.350 Right. 1:06:18.350 --> 1:07:38.450 So, but I think in that case, if, if the things people are signing up are doing are kicking off these, these workflows, I need to circle back to, you know, talking about failures. But I think there's a couple things maybe that are less obvious or, or maybe important about these. So one is you were mentioning, you know, something like YouTube size, but when you're kind of small, things generally fail because you did something wrong or there's a bug or a problem. But when you reach A certain size. Even if, you know, if you look at what, what I guess is called meantime between failure, like you said, a hard drive or cosmic ray or whatever it may be. But when you're running a million, you know, jobs a day, what is the chance that at least one job is gonna, you know, have an issue in a day? It's, it's, it's non trivial, right? It's, it's noticeable. And so you're getting these failures because someone is walking into the server room and pulling out a server rack and like blowing the dust out of it for maintenance or whatever, right? And you can either have that be very carefully planned or just say, look, it's, it's gonna get handled. And I think kind of that is the same vein as you see stuff like, I think Netflix was famous for doing chaos Monkey. Something that goes in and just basically kills off nodes or drop nodes to make sure, you know, nothing, nothing breaks. It just randomly sews chaos. 1:07:39.489 --> 1:07:42.130 I think that was, that was my job when I was there. 1:07:46.690 --> 1:07:58.140 That is my job in my family. I'm just, oh, dad's here. Okay, wow. Been distracted today. 1:07:58.300 --> 1:07:58.700 The. 1:07:58.780 --> 1:09:04.240 But the other thing is, you know, we were talking about, you know, and I think we get to it in a minute. But, but something like training an LLM and it's, you know, $10 million. Some of these workflows could be very, very, very expensive. And like you said, imagine there's some error, but imagine the error occurs at like a very late stage. You don't want to start all the way again from the beginning. So being able to kind of, it's not exactly hot swap, but being able to resume the pipeline with a new version of like a later node or even you just, maybe it's not an error, maybe the whole thing finishes. But then you realize, oh, here in this last stage or this last. But we could do better or we could do something different. So having a system that automatically is like preserving those stages and inputs and you know, that kind of stuff so you can sort of rerun later parts becomes like super useful. And tying that all with the dashboard and management, which is things I have seen people not use one of these and then you end up just basically reinventing them. So you can either like not use them or you can reinvent one with half the features that you do yourself and you just slowly try to rebuild a crappier version of like an existing solution. 1:09:04.880 --> 1:12:18.160 Yeah, totally, totally agree. Yeah, I think the part you mentioned about history and backfill is, is super important. Like there have been so many times where exactly what you said happens. You know, the job runs successfully, but it wasn't actually a success. Like we actually went the past three days with no artifacts because, you know, maybe the upload failed, but it didn't let the job know that something was wrong. So, so for example, like a very common pattern is where you catch an error in Python or in any language, you catch the error and then log an error message. And that's great and all, but because you caught the error and you didn't bubble it all the way up, now your job thinks it finished successfully. Like if you didn't set the exit code correctly on your program. And so you know that that can happen and then you find out about it. Even if you find out about it an hour later, if your website's popular enough, that could be millions of jobs that have run in that hour that need to be rerun. So, so having all of that ledger and then being able to backfill and say, okay, you know, all the jobs from, you know, three days ago to now, I need to rerun all of them. And it just goes through and, and does that and auto scales to handle it all. That, that's, that's, and there, there's sort of this, it's kind of like, you know how cars have the, what's it, there's basically like a backup system. So imagine, so like imagine on your hud, imagine your HUD just like disappears. Like the, the, the OS powering, the heads up display on your car fails. It's got some kind of backup where you'll at least see like the miles per hour or something. But now that, yeah, right, but, but that contingency, that's where I was thinking of. There's a contingent operating system in most of these cars. But now that contingent operating system that really needs to not fail. Right? Because now you're really host. And so similarly here, you know, the workflow orchestrator itself, if it starts failing, it's, that can be a really difficult thing to survive. Right. And so to your point, you know, I've seen it as well where people try to build their own and it happens so innocently. You know, the way it happens is, you know, you're, you have some task and this, this happens at big companies, you know, so it's not a small company thing specifically. Like you have some task and it's relatively minor and so you have this script that does it and you know that starts to succeed. So it's like, well, you know, I'm Just going to make a cron job and have it run this Python script every hour. And then it's like, oh, okay, now I need to run it, you know, every minute. Oh, now I need like three machines. So I'll just spin up three ec, two instances, all running this Python script every minute. And you kind of like, at some point you have to just eat the technical debt and like set up a workflow orchestrator. But I've seen too many cases where people kind of go too long and then there, when there is an issue, it's becomes very difficult. 1:12:18.240 --> 1:13 I think the other term you'll hear with a lot of these is, is dag, which isn't unique to these, but that all of these jobs form a direct basic, like no cycle graph. So they're not looping. So they have a start, they process some number of jobs with some interdependencies and then they reach a conclusion. And so you'll hear them, hear them talked about that way. Maybe help me there, Jason. So, and not to, to throw a wrinkle here, but so workflow orchestration versus something that is more seen as like, I would call like a pipeline, like spark or something. What's the sort of distinction, you know, between. Or where's a line? Or is it blurry? Like, what's the difference between something that would be viewed as more like pipeline versus workflow orchestration? 1:13:00.160 --> 1:15:12.680 Yeah, that's a great, great question. I don't have an easy answer, but basically I would say, okay, one thing a workflow orchestrator cannot do is it cannot fuse nodes together. So you kind of explicitly define like, do this task, do that task, is that task, and they all depend on each other or, or in, in a sequence or something. And if tasks like B and C really could be fused into one operation, it won't do that because it, it makes no assumption about what programming language or what version of Ubuntu is running on that node or anything like that. They're all just basically Docker containers that are getting spun up. You can actually, depending on the orchestrator you use. Some orchestrators have it where you can spin up a container for a particular node and that container can do multiple tasks. So let's say I have 10 videos, I have to transcode all of them to QuickTime. I have a QuickTime node and that node could do all 10 of them. Right? But by default, most of these orchestrators, you actually spin up a Docker container, do that one thing that one time, and then tear it back down. And so there is A pretty heavy spin up, spin down time for four tests. So conversely, with Spark, like you kind of create, you do create sort of a dag. And same with early versions of Pytorch or If you do pytorch.compile or you know, early versions of TensorFlow, you would create a DAG of operations. And then what the Spark compiler or Pytorch compile or tensorflow will do is, is it will fuse operations and it will create this like super optimized dag. And it does that because all of the operations are within its own ecosystem. So like, you know, Spark has a limited vocabulary, as does Pytorch. And so this being so open ended, it can just run any Docker container with any arguments you want. That open endedness comes at a cost. 1:15:12.920 --> 1:15:14.050 All right, yeah, that makes sense. 1:15:14.920 --> 1:17:59.180 Cool. So yeah, quick rundown on the steps to use a workflow Orchestrator. Typically you containerize your tasks. That's usually pretty straightforward. Most people are using Docker anyways. But you also have to containerize the workflow itself. So the file that says run this task and then run the second task afterwards, like that, that logic has to go somewhere. And so all of these examples that we'll talk about in the future, they all do it differently. And then after that you're kind of off to the races. You submit jobs. One thing that is, that varies a lot from orchestrator to orchestrator is how to pass data among the the tasks. So so the early versions of these, the only way you could pass information was through arguments. So so you know, job one finishes and then it's allowed to inject some command line arguments into job two, but that's it. And because it's a command line argument, you can't stick like a whole JSON blob in there, a large JSON blob or anything like that. You couldn't put an image in there or something. And so what you have to do for these early ones is use something else to store everything. So you know, the task transcodes the video and then it doesn't pass the transcoded video onto the next task, which may be like sets a flag in a database saying it's ready, but what it does, it puts the transcoded video into some object store like S3 or something else. And then all it passes to the next task is maybe a unique ID of the video. Later versions, they allow you to pass arbitrary data, but under the hood they're doing the same thing. So when you pass like a Pandas data frame or an image or something from one task to the other under the hood. It's actually storing it in S3 in some temporary bucket which has like a low retention policy and then pulling it back out for the next. So. But yeah, that's, that's all there is to it, so it's really not that hard. I would say this is in the category of things that kind of like functional programming languages where it's kind of a unique paradigm. If you've never used a workflow orchestrator, it can look very daunting. But once you've learned one now, you kind of can do pretty much all of them pretty easily. 1:17:59.420 --> 1:18:04.780 I haven't used one, but I, looking forward, I, I, A lot of these names are familiar, so. 1:18:05.580 --> 1:18:10.820 Wait, really, you haven't. Well, what about like a proprietary one or something? I'm sure you've got. 1:18:10.820 --> 1:18:30.310 Yeah, I guess that's true, but I feel like it's a bit different. Like if someone else already has like all of this set up, I've sort of been a part of one of them, I guess. So other people sets up this stuff and I, you know, define some portion of some code that needs to run and someone is sucking it up into a container and running it. 1:18:30.310 --> 1:18:31.390 Yeah, that way. 1:18:31.390 --> 1:18:37.110 But I've never sat down and had to write in the language of describing the workflow as an example. 1:18:37.430 --> 1:19:28.580 That makes sense. Yeah. At Meta we had something called Data Storm, which is very similar to the ones we'll talk about next. But I never worked on Data Storm. I was mainly mostly a customer. But, but, but they all are extremely similar and I think a lot of these cases like data storms. An example there was something called Flume Flume, which was similar to Spark. It's kind of like a combination of Spark and a workflow orchestrator and then that, that someone made an open source version of that. But, but yeah, I think, I think most people will encounter workflow orchestrators. It might just be a homegrown one instead of one of these ones. 1:19:28.900 --> 1:19:29.860 Yeah, that's true. 1:19:30.100 --> 1:25:07.290 But the, the big, the, the big market share ones are, you know, Airflow. Airflow is an Apache project. I don't know the history of Airflow, but it's kind of the 900 pound gorilla in the room. So if you're on AWS, there's something called N Mwaa which is managed. I don't know what the W is, but the A's are Apache Airflow. And basically you don't even have to set up airflow yourself on any computer or anything like that they just do everything for you. It's kind of like RDS for databases. So RDS example, like you don't have to like install postgres on a machine yourself or anything like that. They have the same thing for airflow. Airflow is, you know, legacy. It's, it's doesn't have a lot of the features of the other ones we'll talk about, but you know, you can feel confident that airflow can do almost anything. Like if you can't get it done in airflow, what that means is of the thousands and thousands of developers, nobody's tried to do what you're trying to do, which is, may be a sign that you're doing something wrong. It's kind of in that category where too big to fail, I guess. But, but it is legacy. I think it's originally in Java and because airflow is so open ended, you know, it's not hooked into any one language that, well, I would say is my biggest detractor of airflow. The some other examples I used Dagster a lot. Dagster is quite nice. I think it's PYTH only or if it's not, there's maybe a REST API that would be pretty ugly to use, but it's really meant for Python. It's nice. It gives you little decorators. You can decorate functions as Dagster nodes and so when a function calls another if, if, if it calls a dagger node as, as a, you know, function, it'll return back a future and Dagster kind of handles all of that for you. So it kind of pauses the operation of your function, calls that dependent task and then comes back and gives you your function back, which is pretty clever, pretty cool. So temporal is they actually just raised, I think a series B. I'm going to double check. Yeah, they raised 2.5. Oh, 2.5 billion is their valuation. Series C they raised. And it's pretty cool. So it's, it's, it's kind of a different take. You don't have the dashboard, you don't have a lot of the things we talked about that are important, but it just gives you the bare thing that really matters, which is being able to run a function and know you're going to get an answer regardless of how long that function takes. So temporal would be a good thing if you're building your own workflow orchestrator. I think temporal would be a good thing to build it on top of. And it might even be that Dagster uses temporal. It's very Possible, but it is kind of like lower level, you know, really calling individual functions durably similar to temporal. There's Apache, Ray. So Ray comes from this company called Any Scale. Ray is also pretty low level. So, you know, you're calling individual functions and it has to be kind of in Python. And so it's, it's hooking in at a pretty low level, kind of like temporal. The thing that's really powerful about Ray is just how unbelievably fast it is. So with Ray, you spin up all the machines in advance. So you know, all these other workflow orchestrators, you know, you're spinning up a machine, doing some work and then spinning it back down. With Ray, you spin up a whole cluster. When that whole cluster is up and running, you then execute a bunch of functions and those functions get farmed out to that cluster. You span up for that particular task you're doing. And so it's a fully connected mesh of machines. And then when you're finished your task, that entire cluster gets spun down. So you're paying a bigger hit up front, but you're getting like super, super low latency. Like, you never have to worry once the whole cluster is up, you never have to worry about any node requiring another machine or anything like that. And the final example I have, which, which folks should know about is kubeflow. Kubeflow is similar in a sense to Airflow. The big difference is they've integrated a dashboard where you can upload different statistics. So, for example, airflow is a dashboard where you can monitor the jobs and see, oh, I have five failed jobs yesterday. Kubeflow has that too. But kubeflow allows you to, like, each run, have a dashboard for that specific run. And so it's like, okay, task three, you know, output a time series graph, and here it is. Task four, created a scatter plot and there it is. And so for workflows that are producing statistical outputs, that's like a nice feature to have. 1:25:07.370 --> 1:26:07.180 It sounds like some of them are, like you said, sort of lower level. Like, Ray sort of sounds like it's short, shorter, not shorter. You want higher, you know, response, lower response times, lower latency. You want to be able to do lots and lots and lots of things. You want to sort of shard them all up and have them sort of distributed. And then other things, like you were sort of saying, like transcoding a video or something, you pretty much want to like, do a bunch of data reading, do a bunch of processing, kind of do a bunch of data writing. You Know, as far as like externally, so you're sort of dealing with large blobbed objects and very long run times. And so it feels, it feels like there isn't a generic solution that just sort of like meshes over all of it, which, which I guess kind of makes, kind of makes sense. And as you were sort of saying, I think whether one sort of container can be multipurpose or whether you really are needing different containers for all of the different jobs probably plays a huge role and sort of like how easy. 1:26:07.180 --> 1:28:58.640 Or hard it is. Yeah, yeah, totally. It's one of these things that like, the lines are a little blurry because I'm sure, I'm sure someone's going to email us saying, oh well, you could do actually this in airflow or you know, Ray could also do that. But just generally speaking, like at the baseline level, you know, Ray is not durable. So if, if your Ray job, you know, you're responsible for doing checkpointing and things like that, you know, if your RAID job dies halfway through, it's just gone. Whatever checkpoint you have is whatever you created. You know, these other ones, you know, they'll, they'll just by their nature, they're constantly creating checkpoints. And so yeah, there's basically several things you're juggling. You know, how durable do you want these tasks to be, these parts of your workflow and then what's your latency requirement? And then what, what's your sort of cost you're willing to spend? Because Ray, you know, for every. So just to give an example, let's say you have some type of task and that task has a list of Python third party libraries that need to be run, need to be included. So if you do Airflow, you know, when that task node spins up, it spins up your Docker container where you've already put those libraries in it, right? And it's ready to go and it executes your code. And that's fine with Ray. You create a Docker container with those dependencies similar to, with Airflow, but you spin up an entire cluster and every node on that cluster has all those Python dependencies, you know, so if you have especially like heterogeneous things where maybe you have one task that requires Torch, which is like a 2 gigabyte pypy file, right? And then you have another task that's just send an email, you know, that only requires like the base Ubuntu, you know, then Airflow is going to save you a little bit there. So yeah, there's no free lunch there's like kind of several things to juggle, I guess. So some advice I think Ray is, is you should only use Ray if latency is important. So if you're doing ML training, latency is very important. You have this gpu, it's very expensive, you don't want to waste time on it. So that's why Ray is extremely popular for training these AI models. If you're doing ML stuff or things where you're producing a lot of, you know, graphs and a lot of those artifacts, I think kubeflow could be a good choice. And for everything else I would use Dagster if you're doing IS in Python. Otherwise I would use Error flow. So that's sort of the workflow diagram of which workflow orchestrator to pick. 1:28:59.520 --> 1:29:05.400 Yeah, all prescribed out for us. I guess this is called a flowchart, but yeah, okay. 1:29:05.400 --> 1:29:47.430 Oh, that's true. Cool. Yeah, I guess, you know, some like things to avoid other than rolling your own orchestrator is I think you can go the other direction and kind of make things super complicated with, you know, auto scaling and a lot of these other things. I think, you know, you can take baby steps with these things. Just like anything, you can kind of start getting into it and end up diving super, super deep and saying like, oh, I'm going to preload the nodes with these packages and at the end of the day like, you know, this, you want this to save you time. You don't want this stuff to cost you a lot of time. 1:29:47.590 --> 1:29:49.270 Makes sense. Thanks. Thanks for the overview. 1:29:49.510 --> 1:29:51.070 Yeah, Deep dive. 1:29:51.070 --> 1:29:52.230 Yeah. The education. 1:29:53.910 --> 1:31:22.800 Yeah. This is a good thing to learn, folks. I think this is super, super useful if you are like looking to get started. Imagine you have like a. You have a website that's, that has any type of job that needs to get done. Trying to think of a good example like, like you have a website that just ocrs pictures. Someone puts a upload, drags a picture into your website and you send them back some text or something in an email. And nowadays OCR is so bloody fast, like you can just respond instantly but let's pretend like we're still where. That took a long time. You know, just spin up temporal. You know, it's. I think it's relatively easy to stand up. I think they have a managed service too that's very reasonably priced and say, look like I'm going to call this function to do the OCR and that function is going to do the OCR and send the email and I'm just going to wrap it in temporal, and now it's durable. And that could be just an easy way to get started. Cool. All right. Yeah, we hope you folks like the episode and definitely keep sending us, you know, requests for new content, new show ideas. We take them super seriously. We've implemented many of them, so our list is getting smaller. So if you have a show topic, please slack us or discord us. I mean, email us however you want to reach out to us. 1:31:22.800 --> 1:31:23.480 Thank you, everyone. 1:31:39.520 --> 1:32:01.120 Music by Eric Barndaler Programming throwdown is distributed under a Creative Commons attribution Share alike 2.0 license. You're free to share, share, copy, distribute, transmit the work to remix, adapt the work. But you must provide attribution to Patrick and I and share alike in kind.