1 00:00:00,000 --> 00:00:08,479 Welcome to the Maintainable Software Podcast, where we explore the art of improving existing 2 00:00:08,479 --> 00:00:12,460 software with seasoned practitioners who have helped organizations overcome the problems 3 00:00:12,460 --> 00:00:17,719 often associated with technical debt and legacy code. I'm your host, Robbie Russell. On this 4 00:00:17,719 --> 00:00:23,139 episode, Obi Fernandez, who is a software engineer, speaker, DJ, author, and is currently 5 00:00:23,299 --> 00:00:27,680 chief scientist at Olympia. Obi joins us from Mexico City, Mexico. 6 00:00:36,619 --> 00:00:39,520 Obi Fernandez, we're so glad to have you join us on Maintainable. Welcome. 7 00:00:39,939 --> 00:00:41,220 Thanks, Robbie. Glad to be here. 8 00:00:41,599 --> 00:00:45,459 So as you reflect on your experience in the industry, what do you believe are a few common 9 00:00:45,580 --> 00:00:48,720 characteristics of, dare I say, well-maintained software? 10 00:00:49,159 --> 00:00:53,139 Oh man, we're just getting right into it. I mean, definitely having some test coverage, 11 00:00:53,279 --> 00:00:57,020 because if you have test coverage, you can make changes. But how boring must that answer 12 00:00:57,020 --> 00:00:58,700 be? I'm guessing every guest says that. 13 00:00:58,700 --> 00:01:00,060 Some variation of that. 14 00:01:00,560 --> 00:01:05,860 But one of the reasons after nearly two decades that I still love Ruby and still insist on 15 00:01:05,860 --> 00:01:10,800 using it, even these days where almost everything that I'm personally involved in is done in 16 00:01:10,800 --> 00:01:15,680 Python, it seems like. I think Ruby syntax leads itself, especially for someone who's 17 00:01:15,680 --> 00:01:20,800 been doing it for so long since me, you know, like me, is to write readable code. And I 18 00:01:20,800 --> 00:01:25,099 think that's a very important factor in your software being maintainable is, you know, 19 00:01:25,099 --> 00:01:30,279 can I go back to what Obi from six months ago or Obi from a year and a half ago wrote 20 00:01:30,459 --> 00:01:35,559 and figure it out? And I think my track record's pretty good on that. And I think Ruby is to 21 00:01:35,559 --> 00:01:41,720 blame for some of that in terms of writing code that is fluent and, you know, reads like 22 00:01:41,720 --> 00:01:44,680 natural language. And yeah, I mean, that's my answer. 23 00:01:44,779 --> 00:01:49,980 I think about also you mentioned test coverage and precursor to this conversation, you had 24 00:01:49,980 --> 00:01:54,059 also mentioned that code that's like robust code. And I was kind of curious, maybe hear 25 00:01:54,059 --> 00:01:58,739 your thought on like, what makes code robust? Like, how would you kind of delineate that? 26 00:01:59,279 --> 00:02:02,660 Well, I think if you've been doing this long enough, we're all familiar with like the code 27 00:02:02,699 --> 00:02:07,120 that blows up for no, for no explicable reason. I've gotten a lot of that in the last year, 28 00:02:07,139 --> 00:02:13,500 the app that I'm currently working on is called Olympia. And it, it probably is the most post 29 00:02:13,500 --> 00:02:18,979 modern software I've ever written. I've hit a new peak of like external dependencies with, 30 00:02:18,979 --> 00:02:25,940 you know, all the different APIs and kind of offloading major subsystems to providers and 31 00:02:25,940 --> 00:02:32,160 things. And even the main one, which is OpenAI, which I use for, you know, for GPT-4 for the main 32 00:02:32,179 --> 00:02:38,660 conversation loop of Olympia. It's the engine behind how our virtual assistants communicate 33 00:02:38,660 --> 00:02:43,800 with our users. And over the last six months, I think I've done a pretty good job of making 34 00:02:43,800 --> 00:02:50,039 the conversation loop robust in the sense that pretty much anything that OpenAI throws at me in 35 00:02:50,039 --> 00:02:55,179 terms of error messages, I can handle gracefully, you know, whether it's retrying or giving feedback 36 00:02:55,179 --> 00:02:59,979 to the user about what's going on or alerting customer support that they can be on top of it. 37 00:03:00,160 --> 00:03:03,940 But it's taken a while to get there. There's definitely been an investment, you know, 38 00:03:03,940 --> 00:03:08,479 in the less mature that the services that you're integrating with, the more uncertainty you tend 39 00:03:08,479 --> 00:03:12,479 to have in terms of what's going on. Like I can think off the top of my head of like, 40 00:03:12,479 --> 00:03:16,660 probably at least six or seven different ways that a request to OpenAI can fail, 41 00:03:16,759 --> 00:03:20,779 you know, at the moment. That's interesting. You know, I was thinking about, you know, 42 00:03:20,800 --> 00:03:25,300 definitely we'll get into talking about your book and stuff like that and definitely talk a little 43 00:03:25,300 --> 00:03:29,339 bit about the software project you're primarily focused on right now, Olympia. But when you're 44 00:03:29,339 --> 00:03:33,720 integrating with external dependencies, APIs on any software project can be a little challenging. 45 00:03:33,779 --> 00:03:39,940 But I don't know that I've, because I personally haven't done much integration at all with any of 46 00:03:40,639 --> 00:03:44,820 like, one, it's trying to find good useful use cases for particular projects we're already 47 00:03:44,820 --> 00:03:48,339 working on. And it hasn't just been necessarily a big requirement for most of our current client 48 00:03:48,339 --> 00:03:53,119 base. But when I try to wrap my head around, like how do you validate if it's a good response or not 49 00:03:53,119 --> 00:03:56,779 from something that's, it's not like a data point that you can easily just check, like this thing is 50 00:03:56,779 --> 00:04:00,220 a hundred percent accurate data point. And this, you know, how do you think about that when you're 51 00:04:00,220 --> 00:04:03,740 having a conversational based back and forth with an API and stuff like that, and it gives you 52 00:04:04,279 --> 00:04:08,839 something that's relatively confident in itself, but is this actually a match that you're saying, 53 00:04:08,839 --> 00:04:11,119 pulling back weird areas, but like, it's like, 54 00:04:11,740 --> 00:04:17,720 Oh man. So, so I would, I wasn't even talking about the, I wasn't even talking about like the 55 00:04:17,720 --> 00:04:22,440 qualitative, you know, kind of more subjective, you know, correctness of the response. I was 56 00:04:22,440 --> 00:04:28,320 talking about whether the response even worked to begin with. I've had to familiarize myself 57 00:04:28,320 --> 00:04:33,579 with a whole gamut of, of HTTP error codes that I was, you know, kind of had forgotten for a while, 58 00:04:33,579 --> 00:04:40,179 like 426, 427, you know, different kinds of five hundreds. Cause we get them all. Cause I mean, 59 00:04:40,959 --> 00:04:45,320 there's, I'm picking on OpenAI, but it's just because, you know, we, we have a, they have a 60 00:04:45,320 --> 00:04:50,820 fair amount of instability in their API. I think sometimes they're just, they just have too much 61 00:04:51,079 --> 00:04:55,100 load instability, which is not reflected on their status page. So, I mean, you can't even just 62 00:04:55,100 --> 00:04:59,920 necessarily rely on that to know whether something's going on. Luckily for the most part, 63 00:04:59,920 --> 00:05:04,200 if you have a smart retry strategy, you just end up waiting. You do a little bit of back off and 64 00:05:04,200 --> 00:05:09,260 it's usually okay. The other thing that's nice is that there's, at least for now, the users have a 65 00:05:09,260 --> 00:05:15,119 super high tolerance of errors because chat2BT is just, isn't reliable itself and you know, 66 00:05:15,119 --> 00:05:19,440 will fail. It's, it's really an interesting topic. It's a different topic, but it's an 67 00:05:19,440 --> 00:05:25,339 interesting topic in terms of eval. Evals is what you call, you know, kind of systems or patterns 68 00:05:25,339 --> 00:05:29,920 that are meant to, to check the correctness of the response, whether it aligns with what your, 69 00:05:29,980 --> 00:05:35,399 you know, particular guidelines are. So there's different kinds of evals that you can do. And 70 00:05:35,399 --> 00:05:39,820 actually this is something I get to in the book on the final chapter called quality control. 71 00:05:40,040 --> 00:05:45,260 So there's kind of two sides of the same, two faces of the same coin. There's evals and guardrails 72 00:05:45,739 --> 00:05:52,160 and guardrails are what you implement to eval on the fly the responses you're getting back from the, 73 00:05:52,160 --> 00:05:56,600 from the AI. So for instance, a really simple guardrail that we have is to make 74 00:05:56,600 --> 00:06:01,839 sure that only official languages are, the AI only responds in official languages. 75 00:06:02,720 --> 00:06:08,000 And we do that with a super, super cheap model. That's really, really fast. Essentially we just 76 00:06:08,000 --> 00:06:12,660 want, we want to spot check and make sure that no one's like kind of hacked our system to talking 77 00:06:12,660 --> 00:06:17,519 Vietnamese or Chinese or something that blows up our costs by like 10x because of the nature of it. 78 00:06:19,440 --> 00:06:24,799 Hey folks, it's me, Robbie. I want you to take a moment, close your eyes. Now picture your code 79 00:06:24,799 --> 00:06:29,420 and your applications as a symphony. Now to keep that symphony playing smoothly, 80 00:06:29,679 --> 00:06:35,559 you need an orchestra of tools. That's where our podcast sponsor AppSignal takes center stage. 81 00:06:35,820 --> 00:06:39,679 They combine the elegance of error tracking, the precision of performance monitoring, 82 00:06:39,739 --> 00:06:44,420 and the harmony of logging into one symphonic suite. Whether you're composing with Ruby, 83 00:06:44,420 --> 00:06:49,239 jamming with Elixir, orchestrating with Node.js, or harmonizing with Python, 84 00:06:49,519 --> 00:06:54,420 or maybe even a little bit of flourish of JavaScript, AppSignal's got the sheet music 85 00:06:54,420 --> 00:07:00,579 for you. And here's the crescendo. Plans start at just $23 US a month. That's got to be music 86 00:07:00,579 --> 00:07:06,880 to your budget's ears. Plus they're certified ISO 27001, and they dance the GDPR and HIPAA 87 00:07:06,940 --> 00:07:11,619 compliance beats. So don't miss a beat, my friends. Head on over to AppSignal.com and 88 00:07:11,839 --> 00:07:16,540 meet your good friend, Robby from Maintainable Centia. Now open your eyes and let the symphony 89 00:07:16,540 --> 00:07:26,059 of smooth coding begin. Let's get back to our show. So you could potentially, it could be very 90 00:07:26,059 --> 00:07:31,579 easy for someone if they just loosely integrate with something like OpenAI to kind of change 91 00:07:31,880 --> 00:07:36,339 how they're using it, whether it's malicious or not, or they're just changing the language and 92 00:07:36,339 --> 00:07:41,540 that adds a whole nother level of complexity to the infrastructure. For this kind of app, 93 00:07:41,540 --> 00:07:46,079 it's a conversational app and you're letting the user interface directly with the AI. 94 00:07:46,600 --> 00:07:52,299 Our users are great. They pay quite a bit to use the service. So we haven't really had too many 95 00:07:52,299 --> 00:07:59,720 out and out flagrant abusive cases or things where people were wasting resources or using us 96 00:07:59,839 --> 00:08:04,500 inappropriately. A lot of the things that we implement that are kind of like extra layered 97 00:08:04,500 --> 00:08:11,220 on top, whether it's guardrails or eval, kind of after the fact based evaluation of the conversations 98 00:08:11,220 --> 00:08:15,820 for quality control, for customer satisfaction reasons, things like that. They're just that, 99 00:08:15,940 --> 00:08:20,399 I mean, they're, you know, they're meant to provide a better service, right? But AI changes 100 00:08:20,399 --> 00:08:26,119 the game because it's no longer just about the quality of your code, you know, that makes things 101 00:08:26,179 --> 00:08:31,980 maintainable. I mean, some parts of what, some of the ways that you can use AI, and this is probably 102 00:08:31,980 --> 00:08:35,679 what's most unique about my book and what I'm most excited about and what readers seem to be 103 00:08:35,679 --> 00:08:42,419 most excited about as well is that I advocate for dropping in little AI black boxes as discrete AI 104 00:08:42,719 --> 00:08:48,140 components to replace otherwise traditional implementations of functionality. So for 105 00:08:48,340 --> 00:08:53,580 instance, I've been playing around with this concept within Olympia for some of our account 106 00:08:53,580 --> 00:09:00,000 management. So in Olympia, you subscribe to one of three base plans that gives you a team of AI 107 00:09:00,000 --> 00:09:05,820 powered consultants. And starting about four or five months ago, we started adding the ability 108 00:09:05,940 --> 00:09:11,559 to add extra assistance to your plan. You know, if you're an experienced, if you're listening and 109 00:09:11,559 --> 00:09:15,039 you're an experienced software developer, you can imagine, okay, there's like kind of a catalog of 110 00:09:15,299 --> 00:09:19,880 extra, you know, extra bots you can add to your account. So you look at those and you have a 111 00:09:19,880 --> 00:09:26,099 button that lets you add them. And then there's some validation that goes on. And then I have 112 00:09:26,099 --> 00:09:29,739 to create a new line item and add it to their Stripe subscription. 113 00:09:30,859 --> 00:09:34,280 And I have to modify them as, you know, it's kind of like the typical things you would expect when 114 00:09:34,280 --> 00:09:40,659 account teams request. So when I was implementing that, instead of doing it in what I would call 115 00:09:40,659 --> 00:09:46,739 just a normal traditional kind of way, what I did was I modeled an account manager class, almost 116 00:09:46,739 --> 00:09:54,239 like a microservice, the way you think about a microservice, that's responsible for that kind of 117 00:09:54,239 --> 00:10:02,200 change. And I made like a little worker. In the book, I call this approach multitude of workers. 118 00:10:02,719 --> 00:10:07,919 So I made a worker who is my account manager. And the interface between the rest of the app 119 00:10:07,919 --> 00:10:15,940 and this account manager is English, it's plain text. So basically, if the user goes in and looks 120 00:10:15,940 --> 00:10:21,059 at an extra assistant they want to add to their account, and they push that button to do so, 121 00:10:21,059 --> 00:10:27,780 internally, there's an English statement that's generated, you know, please add bot config 122 00:10:28,020 --> 00:10:37,140 with a UID to account with a UID. That's packaged inside of an account change request. And it's 123 00:10:37,140 --> 00:10:43,700 sent to this worker. And then the worker looks at the request. And it has at its disposal, 124 00:10:43,700 --> 00:10:53,020 a number of tool functions that allow it to do a number of things, including adding new bot 125 00:10:53,020 --> 00:11:01,000 configs to an account. But it can also send an email to the user or to customer service or both. 126 00:11:01,359 --> 00:11:08,580 It can also reference Olympia documentation and guidelines. And it can also finalize the request. 127 00:11:08,580 --> 00:11:14,380 So then at that point, programming it and maintaining that business logic, let's say, 128 00:11:14,500 --> 00:11:19,780 around how that account manager works becomes a matter of prompting. So the prompt that's given 129 00:11:19,780 --> 00:11:26,479 to account manager tells it how to behave. And in fact, while we're talking, I'll fire up my editor 130 00:11:26,919 --> 00:11:34,020 and I'll bring it up so I can actually read it. But the instructions literally tell it what to do 131 00:11:34,020 --> 00:11:38,919 when it gets a request. I'll read you the system directive, quote, unquote. This is the 132 00:11:38,960 --> 00:11:44,679 configuration for this bot, right? You are an account manager for Olympia. The user will request 133 00:11:44,679 --> 00:11:49,960 changes to their account and you will process those changes by invoking one or more of the 134 00:11:49,960 --> 00:11:55,359 functions provided. Make sure to notify the account owner of the result of the change request. 135 00:11:55,900 --> 00:12:01,659 Always end by calling the finished function so that we save the state of the change request 136 00:12:01,659 --> 00:12:06,979 as completed. That's it. Between that and the functions that it gets and the functions it has 137 00:12:06,979 --> 00:12:14,520 is add bot config to account, add new line item to subscription, notify account owner, and completed. 138 00:12:14,739 --> 00:12:20,919 Those are the functions that it has available to it. Those are just normal methods that have 139 00:12:20,919 --> 00:12:26,460 some imperative logic associated with it. Add a bot config to an account basically starts a 140 00:12:26,460 --> 00:12:30,440 transaction, finds the right bot, finds the right account, adds it to it. And then the output of 141 00:12:30,440 --> 00:12:36,119 those functions give a little bit more instructions as well. So for instance, if you add a bot config 142 00:12:36,119 --> 00:12:42,960 to the account, the output of that function, as opposed to being, I don't know what you would 143 00:12:42,960 --> 00:12:48,159 typically return from a function like that, but the output of this particular function is bot 144 00:12:48,159 --> 00:12:53,320 config with name, fill in the blank, add it to user's account, make sure to use the bot's name 145 00:12:53,320 --> 00:12:59,880 when communicating with the user. And then it says new account, new client for bot config name, 146 00:12:59,880 --> 00:13:04,739 account name signed up. I'm kind of going through this a little quickly. Make sure to add a line 147 00:13:04,739 --> 00:13:10,400 item to the user's subscription. The price is fill in the blank. The monthly charge in cents is 148 00:13:10,859 --> 00:13:16,400 fill in the blank, right? So that's what's returned to the bot when it calls that function. 149 00:13:16,520 --> 00:13:22,840 The reason I call it multitude of workers is because it changes your perspective on how you're 150 00:13:23,520 --> 00:13:29,340 implementing logic in your system to being more like you have a little human that is following 151 00:13:29,340 --> 00:13:35,260 instructions and you plop them into your code base. And then in kind of thinking through that, 152 00:13:35,520 --> 00:13:40,539 do you feel like that approach is just helping you kind of think about how you could potentially 153 00:13:40,539 --> 00:13:45,140 tap external workers or things like that versus how you're doing it within your own internal app? 154 00:13:45,140 --> 00:13:48,039 Because you could be like, click a button and it upgrades you through the normal process, but for 155 00:13:48,039 --> 00:13:53,400 you to change your mental model of how it's its own bot in a way that you're interacting with, 156 00:13:53,739 --> 00:13:57,659 you know? And so then it's taking advantage of that in that sort of way. Then is it safe to 157 00:13:57,840 --> 00:14:04,000 assume that when the emails might come from the account manager or whoever the emails get sent, 158 00:14:04,159 --> 00:14:07,280 they might be a little bit more tailored toward the particular... They might not always be the 159 00:14:07,280 --> 00:14:12,700 same because it's not going to be using the same email. Your account's been upgraded very 160 00:14:12,700 --> 00:14:17,119 consistently. Everybody gets the same email or might have a little bit more personalization and 161 00:14:17,140 --> 00:14:21,659 flavor and feel like you're interacting with an application, but you're actually interacting with 162 00:14:21,659 --> 00:14:26,619 an organization, with a team of people that you're working with in some capacity. There's 163 00:14:26,619 --> 00:14:31,619 air quoting, account manager over there that's helping take care of me, make me feel like a 164 00:14:31,619 --> 00:14:37,140 little bit more of a white glove service. I could have it do that, right? It would probably be as 165 00:14:37,140 --> 00:14:42,299 easy as just saying, insert a little bit of friendliness into it. I didn't in this case. 166 00:14:44,000 --> 00:14:49,239 it for free is that if something goes wrong or something is, uh, is different 167 00:14:49,360 --> 00:14:53,800 than what is expected, it is more robust in the sense that it can, it can deal 168 00:14:53,800 --> 00:14:59,060 with it, uh, or it can escalate to a human in the loop to fix it, but kind 169 00:14:59,060 --> 00:15:03,420 of going back to something you said a minute or two ago, it does allow me to, 170 00:15:03,979 --> 00:15:07,280 to do some pretty cool things that, that I wouldn't be able to do as 171 00:15:07,280 --> 00:15:08,819 easily with traditional code. 172 00:15:09,139 --> 00:15:10,979 And I'll give you a concrete example. 173 00:15:10,979 --> 00:15:15,920 We rolled this out and very soon thereafter, I got a Slack message from 174 00:15:16,080 --> 00:15:21,439 our customer service team saying, Hey, we had a failure with this new code. 175 00:15:21,860 --> 00:15:24,939 Uh, a user with an account that was inactive because they had 176 00:15:24,939 --> 00:15:27,279 scheduled it for cancellation, added a bot. 177 00:15:27,560 --> 00:15:30,939 And we don't think that that should be allowed, like you 178 00:15:30,939 --> 00:15:32,040 should have an active account. 179 00:15:32,159 --> 00:15:33,419 And I said, I think you're right. 180 00:15:33,540 --> 00:15:35,340 Like you shouldn't be able to add an account. 181 00:15:35,580 --> 00:15:37,540 It causes operational issues. 182 00:15:37,659 --> 00:15:38,080 Right? 183 00:15:38,919 --> 00:15:42,540 So JP said, you know, do you want me to add a story for that? 184 00:15:43,380 --> 00:15:46,619 And I said, no, I think I got it because I literally just opened up this 185 00:15:46,619 --> 00:15:47,979 file I have in front of my face right now. 186 00:15:47,979 --> 00:15:51,500 And I added a line to the directive that I didn't read the first time through. 187 00:15:51,900 --> 00:15:51,900 188 00:15:53,260 --> 00:15:56,660 So after it said, make sure to notify the account owner of the 189 00:15:56,660 --> 00:15:57,900 result of the change request. 190 00:15:58,300 --> 00:16:02,339 I inserted one line that said, do not allow the user to add a new bot config 191 00:16:02,339 --> 00:16:08,180 unless their account is active as part of what, uh, is provided to this bot. 192 00:16:08,199 --> 00:16:10,379 They get a dump of the account. 193 00:16:10,639 --> 00:16:13,819 So they, this bot knows whether the account is active. 194 00:16:14,099 --> 00:16:18,040 So with the addition of that one statement into the directive, I was 195 00:16:18,040 --> 00:16:22,300 able to handle the change request, which was don't let people with 196 00:16:22,300 --> 00:16:23,739 inactive accounts, add bots. 197 00:16:24,040 --> 00:16:24,599 Right. 198 00:16:24,599 --> 00:16:31,900 Not only that, if the, and I tested this to make sure if you do try to add a 199 00:16:31,900 --> 00:16:38,000 bot and you have an inactive account, the bot will write to you and say, Hey, I know 200 00:16:38,000 --> 00:16:43,040 you requested this to happen, but your account is inactive. 201 00:16:43,160 --> 00:16:47,440 If you'd like to try again after making your account active, you know, please do so. 202 00:16:47,500 --> 00:16:51,059 It's kind of blowing my mind a little bit and in a good way. 203 00:16:51,800 --> 00:16:54,960 Uh, trying to, you know, thinking about this, you know, I'm curious, uh, take 204 00:16:54,960 --> 00:16:57,080 quick step back cause you've mentioned your book a few times and I'll mention 205 00:16:57,080 --> 00:17:00,539 it in the intro section there, but could you share like a little bit about what 206 00:17:00,610 --> 00:17:03,729 inspired you to write your upcoming book, uh, patterns of application 207 00:17:03,729 --> 00:17:04,850 development using AI? 208 00:17:05,149 --> 00:17:05,929 Yeah, sure. 209 00:17:05,929 --> 00:17:10,330 I mean, I, um, like over a year ago when I started working on Olympia, because 210 00:17:10,330 --> 00:17:14,470 I've written books before, and this is such a greenfield, you know, with AI and 211 00:17:14,470 --> 00:17:19,410 stuff, I started taking notes of things that I thought were novel or, you know, 212 00:17:19,490 --> 00:17:23,250 cool in different ways for any of you out there who dream of writing a book. 213 00:17:23,289 --> 00:17:26,789 That's, that's one of the main things you should do, you know, like use 214 00:17:26,789 --> 00:17:31,029 Evernote or whatever, you know, and just keep track of things. 215 00:17:31,229 --> 00:17:35,570 I think Kent Beck or maybe it was Martin Fowler used to use the actual index 216 00:17:35,570 --> 00:17:37,229 cards and told us about that. 217 00:17:37,229 --> 00:17:40,690 So, you know, eventually what you do is when you have a stack of those index 218 00:17:40,690 --> 00:17:43,690 cards, whether they're physical or virtual, you know, then you have a book 219 00:17:43,710 --> 00:17:45,789 because you have content to write about. 220 00:17:45,809 --> 00:17:49,830 It's very hard to sit down and just like materialize a book out of nowhere. 221 00:17:50,570 --> 00:17:51,009 Right. 222 00:17:51,110 --> 00:17:52,589 So I've been doing that for a while. 223 00:17:52,589 --> 00:17:56,710 And then in Vegas, my old friend, Tom Rossi raised his hand during my, a talk 224 00:17:56,710 --> 00:17:58,929 I was giving about AI, which you may have been in the room. 225 00:17:59,529 --> 00:18:02,630 He said something like, Oh, this, this all seems really cool what you're 226 00:18:02,630 --> 00:18:07,009 talking about, but I'm finding this AI overwhelming, like, how do I get started? 227 00:18:07,449 --> 00:18:08,990 You know, how would you recommend I get started? 228 00:18:09,149 --> 00:18:10,509 And that's a really good question. 229 00:18:10,509 --> 00:18:12,990 And I was like, Oh, I had a voice in the back of my head going, there's 230 00:18:12,990 --> 00:18:17,229 got to be like a million people asking this same exact question right now. 231 00:18:17,270 --> 00:18:19,270 You know, AI is this big, big thing. 232 00:18:19,309 --> 00:18:23,009 That's kind of like encroaching on all of our lives, whether we want it to or not. 233 00:18:23,690 --> 00:18:25,750 And so what do you, what do you do with it? 234 00:18:25,750 --> 00:18:26,710 Like, how do you get started? 235 00:18:26,970 --> 00:18:30,929 If you do as much as just subscribe to one or two AI newsletters, it's 236 00:18:30,929 --> 00:18:35,250 already overwhelming because literally every, literally every day you're, you 237 00:18:35,250 --> 00:18:38,649 can have an avalanche of new things going on and whatnot. 238 00:18:38,850 --> 00:18:39,990 So I said, okay, cool. 239 00:18:40,190 --> 00:18:42,470 Let's write a book that describes how to get started. 240 00:18:42,669 --> 00:18:47,809 So I took inspiration from classic Greenfield books, like the Gang of 241 00:18:47,809 --> 00:18:50,910 Four Patterns book, Patterns of Enterprise Application Architecture by 242 00:18:50,979 --> 00:18:56,059 Martin Fowler, Patterns of Enterprise Integration, you know, by, by Gregor 243 00:18:56,059 --> 00:19:00,260 Hopi and David Rice, old colleagues at ThoughtWorks from when I worked there 244 00:19:00,260 --> 00:19:04,799 in the 2000s, and I said, I want to write a book like that, like that is 245 00:19:04,799 --> 00:19:11,019 literally, you know, has a shot at becoming like the reference book for how 246 00:19:11,019 --> 00:19:16,380 people talk about integrating AI components into their application development. 247 00:19:16,380 --> 00:19:21,279 And I took a very, very specific approach, which was to ignore 248 00:19:21,440 --> 00:19:23,899 things that I feel are transient. 249 00:19:24,880 --> 00:19:31,519 So like the whole debate about what AI's appropriate role is, whether it will 250 00:19:31,760 --> 00:19:36,480 replace engineers, specific technologies that are current, you know, currently 251 00:19:36,480 --> 00:19:39,940 in vogue or whatnot, like I tried to, I tried to stay away from those 252 00:19:39,940 --> 00:19:41,799 topics and not cover those. 253 00:19:41,799 --> 00:19:47,760 And I tried to take real code from Olympia and show how they represent 254 00:19:47,940 --> 00:19:53,820 patterns of interaction between an AI and a normal code base, and, and very 255 00:19:53,820 --> 00:19:57,760 much putting the emphasis on normal day-to-day application development 256 00:19:57,760 --> 00:19:58,760 that everyone does. 257 00:19:59,179 --> 00:20:02,100 So the examples are in Ruby on Rails, because that's what I know. 258 00:20:02,540 --> 00:20:06,040 I extracted the code examples in the book from Olympia. 259 00:20:06,179 --> 00:20:07,739 They're meant to be very practical. 260 00:20:07,739 --> 00:20:12,700 There's no like car and wheels, like you might see in a, like an OO primer 261 00:20:13,000 --> 00:20:14,500 or, you know, stuff like that. 262 00:20:14,940 --> 00:20:19,440 It's all kind of real examples, you know, of things. 263 00:20:19,739 --> 00:20:24,760 And then in terms of what the book covers, part one starts from the beginning, 264 00:20:24,820 --> 00:20:28,700 like kind of if you, you know, going through a primer of what is a large 265 00:20:28,700 --> 00:20:29,940 language model, how does it work? 266 00:20:30,000 --> 00:20:31,119 What are the components of it? 267 00:20:31,239 --> 00:20:33,899 What is a late, you know, what is the latent space? 268 00:20:33,899 --> 00:20:37,179 I mean, I talk about narrowing the path for when you're trying to 269 00:20:37,739 --> 00:20:39,279 generate something with, with an AI. 270 00:20:40,140 --> 00:20:41,720 And why do I call it that? 271 00:20:41,760 --> 00:20:46,720 You know, like, because these frontier models, which is the terminology 272 00:20:46,720 --> 00:20:52,260 they use to refer to GPT-4 and Cloud3, like the really, really big ones. 273 00:20:52,459 --> 00:20:55,940 For intents and purposes, they know everything that there is to 274 00:20:55,940 --> 00:20:57,559 know at at least a low level. 275 00:20:57,959 --> 00:21:02,380 And because of that, you know, take any three letter acronym that you throw 276 00:21:02,380 --> 00:21:05,299 at it, just to give you an example, and there's probably 20 different 277 00:21:05,299 --> 00:21:10,339 versions of it that it could, that you could be talking about, so they get 278 00:21:10,399 --> 00:21:13,459 confused, you know, like if you don't narrow the scope of what you're asking 279 00:21:13,459 --> 00:21:17,339 for and what you're talking about, and that that's an art, you know, at this 280 00:21:17,339 --> 00:21:22,480 point, there's a lot of craft in using these things, but you can learn it, you 281 00:21:22,480 --> 00:21:24,559 know, and that's what people refer to as prompt engineering. 282 00:21:25,339 --> 00:21:29,119 So there's a lot of material that's just kind of fundamental material 283 00:21:29,119 --> 00:21:31,420 about prompt engineering in there. 284 00:21:31,600 --> 00:21:33,459 And how, you know, how do you come up with a prompt? 285 00:21:33,700 --> 00:21:35,179 How do you refine prompts? 286 00:21:35,459 --> 00:21:36,320 How do you manage them? 287 00:21:36,399 --> 00:21:37,459 How do you version them? 288 00:21:37,779 --> 00:21:39,339 How do you template them? 289 00:21:39,660 --> 00:21:40,619 You know, things like that. 290 00:21:40,619 --> 00:21:42,579 So there's a lot of that kind of fundamental stuff. 291 00:21:42,640 --> 00:21:48,059 If you're going to use AI to drive intelligent workflows, how do you think 292 00:21:48,059 --> 00:21:51,559 about that, you know, from an architectural perspective, how do you model it? 293 00:21:51,880 --> 00:21:55,399 In your mind, how do you model it in terms of how you will implement it? 294 00:21:55,399 --> 00:22:02,140 If you're going to use AI to do really cool things like self-healing data or 295 00:22:02,399 --> 00:22:05,940 intelligent error handling, you know, when we talk about robustness, I mean, 296 00:22:05,940 --> 00:22:11,019 you can use AI to do pretty cool things for your application in terms of robustness. 297 00:22:11,040 --> 00:22:14,500 So the example I give the book in terms of self-healing data, I mean, we, we 298 00:22:14,500 --> 00:22:18,559 deal with a lot of JSON because we have, we have the AI models generate JSON. 299 00:22:19,399 --> 00:22:26,079 And because they are not perfect at generating anything, really, they make 300 00:22:26,079 --> 00:22:30,559 mistakes, just like, like if you were a human writing JSON, you would make, 301 00:22:30,779 --> 00:22:31,940 you would make mistakes, right? 302 00:22:31,940 --> 00:22:35,700 You would forget quotes, you would mess up your quotation or things like that. 303 00:22:35,700 --> 00:22:39,079 Or you would deviate from schemas and they do the same thing. 304 00:22:39,480 --> 00:22:43,079 So you go through enough of these errors and you have it blow up your code enough. 305 00:22:43,079 --> 00:22:48,019 And you start thinking, maybe I can just get it to fix the JSON that was broken. 306 00:22:48,559 --> 00:22:49,760 Just give it another crack. 307 00:22:50,279 --> 00:22:50,279 308 00:22:50,279 --> 00:22:51,880 So you can do stuff like that. 309 00:22:51,920 --> 00:22:55,000 It's not so much a retry, but like, if you have broke, you know, like if you have 310 00:22:55,000 --> 00:23:00,440 broken JSON and you want to try to get some value out of it, out of it, like you 311 00:23:00,440 --> 00:23:04,079 can have, we have a JSON fixer class. 312 00:23:04,519 --> 00:23:07,839 So you give it the JSON and you give it the error message and you say, Hey, here, 313 00:23:07,839 --> 00:23:12,019 fix this while preserving what it, what it's supposed to do or what it meant. 314 00:23:14,019 --> 00:23:16,299 We'll be back with our interview with Obi in just a moment. 315 00:23:16,599 --> 00:23:17,559 Hi, it's me, Robbie. 316 00:23:17,559 --> 00:23:20,020 I just want to take a quick moment to say thank you for listening to Maintainable. 317 00:23:20,239 --> 00:23:23,099 If you're finding these discussions valuable, please consider sharing them 318 00:23:23,099 --> 00:23:25,780 link amongst your peers and or writing a review on Apple podcast or a 319 00:23:25,780 --> 00:23:27,020 Spotify to help spread the word. 320 00:23:27,299 --> 00:23:29,440 Also, do you think there's someone I should have on the podcast sometime? 321 00:23:29,859 --> 00:23:33,500 Shoot me an email to Robbie with a Y at maintainable.fm and state your case. 322 00:23:33,520 --> 00:23:36,080 And now let's get back to our interview, Obi Fernandez. 323 00:23:38,640 --> 00:23:42,799 You know, I think about, you know, like when you're building out applications 324 00:23:42,799 --> 00:23:46,940 and taking advantage of using these types of, you know, these LLMs and 325 00:23:46,940 --> 00:23:50,559 their, their API endpoints that they're offering us, I'm also curious if you've 326 00:23:50,559 --> 00:23:55,840 thought much about how just maintenance in the work of the projects that you're 327 00:23:55,840 --> 00:23:59,840 working on, how, how AI can be a benefit to just, you mentioned self-healing as 328 00:23:59,840 --> 00:24:03,539 a thing, but also just from like coming to existing legacy code. 329 00:24:03,640 --> 00:24:06,080 I know that you primarily, maybe these days work a lot more greenfield 330 00:24:06,099 --> 00:24:09,159 applications, but I think about projects that are around for awhile and like, how 331 00:24:09,159 --> 00:24:13,580 are you thinking about how AI may or may not be able to aid in the development 332 00:24:13,580 --> 00:24:18,039 process and help maybe expedite some of those, the challenges that some teams 333 00:24:18,039 --> 00:24:21,580 face from accruing a lot of technical debt or just being really outdated on 334 00:24:21,580 --> 00:24:25,119 versions and stuff like that and dealing with upgrades and removing some of that 335 00:24:25,119 --> 00:24:28,820 fear, knowing that AI is not perfect, but neither are humans. 336 00:24:29,000 --> 00:24:31,020 I can't really speak to that. 337 00:24:31,500 --> 00:24:36,539 I don't have direct experience with it to the extent that I've used, like I tried 338 00:24:36,539 --> 00:24:42,719 using a new product from GitHub called, I think called code workspaces or something 339 00:24:42,719 --> 00:24:45,539 like that, they have a beta, there's a waiting list. 340 00:24:45,619 --> 00:24:47,520 I got, I finally got access to it. 341 00:24:47,880 --> 00:24:52,419 And I don't know if it's just bad with Ruby, it's entirely possible, but the way 342 00:24:52,419 --> 00:24:57,320 that this thing works is it supposedly has access to your whole code that you 343 00:24:57,320 --> 00:25:01,440 have, you know, when you add a repo to it and then you, you start a new session, I 344 00:25:01,440 --> 00:25:02,479 believe it's what it's called. 345 00:25:02,479 --> 00:25:06,340 And then you add a, you know, you, you say what you're working on. 346 00:25:06,539 --> 00:25:09,179 So it's like changing a feature, fixing a bug or whatever. 347 00:25:09,760 --> 00:25:14,280 And then it goes through and it creates a plan of action and it runs it by you. 348 00:25:14,900 --> 00:25:19,000 And if you agree with the plan of action, I mean, you can modify it if you want, but 349 00:25:19,000 --> 00:25:23,320 it, you know, once, once it's good, you hit go, and then it uses AI to kind 350 00:25:23,320 --> 00:25:24,200 of do the work for you. 351 00:25:24,320 --> 00:25:31,340 So I tried to do that with Olympia and to have it, so, so I, I, you're working 352 00:25:31,340 --> 00:25:34,840 with these frontier technologies, you know, you end up, you know, things 353 00:25:34,840 --> 00:25:35,960 change out from under you. 354 00:25:36,000 --> 00:25:38,820 So like, for instance, in Olympia, when I started out, I built it on MagnetChat, 355 00:25:38,820 --> 00:25:44,820 which is an open source chat to BT clone that I wrote and like back in February 356 00:25:44,820 --> 00:25:49,219 of last year, and so some things were still using kind of the magma chat 357 00:25:49,400 --> 00:25:51,260 backend to do requests to the AI. 358 00:25:51,479 --> 00:25:56,539 And nowadays, most of my stuff is switched over to Ray, which is my library 359 00:25:56,580 --> 00:25:59,179 for, for doing AI that I talk about in the book. 360 00:25:59,320 --> 00:26:02,320 So I took one particular class that was still using magma. 361 00:26:02,340 --> 00:26:05,580 And I said, I want to switch this over to use this module chat 362 00:26:05,580 --> 00:26:06,859 completion, which is from Ray. 363 00:26:07,500 --> 00:26:09,080 And I was pretty excited about it. 364 00:26:09,200 --> 00:26:12,859 And I hit go, and then it thought about it and it's even animated. 365 00:26:13,140 --> 00:26:14,400 So it showed me that it was doing stuff. 366 00:26:14,500 --> 00:26:17,479 And then it just deleted like 80% of the classes that I was done. 367 00:26:17,700 --> 00:26:20,919 And what it left, what it left me with was nonsensical, just didn't work. 368 00:26:21,039 --> 00:26:27,780 So I think it's quite not, not quite there yet, at least not for Ruby or at 369 00:26:27,780 --> 00:26:31,960 least not for big, you know, big apps, which Olympia at this point is, so. 370 00:26:32,820 --> 00:26:33,700 Yeah, it's interesting. 371 00:26:33,700 --> 00:26:34,059 Yeah. 372 00:26:34,059 --> 00:26:38,140 I'm always kind of curious about it on that front, thinking about, will there 373 00:26:38,140 --> 00:26:41,840 be some, be able to expedite some of those painful things that some teams are 374 00:26:41,840 --> 00:26:45,659 dealing with and I think it might, might still be a little bit out there for us. 375 00:26:45,859 --> 00:26:49,039 But I'm also curious a little bit more about, can you tell us a 376 00:26:49,039 --> 00:26:49,940 little bit about Olympia as well? 377 00:26:50,000 --> 00:26:52,679 You kind of hinted at it a little bit, but like what's, what's the 378 00:26:52,679 --> 00:26:54,239 core idea behind the product? 379 00:26:55,179 --> 00:27:00,419 The core idea behind the product is to provide as close as we can get with the 380 00:27:00,599 --> 00:27:05,619 current technology to a human team mate for your small business. 381 00:27:06,239 --> 00:27:11,080 So we have like 12 or I think we have 14 now different kinds of profiles. 382 00:27:11,219 --> 00:27:15,380 So it's like things like a legal consultant, business strategist, social 383 00:27:15,380 --> 00:27:20,340 media expert, content specialist, copywriter, and some of them have special 384 00:27:20,340 --> 00:27:25,080 tools for, so like for instance, the copywriter understands how to create a, 385 00:27:25,479 --> 00:27:27,419 to collaboratively create a content brief with you. 386 00:27:27,599 --> 00:27:30,200 Let's say you want to write a blog post or something. 387 00:27:30,200 --> 00:27:36,919 So it will go over and you define the requirements and then it will, it has 388 00:27:36,919 --> 00:27:42,059 tools for writing long form content in one shot, so it can generate like 389 00:27:42,059 --> 00:27:46,280 thousands of words, you know, kind of complicated outlines from a brief 390 00:27:46,359 --> 00:27:50,859 compared to what you get out of a chat TPT, which, you know, you can, you 391 00:27:50,859 --> 00:27:54,679 can tell chat TPT, I want to write a blog post and it'll, you know, spit 392 00:27:54,679 --> 00:27:56,239 out a certain amount, but then it stops. 393 00:27:56,479 --> 00:27:59,619 The copywriter has tools to do, you know, kind of big things. 394 00:27:59,619 --> 00:28:03,000 Some of them, some of them have extra knowledge and are trained. 395 00:28:03,460 --> 00:28:09,419 We're very gradually ramping up, content ingestion for some of our experts so 396 00:28:09,419 --> 00:28:13,580 that they get like, basically we can subscribe them to newsletters and then 397 00:28:13,580 --> 00:28:17,400 the content of the newsletter gets, digested and then put into their 398 00:28:17,400 --> 00:28:19,719 knowledge base so that they have like kind of special knowledge. 399 00:28:19,780 --> 00:28:21,859 We also have human augmented AI. 400 00:28:22,559 --> 00:28:27,140 Most of our assistants are just pure AI constructed characters, but a subset of 401 00:28:27,140 --> 00:28:32,039 our assistants actually have a human, they're clones of a human that is a 402 00:28:32,039 --> 00:28:34,260 particular expert in, you know, something. 403 00:28:34,359 --> 00:28:38,520 So like for instance, Chandel Stone, who is well known here in Mexico city, 404 00:28:38,679 --> 00:28:42,940 because she's, she runs a startup accelerator is kind of like our 405 00:28:42,940 --> 00:28:51,799 resident startup advisor, so her bot is trained on a bunch of her, you know, 406 00:28:51,799 --> 00:28:57,640 material, her, her talks and blog posts and, and she's sunk a substantial amount 407 00:28:57,640 --> 00:29:01,820 of time into training the bot as well, having conversations with it to get 408 00:29:01,820 --> 00:29:03,460 it to respond the way that she would. 409 00:29:04,159 --> 00:29:10,700 What makes this different than just like a custom GPT is that it is very, very 410 00:29:10,700 --> 00:29:15,320 much supposed to be like a little mini me version of the human that acts as an 411 00:29:15,320 --> 00:29:16,179 assistant. 412 00:29:17,140 --> 00:29:24,880 So it has the ability to escalate to its human and its human is expected to review 413 00:29:24,880 --> 00:29:25,979 everything that it does. 414 00:29:26,599 --> 00:29:27,900 So Olympia features an. 415 00:29:28,000 --> 00:29:32,719 for being able to, you know, 416 00:29:32,719 --> 00:29:34,699 Chantel can go in there and monitor the conversations 417 00:29:34,699 --> 00:29:37,399 and steer the conversations in certain directions. 418 00:29:38,060 --> 00:29:39,880 Custom GPTs are meant for scale. 419 00:29:40,079 --> 00:29:41,760 They're kind of like an app store, you know, 420 00:29:41,760 --> 00:29:43,340 mobile app kind of economic model. 421 00:29:43,420 --> 00:29:47,260 This is more of a AI assistant for consultants 422 00:29:47,540 --> 00:29:50,780 and people who deal one-on-one as experts. 423 00:29:51,000 --> 00:29:51,659 That's interesting. 424 00:29:51,739 --> 00:29:53,260 So when they're going through that, 425 00:29:53,440 --> 00:29:55,159 when your customers are using these, 426 00:29:55,159 --> 00:30:00,080 like using Chantel's tool or assistant clone or mini me, 427 00:30:00,139 --> 00:30:01,459 whatever, however you want to describe that, 428 00:30:01,680 --> 00:30:04,080 is most of those requests happening in real time 429 00:30:04,080 --> 00:30:05,439 with your customers? 430 00:30:05,439 --> 00:30:08,240 And then occasionally it escalates something to her 431 00:30:08,240 --> 00:30:09,979 to get some feedback on, 432 00:30:10,060 --> 00:30:13,520 or does the AI assistant know how to say, 433 00:30:13,599 --> 00:30:15,159 here's some of the different types of things 434 00:30:15,159 --> 00:30:17,419 that I've been helping different customers with, 435 00:30:17,880 --> 00:30:19,159 review my work, like, 436 00:30:19,319 --> 00:30:22,099 and then is it just waiting for you to kind of interact 437 00:30:22,099 --> 00:30:24,900 with a prompt and then expand on its knowledge base there? 438 00:30:25,500 --> 00:30:26,799 I mean, this is all, 439 00:30:26,880 --> 00:30:29,580 I want to be humble and say this all in its infancy. 440 00:30:31,919 --> 00:30:36,000 So you see occasional glimpses of amazing magic 441 00:30:36,319 --> 00:30:38,720 that should be coming when the models 442 00:30:38,720 --> 00:30:40,480 get a little more powerful, 443 00:30:40,599 --> 00:30:42,940 but yes, it happens in real time. 444 00:30:43,220 --> 00:30:45,559 For the most part, they're dealing with the AI. 445 00:30:45,900 --> 00:30:47,900 The AI is impersonating, in this case Chantel, 446 00:30:49,500 --> 00:30:52,440 and Chantel can speak to the AI 447 00:30:52,440 --> 00:30:55,660 the way that you would speak to your junior assistant 448 00:30:55,660 --> 00:30:59,260 and say, hey, what's going on with such and such client 449 00:30:59,260 --> 00:31:00,220 and get a report. 450 00:31:00,419 --> 00:31:05,239 And the AI understands that it's talking to its clone boss. 451 00:31:05,940 --> 00:31:06,779 That's fascinating. 452 00:31:07,180 --> 00:31:08,419 I'm curious also, like, 453 00:31:08,680 --> 00:31:12,160 did many of these assistants that you can interact with, 454 00:31:12,199 --> 00:31:14,620 do they have a lot of access to performing actions 455 00:31:14,620 --> 00:31:16,339 on behalf of your customers 456 00:31:16,339 --> 00:31:18,720 or like integrating with other APIs and stuff like that, 457 00:31:18,720 --> 00:31:20,160 or is that something that's kind of on the roadmap? 458 00:31:20,160 --> 00:31:23,059 Generate the marketing content idea or the blog post, 459 00:31:23,160 --> 00:31:24,860 but also generate the images and assets. 460 00:31:25,059 --> 00:31:27,059 And actually, can you just take care of the publishing it 461 00:31:27,360 --> 00:31:29,580 and scheduling the things across social media 462 00:31:29,580 --> 00:31:30,180 and everything? 463 00:31:30,720 --> 00:31:32,919 So we're in the process of rolling out 464 00:31:32,919 --> 00:31:34,319 image generation this week. 465 00:31:34,519 --> 00:31:38,080 So that's already implemented with stable diffusion. 466 00:31:38,240 --> 00:31:40,419 It's like the newest new thing that we're rolling out. 467 00:31:40,639 --> 00:31:44,279 And then we're working on the ability 468 00:31:45,100 --> 00:31:47,460 to let your bots kind of do anything 469 00:31:47,460 --> 00:31:48,800 that you can do on the web. 470 00:31:48,800 --> 00:31:51,679 So there's different approaches to this problem. 471 00:31:51,800 --> 00:31:56,740 You can do one by one implementations on a per API basis, 472 00:31:57,500 --> 00:32:02,020 but it's time consuming and it's difficult 473 00:32:02,020 --> 00:32:03,399 and you can only do one at a time. 474 00:32:03,419 --> 00:32:04,800 There's another approach, 475 00:32:04,899 --> 00:32:07,000 which is the one that we're working on, 476 00:32:07,039 --> 00:32:08,699 which uses some pretty cool technology 477 00:32:08,919 --> 00:32:13,360 to basically give your bot a browser, 478 00:32:13,699 --> 00:32:14,899 a web browser of their own. 479 00:32:15,119 --> 00:32:18,559 And then it can log in with your help 480 00:32:18,559 --> 00:32:27,039 if necessary to Twitter, to whatever, your CRM, 481 00:32:27,039 --> 00:32:30,160 and then do stuff on your behalf. 482 00:32:30,339 --> 00:32:32,440 And that session stays stored 483 00:32:32,500 --> 00:32:34,759 so that it can use it in the future. 484 00:32:34,860 --> 00:32:36,339 And I think this is the direction 485 00:32:36,679 --> 00:32:38,679 that the industry is gonna go. 486 00:32:38,880 --> 00:32:42,500 Actually OpenAI just acquired a company 487 00:32:42,660 --> 00:32:45,759 that probably has the, like literally yesterday, 488 00:32:45,960 --> 00:32:46,619 I saw the news. 489 00:32:46,839 --> 00:32:47,860 I forget what they're called. 490 00:32:47,860 --> 00:32:51,600 I think it's RockAI, something like that. 491 00:32:51,979 --> 00:32:53,979 That probably has some of the leading edge technology 492 00:32:54,820 --> 00:32:56,800 for giving the bot the ability 493 00:32:57,320 --> 00:32:59,160 to understand a browser screen, 494 00:32:59,539 --> 00:33:01,800 like kind of buttons and things like that. 495 00:33:02,039 --> 00:33:04,279 But I think that's where we're going. 496 00:33:04,960 --> 00:33:06,119 We're gonna want, as humans, 497 00:33:06,320 --> 00:33:07,919 we're gonna want our AI agents 498 00:33:08,059 --> 00:33:11,100 to pretty much do anything that we can do for us. 499 00:33:13,220 --> 00:33:14,020 Hey folks, it's me, Robbie. 500 00:33:14,119 --> 00:33:15,860 Just wanna take a quick moment to plug my own company, 501 00:33:16,080 --> 00:33:17,020 Planet Argonne. 502 00:33:17,020 --> 00:33:19,899 We're a Ruby on Rails software consultancy, primarily. 503 00:33:20,139 --> 00:33:21,539 We work with some other technologies as well, 504 00:33:21,639 --> 00:33:22,699 but if you've got our existing 505 00:33:22,860 --> 00:33:24,800 and aging legacy Ruby on Rails application, 506 00:33:25,259 --> 00:33:27,860 or maybe Laravel, maybe React, we do that too, 507 00:33:28,440 --> 00:33:30,539 and need some help, get in touch with us 508 00:33:30,639 --> 00:33:31,740 at planetargonne.com. 509 00:33:31,860 --> 00:33:33,580 We help organizations with existing applications 510 00:33:33,580 --> 00:33:35,320 make them better and more maintainable. 511 00:33:35,479 --> 00:33:37,639 That's right, planetargonne.com. 512 00:33:38,440 --> 00:33:39,960 And now, back to our show. 513 00:33:42,820 --> 00:33:44,640 Yeah, I think about that in terms of like 514 00:33:44,739 --> 00:33:46,320 where we've used it ourselves, 515 00:33:46,320 --> 00:33:47,700 or when I talk with some of our customers 516 00:33:47,700 --> 00:33:48,700 that are using it for different things, 517 00:33:48,880 --> 00:33:50,420 but there's always this kind of point of being like, 518 00:33:50,519 --> 00:33:52,239 well, do we then take the next step 519 00:33:52,239 --> 00:33:53,640 and like integrate things with like Zapier 520 00:33:53,640 --> 00:33:56,040 and their huge data set of, 521 00:33:56,100 --> 00:33:57,519 or at least a collection of APIs 522 00:33:57,519 --> 00:33:58,739 that you can potentially integrate with, 523 00:33:58,739 --> 00:34:00,760 and like, then you can kind of figure out a way 524 00:34:00,760 --> 00:34:02,640 to maybe navigate it that way. 525 00:34:02,640 --> 00:34:04,000 But the browser-based thing is also, 526 00:34:04,200 --> 00:34:06,220 it's an interesting angle as well. 527 00:34:06,299 --> 00:34:08,540 And then just wondering, like, how do you... 528 00:34:08,540 --> 00:34:09,640 Like, I looked into Zapier 529 00:34:10,260 --> 00:34:12,000 and I have kind of half-written code 530 00:34:12,220 --> 00:34:13,000 that integrates with Zapier, 531 00:34:13,000 --> 00:34:16,279 but when this is really gonna take off like crazy, 532 00:34:16,660 --> 00:34:18,239 well, first of all, I think we need to go, 533 00:34:18,559 --> 00:34:22,000 we need like one or more evolutionary steps. 534 00:34:22,519 --> 00:34:25,000 Like, I don't know if it's GPT-4.5 or GPT-5 535 00:34:25,000 --> 00:34:26,760 or whatever the equivalent class is there, 536 00:34:26,799 --> 00:34:29,420 because the models are not quite good enough. 537 00:34:29,459 --> 00:34:30,899 And like anyone who's working on this, 538 00:34:31,100 --> 00:34:33,079 I hear the same thing from other people as well. 539 00:34:33,079 --> 00:34:35,640 It's like, they're not quite intelligent enough, 540 00:34:35,799 --> 00:34:36,619 hopefully they get there. 541 00:34:37,600 --> 00:34:39,779 But this is all still really nerdy. 542 00:34:39,859 --> 00:34:42,559 Like even the agent stuff, like, you know, 543 00:34:42,559 --> 00:34:45,500 we keep a close eye on certain competitors 544 00:34:45,500 --> 00:34:46,799 that are offering agent behavior 545 00:34:46,799 --> 00:34:49,040 because we're going in that direction as well with Olympia. 546 00:34:50,059 --> 00:34:53,500 And a lot of the stuff that's being called agent behavior 547 00:34:53,899 --> 00:34:57,899 is not exactly like the kind of science fiction agent thing 548 00:34:57,899 --> 00:35:00,459 that we're kind of all hoping for 549 00:35:00,459 --> 00:35:01,779 or dreading depending on your perspective. 550 00:35:02,640 --> 00:35:04,579 It's really more like, okay, 551 00:35:04,600 --> 00:35:08,160 I configured a bot to do a particular job 552 00:35:08,160 --> 00:35:10,179 and now it's doing it by itself. 553 00:35:10,179 --> 00:35:13,760 And I think if any, you know, that's just a script. 554 00:35:14,640 --> 00:35:16,660 You know, like if you can call it a bot. 555 00:35:16,899 --> 00:35:20,820 I think what makes it legitimately some degree of an agent 556 00:35:20,820 --> 00:35:23,540 is that they do have some amount of robustness 557 00:35:23,540 --> 00:35:25,100 around being able to retry things 558 00:35:25,100 --> 00:35:27,899 or understand different boundary conditions 559 00:35:27,899 --> 00:35:30,679 or something changes up from under them. 560 00:35:30,920 --> 00:35:33,100 They don't break because they're looking 561 00:35:33,100 --> 00:35:36,140 at the source material semantically rather than structurally. 562 00:35:36,440 --> 00:35:37,859 So that's important. 563 00:35:37,940 --> 00:35:38,980 That makes it an agent. 564 00:35:40,619 --> 00:35:43,079 But they have to be programmed. 565 00:35:43,959 --> 00:35:46,239 And one thing that we've learned over and over again, 566 00:35:46,459 --> 00:35:48,040 and I'm sure you've been around long enough 567 00:35:48,040 --> 00:35:49,440 to understand as well, 568 00:35:49,720 --> 00:35:53,260 is that even the most no-code no-code solutions 569 00:35:53,260 --> 00:35:56,260 still end up being used by quasi programmers, 570 00:35:57,799 --> 00:36:00,559 not really normal people, right? 571 00:36:00,820 --> 00:36:03,679 Like it's still people who have some strong motivation 572 00:36:03,679 --> 00:36:06,799 or interest in doing some, what amounts to programming, 573 00:36:07,260 --> 00:36:09,679 even if they're dragging boxes around on a screen 574 00:36:09,679 --> 00:36:12,459 with arrows, they're still programming. 575 00:36:13,420 --> 00:36:16,000 And the vast majority of people in this world 576 00:36:16,000 --> 00:36:17,179 don't want to program. 577 00:36:17,859 --> 00:36:19,339 They just have no interest in it. 578 00:36:19,640 --> 00:36:21,959 Their eyes start blurring or watering, you know, 579 00:36:22,059 --> 00:36:24,160 whenever you give them anything that looks like programming. 580 00:36:25,279 --> 00:36:28,200 So therefore, actually along those lines, 581 00:36:28,200 --> 00:36:30,320 I saw an interesting data point the other day, 582 00:36:30,380 --> 00:36:33,420 which was like some ridiculously high percentage 583 00:36:33,799 --> 00:36:35,559 of people who use spreadsheets 584 00:36:36,119 --> 00:36:40,899 have never ever even once used any sort of equals, 585 00:36:40,899 --> 00:36:43,160 you know, formula or anything for it. 586 00:36:43,160 --> 00:36:46,339 It's literally just a grid to, you know, 587 00:36:46,480 --> 00:36:48,760 to put things in a grid format. 588 00:36:48,779 --> 00:36:50,899 You know, it's just a way to lay out data, right? 589 00:36:50,980 --> 00:36:53,059 Cause yeah, cause most normal people don't want to program. 590 00:36:53,200 --> 00:36:55,160 So where I see this really, really taking off 591 00:36:55,160 --> 00:36:57,059 and we're not just, we're not there yet, 592 00:36:57,700 --> 00:37:01,519 is when you can describe probably by voice, 593 00:37:02,119 --> 00:37:03,820 like, hey, I'd like you to do this thing. 594 00:37:03,920 --> 00:37:04,579 Can you do it? 595 00:37:04,579 --> 00:37:07,540 And it's probably gonna say, I don't know, but I can try. 596 00:37:07,940 --> 00:37:10,119 And then there's an iterative process, you know, 597 00:37:10,119 --> 00:37:12,660 where it tries and you say, no, do this instead. 598 00:37:12,679 --> 00:37:17,079 And then eventually it has a skill, a new skill. 599 00:37:17,799 --> 00:37:19,760 And then that's persisted in some way. 600 00:37:19,779 --> 00:37:21,059 And you can say, can you do that thing 601 00:37:21,059 --> 00:37:22,040 that you did for me again? 602 00:37:22,420 --> 00:37:26,339 Or can you do that every morning, you know, or et cetera. 603 00:37:26,500 --> 00:37:27,420 And that makes a lot of sense. 604 00:37:27,500 --> 00:37:29,160 And I think the, you know, 605 00:37:29,160 --> 00:37:30,839 in terms of like how people are trying 606 00:37:30,839 --> 00:37:31,559 to wrap their head around this, 607 00:37:31,640 --> 00:37:34,019 and I've been loosely kind of keeping it, 608 00:37:34,019 --> 00:37:35,700 I subscribe to newsletters and it's overwhelming, 609 00:37:35,880 --> 00:37:37,040 but just trying to get a sense 610 00:37:37,040 --> 00:37:39,500 of how different teams are accounting for this 611 00:37:39,500 --> 00:37:41,480 or including it into their workflow and process, 612 00:37:41,640 --> 00:37:44,640 but also knowing that you can't just automate it away yet. 613 00:37:44,660 --> 00:37:45,799 We're like not there. 614 00:37:45,799 --> 00:37:49,220 It could be a ways off, but more to come on that. 615 00:37:49,220 --> 00:37:50,359 Well, cool, with that, 616 00:37:50,519 --> 00:37:52,559 I do have a couple of quick last questions for you, Obi. 617 00:37:53,040 --> 00:37:56,299 Is there a non-software, non-technical book 618 00:37:56,299 --> 00:37:59,000 that you do like to recommend to peers? 619 00:37:59,940 --> 00:38:03,320 So I read a lot of science fiction for fun. 620 00:38:03,840 --> 00:38:05,640 I don't know, I'm particularly Ian Banks. 621 00:38:06,159 --> 00:38:09,239 So like pretty much anything that he's written, 622 00:38:09,280 --> 00:38:11,039 I think gives us kind of an interesting, 623 00:38:11,500 --> 00:38:14,419 different perspective for our future, you know. 624 00:38:15,039 --> 00:38:17,219 Excellent, I'll include links to a couple of Ian's books 625 00:38:17,219 --> 00:38:18,460 in the show notes for people. 626 00:38:19,559 --> 00:38:22,039 And where can listeners best follow your thoughts 627 00:38:22,039 --> 00:38:24,780 and ruminations about software engineering and AI online? 628 00:38:26,659 --> 00:38:29,380 Definitely Twitter these days, yeah. 629 00:38:29,539 --> 00:38:30,219 Also, I mean, lately, 630 00:38:30,460 --> 00:38:33,059 and this is kind of why I ended up getting into this 631 00:38:33,119 --> 00:38:37,059 to begin with, I mean, I do still blog on my Medium blog, 632 00:38:37,179 --> 00:38:38,640 which is ob.medium.com. 633 00:38:39,340 --> 00:38:43,299 And I've had a couple of blog posts go viral. 634 00:38:43,440 --> 00:38:46,679 I mean, the original one that kind of set me off 635 00:38:46,679 --> 00:38:48,159 in this journey was the one I wrote 636 00:38:48,159 --> 00:38:51,280 about playing Dungeons and Dragons with my kids in GPT-4. 637 00:38:51,619 --> 00:38:54,760 And that got almost a million page views on Twitter. 638 00:38:55,599 --> 00:38:56,440 Which is insane. 639 00:38:56,719 --> 00:38:58,859 Like I was like, okay, well, 640 00:38:59,000 --> 00:39:01,119 it reminded me of like the old times 641 00:39:01,260 --> 00:39:04,340 where you could blog something and get traction 642 00:39:04,340 --> 00:39:07,840 and got me excited about working on AI stuff. 643 00:39:07,900 --> 00:39:09,340 And then recently I wrote 644 00:39:10,359 --> 00:39:13,219 about why Rubyists should be excited about AI 645 00:39:13,219 --> 00:39:16,799 and why we have a shot at like creating really cool things. 646 00:39:17,780 --> 00:39:20,679 Because culturally, if I had to put it in a nutshell, 647 00:39:20,679 --> 00:39:24,659 I'd say as Rubyists and Ruby on Rails people, 648 00:39:25,299 --> 00:39:29,960 we love magic, like the whole convention 649 00:39:29,960 --> 00:39:31,799 over configuration kind of ethos. 650 00:39:33,239 --> 00:39:34,419 Everything we get around Ruby 651 00:39:34,440 --> 00:39:39,340 and Ruby I find to be very humanistic, culturally. 652 00:39:39,719 --> 00:39:42,739 Like I think that that gives us superpowers 653 00:39:42,739 --> 00:39:46,299 when it comes to like actually using this AI technology, 654 00:39:46,419 --> 00:39:47,260 which is humanistic. 655 00:39:47,859 --> 00:39:49,760 So I've been evangelizing that. 656 00:39:50,400 --> 00:39:53,059 A lot of it is admittedly hand-wavy and has, 657 00:39:54,979 --> 00:39:58,260 could be picked apart, but it's aspirational. 658 00:39:58,520 --> 00:40:00,299 And I wrote about it on my blog, 659 00:40:01,539 --> 00:40:03,179 if people wanna go find it. 660 00:40:03,520 --> 00:40:06,599 Yeah, I mean, that got a lot of traction as well. 661 00:40:06,719 --> 00:40:08,520 That got a huge number of reads 662 00:40:08,520 --> 00:40:09,820 and it's kind of passed around a lot. 663 00:40:10,059 --> 00:40:12,099 So I think there's something there. 664 00:40:12,659 --> 00:40:15,580 So yeah, I blog regularly on Twitter. 665 00:40:16,700 --> 00:40:17,099 Excellent. 666 00:40:17,400 --> 00:40:18,619 I'll pull up those articles 667 00:40:18,619 --> 00:40:20,500 and include them in the show notes for people as well. 668 00:40:20,960 --> 00:40:23,840 And I appreciate you being a, 669 00:40:23,840 --> 00:40:25,739 help be a spokesperson for Ruby and AI 670 00:40:26,039 --> 00:40:27,200 because definitely interesting. 671 00:40:27,219 --> 00:40:29,659 And I think I agree a lot with that sentiment 672 00:40:31,119 --> 00:40:32,840 and curious to see where this kind of goes 673 00:40:32,840 --> 00:40:34,299 in like knowing that the communities 674 00:40:34,299 --> 00:40:35,719 can kind of help drive this stuff. 675 00:40:35,739 --> 00:40:37,419 And it's like, hey, like there's a perception 676 00:40:37,419 --> 00:40:40,059 that Python's primarily used in the science realm 677 00:40:40,059 --> 00:40:42,419 because they've got a bunch of math functionality 678 00:40:42,419 --> 00:40:45,559 and stuff like, none of these things are predetermined 679 00:40:45,559 --> 00:40:48,440 necessarily it's up to like the communities need to show 680 00:40:48,440 --> 00:40:49,380 that we can do this stuff 681 00:40:49,380 --> 00:40:51,219 in other technology stacks as well. 682 00:40:51,559 --> 00:40:53,619 And to show how simple we can make it 683 00:40:53,619 --> 00:40:55,119 and how readable it can be. 684 00:40:55,400 --> 00:40:57,960 And does that, that'd be interesting 685 00:40:57,960 --> 00:40:59,460 to see how that kind of pans out. 686 00:41:00,179 --> 00:41:04,320 I think, like you were there in the beginning as well, 687 00:41:04,400 --> 00:41:04,840 I remember. 688 00:41:06,119 --> 00:41:08,679 And what Rails did for a lot of us 689 00:41:08,679 --> 00:41:12,780 is give us the opportunity to start consulting companies, 690 00:41:14,440 --> 00:41:18,359 to become famous gem, Ruby gem authors and maintainers, 691 00:41:18,700 --> 00:41:20,500 like to start significant projects 692 00:41:20,500 --> 00:41:21,799 that got significant traction, 693 00:41:21,940 --> 00:41:24,739 to write books, the Rails way, things like that. 694 00:41:25,000 --> 00:41:28,359 So I see the same thing happening right now. 695 00:41:28,599 --> 00:41:33,219 So if you're either just getting started in your career 696 00:41:33,219 --> 00:41:34,919 and you want a way to stand out 697 00:41:35,539 --> 00:41:39,760 or to make significant leaps and bounds in your career, 698 00:41:40,679 --> 00:41:44,099 I think there's no better time than I can think of recently 699 00:41:44,700 --> 00:41:46,619 because it's kind of one of those moments 700 00:41:46,619 --> 00:41:47,940 where all the rules are changing. 701 00:41:48,099 --> 00:41:49,320 I mean, that's what happened. 702 00:41:49,799 --> 00:41:53,239 That's essentially what happened in 2005 to 2007 703 00:41:54,000 --> 00:41:55,380 when Rails came out. 704 00:41:55,640 --> 00:41:58,880 It was like all of a sudden the old rules were thrown out. 705 00:41:59,460 --> 00:42:02,280 There was all this interesting, cool stuff to be done. 706 00:42:02,400 --> 00:42:04,280 And I think that's happening now. 707 00:42:04,479 --> 00:42:07,919 And actually because the world is different now 708 00:42:08,520 --> 00:42:10,520 and for instance, a lot of this cool stuff 709 00:42:10,520 --> 00:42:12,219 is being done in Python and JavaScript, 710 00:42:13,020 --> 00:42:15,760 as Rubyists, we can look at some of the stuff 711 00:42:15,760 --> 00:42:16,780 that's getting traction. 712 00:42:17,400 --> 00:42:19,200 And I think we can do what they're doing 713 00:42:19,200 --> 00:42:20,780 actually better than them 714 00:42:21,200 --> 00:42:23,059 because we have more powerful tools 715 00:42:23,460 --> 00:42:24,840 and because of our culture 716 00:42:24,840 --> 00:42:26,359 and because of our way of looking at it. 717 00:42:26,359 --> 00:42:29,179 And we're certainly not afraid to do things that don't scale. 718 00:42:29,700 --> 00:42:31,719 I mean, Rails still doesn't scale, right? 719 00:42:31,719 --> 00:42:33,039 I mean, according to the public. 720 00:42:33,039 --> 00:42:38,119 So I think a lot of the things that, 721 00:42:38,640 --> 00:42:39,900 for instance, I'm advocating in the book 722 00:42:40,380 --> 00:42:43,239 are because of the way that they're driven by AI 723 00:42:43,239 --> 00:42:46,340 at the moment, since AI is expensive and slow, 724 00:42:47,020 --> 00:42:49,119 are subject to those kinds of criticisms. 725 00:42:49,179 --> 00:42:50,400 Like, why would you do this? 726 00:42:50,659 --> 00:42:52,679 This doesn't scale, you know? 727 00:42:52,799 --> 00:42:56,260 But the way that the critic would say it doesn't scale 728 00:42:56,260 --> 00:42:59,559 is exactly the same way that they said it didn't scale, 729 00:43:00,200 --> 00:43:02,440 you know, that Ruby on Rails didn't scale. 730 00:43:03,039 --> 00:43:05,479 Or that Java didn't scale back in 1995 731 00:43:05,479 --> 00:43:06,700 when I started in Java. 732 00:43:06,979 --> 00:43:09,479 You know, like I kind of seen the same story play out 733 00:43:09,780 --> 00:43:12,119 time and time again, which is why I'm not, 734 00:43:12,500 --> 00:43:14,320 it's not only that I'm not scared of it. 735 00:43:14,340 --> 00:43:16,900 I love it when I find something that's super cool 736 00:43:17,200 --> 00:43:18,979 that doesn't scale, quote unquote, 737 00:43:19,900 --> 00:43:21,320 because the more that people say, 738 00:43:21,479 --> 00:43:22,919 hey, don't do that, that doesn't scale, 739 00:43:23,080 --> 00:43:24,340 the more it's like, okay, well, 740 00:43:24,919 --> 00:43:27,020 they're leaving it to us to have fun with 741 00:43:27,020 --> 00:43:28,500 and to innovate with, you know? 742 00:43:29,239 --> 00:43:31,539 Good food for thought there for us to kind of ruminate over. 743 00:43:31,900 --> 00:43:33,320 All right, with that, it's been such a delight 744 00:43:33,320 --> 00:43:35,599 having you stop by and talk shop with us, Obi. 745 00:43:36,580 --> 00:43:37,340 Thank you, Robbie. 746 00:43:37,460 --> 00:43:41,679 I mean, it's a pleasure to see you as always.