290: Open AI to Operator: There is a DeepSeek Outside the Door

[00:00:00] Speaker A: Foreign. [00:00:06] Speaker B: Welcome to the Cloud pod where the forecast is always cloudy. We talk weekly about all things aws, GCP and Azure. [00:00:14] Speaker C: We are your hosts, Justin, Jonathan, Ryan and Matthew. [00:00:18] Speaker A: Episode 290Recorder for the week of January 28, 2025. OpenAI to operator. There's a deep seek outside the door. Good evening, Ryan, Matt and Jonathan. Full house. Yeah. [00:00:31] Speaker B: Hey guys. [00:00:32] Speaker C: Hello. [00:00:32] Speaker D: Two weeks in a row. [00:00:34] Speaker A: It wasn't a full house last week, was it? [00:00:36] Speaker B: No, I wasn't here last week. [00:00:37] Speaker A: Yeah, Jonathan wasn't here. [00:00:38] Speaker D: Never mind. [00:00:38] Speaker B: That's how much you miss me. [00:00:40] Speaker A: Yeah, that was. That was awkward. [00:00:43] Speaker D: There was a few very large tangents last week now that I'm remembering the episode, so. [00:00:47] Speaker A: Oh yeah, we rat holed so hard last week, Jonathan. It's definitely going to listen to like we went on a whole rant about 1Password @1 point and a couple other things. [00:00:57] Speaker D: Yeah, but how did it start again? It was odd, like multi sessions in AWS or something. [00:01:03] Speaker A: Oh yeah, the multi session feature and talking about using passwords to log into multiple accounts. Yeah. All right, well let's get right into. AI is going great I think. Introducing the Genai platform simplifying AI development for all on Digital Ocean if you're struggling to find that AI GPU capacity on your main cloud provider, DigitalOcean is pleased to announce their DigitalOcean Gen AI platform is now available to everyone. The platform aims to democratize AI development, empowering everyone from solo developers to large teams to leverage the transformative potential of generative AI. On the Genai platform, you can build scalable AI agents seamlessly, integrate with workflows, leverage guardrails and optimize efficiency. Some of the use cases they highlighted are for chatbots, E commerce assistance, support Automation, Business Insights, AI, AI driven CRMs, personalized learning and interactive tools. I was a little surprised not to see a training capability or access to models. So, you know, hopefully those things are coming to this platform soon. But the ability to run some scalable AI agents on something other than one of the big clouds is welcome. I appreciate it. [00:02:07] Speaker B: Yeah, I mean inference cost is really the big driver though. So once you, once you build something that's, that's done. But it's nice to see somebody focusing on delivering as a service rather than $50 an hour compute for training models. This is right where they need to be. [00:02:26] Speaker A: Yeah, I think it's a good spot for them. It's a good niche that I think they can be really effective at. So glad to see. [00:02:31] Speaker B: Yeah, my only beef with them is they love their Video tutorials and I hate video tutorials. Just give me the text so I can scan through it. [00:02:40] Speaker C: Oh, really? They used to be so good at write ups. I didn't realize. [00:02:43] Speaker B: I kind of wonder if people just visiting their site for the write ups. [00:02:47] Speaker A: I used to do that. I used to do that. [00:02:49] Speaker B: Yeah, I used to do that too. [00:02:52] Speaker A: But yeah, I think it's also like, it's a youngin problem because we're old men now. Yeah, all the young 20 somethings, they want videos and I agree with you. I can't stand the videos. I just want. Give me the five steps to follow, please. But nice thing with AI It'll solve my problem too because I'll just point AI at it and say, hey, make this into steps and then it'll watch the video for me. [00:03:11] Speaker C: That's my help. [00:03:12] Speaker B: Yeah, I didn't try that. Summarize this video for me. Yeah, I can do that. [00:03:15] Speaker A: Yeah. Turn this video into workable steps. Yeah, thanks. [00:03:19] Speaker B: I was, I was looking at the integrate the Gen AI chatbot into your WordPress website so we could have a virtual Ryan on. On the CloudPub website. [00:03:30] Speaker A: You're talking right there too. Snark on demand through a chatbot. [00:03:34] Speaker D: Way too much snark. Way too much snark. [00:03:36] Speaker C: Guys, we better can't leave that unchecked. [00:03:39] Speaker A: I don't know. This might be a great way to drive listenership. I don't know. And then if you, you know, if you get, if you want to pay for the premium subscription, you get Jonathan's British accent read to you of his snark. So it's. It's Ryan Sniper with Jonathan's accent. Like it could be. This might be a good revenue model. [00:03:57] Speaker D: Again, AI is doing great. Or how AI. [00:04:00] Speaker A: How AI ML makes money. [00:04:03] Speaker D: There we go. [00:04:04] Speaker A: Yeah, well, OpenAI is releasing the preview version of their agent that can use a web browser to perform tasks for you. And they've called it Operator, which I'm not happy about the name, but it's okay. It's available to all OpenAI Pro users, which I was excited because I was like, oh, I think I saw my subscription for a couple more days. And then I remembered, oh, yeah, no, the Pro one's the one that cost $200 that I said I was absolutely refusing to pay for. But at least now it makes more sense why it cost 200 than it did when we made fun of it in December. OpenAI says this is currently a research preview, meaning it has limitations and will evolve based on your feedback. Operator can handle Various browser tasks such as filling out forms, ordering groceries and even creating memes for you. Well, this is use case that I need. The ability to use the same interface and tools that humans interact with on a daily basis broadens the utility of AI helping customers or helping people save time on everyday task while opening up in a new engagement opportunity for business. You can find it at operator.chatgpt.com and operators powered by a new model called Computer using agent or CUA. Combining GPT4O's vision capability with advanced reasoning through reinforcement learning. It is trained to interact with a GUI so it is going to come to the other description levels. So again at some point the $200 price tag won't make any sense, but for right now it's the only way to get access to this thing. And the use case that they showed was that basically find me and book me the highest rated one day tour of Rome on TripAdvisor and then it used TripAdvisor to book your thing, which those of you who ever use TripAdvisor you would never probably do. So OpenAI should probably recommend you use a different tool than TripAdvisor to book your thing. But that's okay. Yeah, minor complaints. [00:05:38] Speaker C: Yeah, I want to play around with this for certain tasks because you know, the minute it can start interacting at this level, you know you can get a lot of value. Although like I find myself sort of like thinking through these things and I realize I'm not really willing to give up the control. Like you know, ordering groceries was the example. I'm like, I don't know if I'm willing to let that. [00:05:57] Speaker A: I mean I love the idea of like pointing my RSS feeds that we use to track stories for the show and saying, you know, go to the thing and summarize the article and put the link and the article summarization in the show notes for me. That would be great. So like I'm kind of looking forward to some of these things can do for me and the workflow for the copod. [00:06:15] Speaker C: Yeah, I mean if I ever finish it, the stupid project it would do that without the ui. [00:06:22] Speaker A: And the way this is going is I'll be able to just code it myself with my rudimentary coding skills and AI helping me do it. So I don't need you anymore. [00:06:29] Speaker C: I'm using AI to code, so I. [00:06:32] Speaker A: Think we're all using AI to go these days. [00:06:34] Speaker B: Yeah, I like uprights and what I really like to see though is I don't want to have to have it open in the browser. I don't have to have, you know, I don't want to watch it doing its work. I want to say, hey, go, go ahead and do this thing. And if it needs credentials, I'll give it credentials or find a way to securely give it credentials. But you could just give a, give an agent these tasks like keep checking this, keep checking the prices of these flights until it's under this much money. I don't want to have to sit there and watch it. I want it to be a background operation. [00:07:03] Speaker A: Well, I mean, you already have that with Google flights. But what would be more interesting is can you make it go log into different browsers around the world to find me the best country to buy my ticket from? [00:07:14] Speaker B: Ah, yes, that too. [00:07:15] Speaker A: See, that'd be more interesting. [00:07:17] Speaker B: Or even the best state. I read about, you know, flights from California being more expensive if you book them from here versus somewhere else. [00:07:24] Speaker A: I tried to go test that the other day because I was, I was looking for a flight to somewhere and I was like, I wonder if that's actually true. And so I was like, I'll go log into my, my vpn. And then five minutes later my outlook is like, you need to log back in again. I was like, oh crap, I was flagged. I didn't turn off Outlook from logging in from a weird country. [00:07:40] Speaker B: Yeah. [00:07:43] Speaker A: I was like, and that's my fifth password change of the month. Great, thanks. All right, well, the world and AI has been turned upside down this week. And gentlemen, I don't know if you know this, but there's a new thing called Deep Seek which there's a lot of jokes I can make, but I'm not going to do that. Last week, Chinese AI lab Deep Seq released its new R1 model family under an Open MIT license, with its largest version containing 671 billion parameters. The company is claiming that the model performs at levels comparable to OpenAI's O1 simulated reasoning model on several math encoding benchmarks. In addition to the main deep seq R1 main and deep seq R1 models, they released six smaller distilled versions ranging from 1.6 billion to 70 billion parameters. And these distilled models are based on existing open source architectures like Qin and Llama training, using data generated from the full R1 model. The smallest version can run on a laptop, while the full model requires a far more substantial computing resources. This stunned the AI market as most open weight models, which can often be run and fine tuned on local hardware, have lagged behind proprietary tools from OpenAI, Claude and others. Sorry, not Claude, Gemini, sorry. In so called reasoning benchmarks. Having these capabilities available in an MIT licensed model that anyone can study, modify or use commercially potentially marks a shift and what's possible with public model. The stock market panicked in response as they do with companies like Nvidia, down 17% yesterday. Although I think they were back up today this week based on the fact that the deepsea jumped to the top of the App Store with free downloads and in the fact it's low cost and freely available. There's only three things that have investors and researchers shocked about the Deep Seq. First, the Chinese startup that trained the model for only $6 million allegedly reportedly 3% of the cost of training OpenAI01 as a so called side project while using less powerful Nvidia H800AI accelerator chips due to US export restrictions on cutting a GPU, they apparently acquired these GPUs for massive crypto mining previously it appeared just four months after OpenAI announced 01 in September, which means it was developed very quickly and released under the MIT license, meaning it's available to anybody to use as long as you don't care about looking for things like Tiananmen Square. This led investors to see that American tech companies which have thrived on proprietary enclosed models have no moat, which means they are technological lead led by cutting edge hardware. Impressive bankrolls doesn't protect them from startup challenges from China and other emerging countries. The question is it any good? And you know, as long as you're not looking for, you know, things that are upsetting to the Chinese government, it's pretty decent. I like some of the things it does around giving you the path it did for the reasoning. So it explains the reasoning it did and why it got to the output, which is kind of cool. But you know, I think, you know, you can game these benchmarks pretty easily and I wonder about how much of it's game to make the benchmarks look good versus actual reality. But I don't know. I'm gonna shut up now so Jonathan can tell us how I'm wrong. [00:10:26] Speaker B: Oh no, they're not wrong. It's really good. I didn't download the app, but I did go to the website and play with it and it is incredibly good for free. [00:10:37] Speaker C: The impact the story has had this week has been a roller coaster like and I don't know if that's because I've been busy and sort of half paying attention and you know, it wasn't really until we were preparing for the show that I really like dove in to figure out what, what this was after seeing it. Like, you know, first it was like a Chinese app taking over the phone. So I thought it was security concerns and all this stuff, especially with all the tick box stuff that's going on. And then to find out it was an AI model, I'm like, oh, it's just there's other Chinese AI models. But then, then the impact to Nvidia stock. So it's kind of crazy to see all of this happen and it really just, you know, proves that, you know, the AI market right now is just very volatile and can very subject to change. [00:11:21] Speaker D: Well, even the app though, I think I was reading like somebody had started breaking down and all the data is essentially going to China. So like I've seen a bunch of sick I interested. [00:11:33] Speaker B: Of course it's going to China. That's where they're based. I read this thing, oh no, don't install the app. It's sending your data to China explicitly. I'm like, really? A Chinese company that's running their model on computers in China is receiving your data when you want to use their service? Seriously, come on. [00:11:49] Speaker D: But there's a bunch of companies and security alerts that came out of the last 24 hours. I was like, don't use this for work or anything else. I was like, you shouldn't really use anything that you want internal in ChatGPT or any other ones because they're all taking him training off of IT guys. So to me this was like, this is no different than anything else. So what? I guess it's China versus they don't. [00:12:14] Speaker A: Want your data to go to China. That's their big risk. And that's where TikTok has been in the news a lot. And I know all these things. But it's interesting to me that the stock that took the biggest bloodbath out of this was Nvidia, which Nvidia is still used in the solution. It's still Nvidia chips. So I'm like, I don't quite understand. Like the reality is you still need Nvidia chips, you still need all the stuff. Like maybe you don't need the highest and fastest cutting edge ones to do everything. But like Amazon was up on the news, Google was up on the news. I'm like, well you know, like they're also spending a lot of money on chips and expensive AI infrastructure. So it doesn't really make sense for the companies that got dinged by the stock market versus the ones that Kind of avoided it. I can maybe make the argument, well, Amazon's actually in better shape now because they could take this model and they could host it in their infrastructure and have a competitive solution to potentially Gemini or to ChatGPT. But again, it's sort of weird, the market reaction. And today definitely Nvidia is back up 10 bucks a share today. But I'm curious to see how it kind of changes for the rest of the week. But I also expect that we'll hear a lot more in the next week or two about how it's actually limited in some key ways that the benchmarks don't show. [00:13:23] Speaker B: I don't think so. [00:13:24] Speaker C: But even, even if it's relatively close, the fact that they trained it for a couple million dollars. [00:13:30] Speaker A: Well, they're saying that $6 million doesn't include a lot of the hardware costs because again it's leftover hardware from their crypto mining and doesn't include the labor costs, which I mean is China. So they're probably very low, but. [00:13:41] Speaker C: Right. [00:13:42] Speaker B: I saw, I read the paper and I read the explanation of how they came up with that fear and it's very reasonable. It's how much it would cost them like fair market rates to rent the GPU hours that they needed to train the model. And so that's, that's really good. But I suspect they also started from a model that was already being trained by somebody else. So there's a bit of extra work that's gone in that wasn't included in that cost. But I think the real reason that Nvidia took it took a hit is because they use these older H800AI accelerators and their PCIe cards and Nvidia charge a huge upsell upcharge on the, the high performance cards that don't use PCI. They use their proprietary, what is it, the SXM4 or something. And you know, they much, much bigger bandwidth, much bigger everything, you know, but much more expensive. And the fact that they deepsea trained this model on basically commodity hardware effectively is probably the real reason they took a hit because, well, if you can do on commodity hardware then what's the point in paying extra? The other stuff? [00:14:50] Speaker A: Well, I mean, I wonder, you know, everyone's analyzing and reverse engineering the crap out of this model. I suspect that they, you know, when you have limited hardware and you have different constraints, you make better choices in some software design things and more efficiencies than you can get. And my guess is that people will figure out the efficiencies that they did to make this work. And then those will go to the bigger models on the higher hardware and then those will become benefits to OpenAI and Gemini and Claude and all the others that are out there. And that'll actually, you know, potentially this helps a bunch of companies leapfrog where they're at right now based on this technology and research that they did. Because I mean, when we, you think about the fact that, you know, the space shuttle, by the time it retired, was still running on the original 386 or 486 processor and software that was designed for the first shuttle, you know, 30 years prior. You know, when you make really small, efficient software code, you can run it on, you know, very low powered hardware. And because we've had Moore's Law for so long that a lot of software we build today is just not very efficient in general. And so when you're forced by a constraint like hardware, you make better choices sometimes in how you design things. [00:15:56] Speaker B: Yeah. [00:15:56] Speaker D: What are you talking about? Excel was only using 28 gigabytes of memory on my laptop the other day. [00:16:02] Speaker C: No, there's no, we all know that solving any technical problem is just give it more, more resources, more hardware, more cpu. Yeah, more cpu, more memory, everyone. [00:16:11] Speaker A: Yeah, I still never forget that conversation when we were, we were deploying, I think it was T instances and some engineering leader was like, my laptop has more memory than that server does. Like. Yep, yep, you're right. But like the idea is that you run hundreds of those little guys. Yeah. And scale out horizontally and you don't need all the, the heavy expensive power of the big boxes that's lost on Windows. Developers for some reason don't really understand. [00:16:36] Speaker B: I think one of the other issues with deepseek is that they're able to offer the service for like a tenth the price of ChatGPT for API access. And I think when you have to factor into that is OpenAI had to buy their hardware. The hardware that Deep Seq are using has been paid over, paid for like thousands of times over by crypto mining. And so they, they're not having to pay that, that debt anymore. So this is really just cost operation at this point, like electricity consumption and you know, website, couple of servers, whatever. [00:17:09] Speaker D: But that goes back to the commodity hardware, you know, can they. [00:17:12] Speaker B: It does, but I, I don't think it's a realistic comparison. It's great to say great. You can get how many million tokens for 10 cents with Deepsea. But the, the fact remains businesses aren't going to want to ship their data to China. They can only offer it for that price while there isn't enough interest to reach capacity. [00:17:31] Speaker D: So the real question is how long before the cloud providers have a bottle horse to bear in all their cloud providers. [00:17:38] Speaker B: Oh, the model's available. You can get it already. It's pretty big and there's these distillations you can get as well. Hugging place have got it. Anyone could run it right now. You could download it from. From hugging places somewhere. [00:17:49] Speaker D: No, no, I meant like aws, Google, Microsoft, all hosting it in there. [00:17:53] Speaker C: Yeah, I don't think anyone's made an announcement that they're hosting in their marketplace. [00:17:56] Speaker D: Yet, but question is, how long is it weeks? Is it days? [00:18:01] Speaker B: You could take the model and run it yourself there now just because it's not one of their offers. [00:18:05] Speaker D: Yeah, I know, I meant, but I. [00:18:07] Speaker B: Meant I don't think they will because of the lack of transparency around what's included and what's not included and what's been censored and what's. I think it's just too much data around the source. But people are working really hard now to reproduce the same outcome here in the US and around the world. [00:18:26] Speaker A: Yeah, yeah, I suspect that you'll start seeing people talking about like, oh, here's how they did it, here's the things that we can do and we can build those same things into our model and we actually get this much more efficient, this much quicker. So like I said, I think it becomes maybe either a moment of inflection for the AI industry that they need to focus on efficiency more and how to use more, you know, use hardware more efficiently, but then that has a result in a much better outcome. Or, you know, maybe, maybe we're all wrong in our assumptions. I don't know, we'll see. [00:18:56] Speaker C: Maybe we shouldn't have killed that story. [00:18:58] Speaker A: Where, you know, Google's going to save. [00:19:00] Speaker C: Another $1 billion in traffic because they're clearly not going for the constraints model. [00:19:06] Speaker A: Yeah, well, I mean, and that, that the thing about that cloud story was that's on top of the 8 billion Amazon's already invested into them, plus the fact they're trying to get a VC round for another two or four billion dollars, which will put them at a $60 billion valuation. There's definitely a lot of money. Although I wonder if that cloud investment for the vcs, all of a sudden, maybe they're getting a little cold feet based on the steep seek data to see. All right, well, we created a new segment this week, Cloud Tools, which is not Just what we call Ryan. So basically we've kind of always traditionally talked about tools, we talk about general news, but we've given a dedicated section now because there's been a lot of really cool tools coming out in the last year that I've wanted to talk about that never felt really fit. So I just executive decision that we're creating a cloud tool section. So you're welcome. First up, Hashicorp has a new module as part of Terraform called TF Migrate. Migrating from HTTP Terraform to Terraform Enterprise or to on prem Terraform can be a bit of a pain, especially when you deal with refactoring your state files or if you're just even refactoring your own code into being more optimal patterns like I did a few about a month ago with Claude. I did deal with some state challenges which was just me, you know, dealing with the fact that I changed things to different modules and I moved things into different files and the way that it segmented things broke my state. So I had to deal with that. But with this new migrate, when you need to migrate between the different things, the state file management will become much easier. As you run TF Migrate and utility for automating state migrations to HP Terraform and Terraform Enterprise or any really Terraform direction you want to go. It also simplify your workstation setup and supports modular refactoring. Right now it's still pretty limited but can get a lot of stuff done. And they are looking to further improve TF Migrate by adding ability to integrate with your source code system like GitHub to enhance migration workflows by embedding migration configurations directly into repositories. Enhancing and extending migration capabilities to support variables, modules and private registries between multiple Terraform deployment options. Improve its handling of moving sensitive data from during migration such as secrets or access tokens and further integration with Terraform Enterprise Terraform Cloud to enhance governance by offering centralized control over migration tasks, audit trails and policy enforcements which are all becoming in the future. So glad to see this kind of existing. I know you guys have all probably felt the pain of the state problem as well. So this one is I think all welcome to all of us. [00:21:29] Speaker C: Yeah, no, it's just anytime state conflict due to either like data recovery or just you know, trying to reconcile manual actions that have happened since or anything like that. It's always so painful. So I'm really happy to see tools like this exist. And it's just another example of Hashicorp building in like really usable Functionality like whether it's upgrading your code to the newest terraform version or migrating state file. I like this a whole lot. [00:21:57] Speaker B: They only had to build this because the rest of it sucked. [00:22:01] Speaker D: Curious about the actual migration piece because at my day job I foresee there's a specific thing that I'm like, okay, we made this layer be way too large. It's going to bite us in the butt in about six months. So I'm like, well, in six months from now. Well, we have to deal with it. This will be an interesting test to see how it works. [00:22:21] Speaker C: It's really tricky, right? And I know, I know several, you know, projects where people have, you know, basically tried to automate this, like because they wanted to, you know, they had a big terraform, you know, deployment all based off of like a root structure where it was all bound together. And that made sense at a certain scale, but then breaking that out so that you can, you can sort of delegate all that out becomes really a challenge and splitting up state into different things. And so like, there's a lot of things that I think that in this tool, if they add enhancements would be great. [00:22:52] Speaker A: Yeah, I hope that they're going to continue to enhance this quite a bit because, you know, the state file, I would say, is probably the one Achilles heel of terraform in it. How it's designed and architected and having to deal with the state file, where do you put it, how do you manage it? Secrets in it where it was a big problem for a long time, although that's now getting less and less of a problem. This terraform migrate fixes a big problem with it. But yeah, I think, you know, IBM scale is probably going to be a benefit to HashiCorp in this particular case, where they're going to be like, these are things that we need to manage IBM infrastructure at scale. And I think that'll help hopefully in a big way. All right, sysdig, who you guys may know as a cloud security vendor, has announced the launch of stratoshark, a new open source tool that extends wireshark granular network visibility into the cloud and provides users a standardized approach to cloud system analysis. And I was very excited about this because wireshark and I are, we're old friends. We go back to my network engineer days when I used to deal with switches and routers and firewalls and I used to do all that stuff before I got into leadership. And then they said Wireshark is over 27 years old. And then I felt super old. They did say it has over 5 million daily users and has been downloaded over 160 million times to help you analyze network traffic control issues. And I still even use it today sometimes when Ryan gets a packet capture. Send me the packet capture and I'll look at it too. Just as I find it fun, it goes back to my roots. But basically as companies move to the cloud, the benefit of Wireshark has kind of diminished over the years. Unless you use a pcap analysts have lacked the same visibility as comparable open source tool for the cloud environment, but stratashark is hoping to fill that gap with features that unlock a deep cloud visibility to assist in analyzing and troubleshooting cloud system calls and logs the level of granularity and workflow familiar to long time Wireshark users so your network people finally have something they can do in the cloud. Strata Shark leverages the Falco libraries, repositories and plugins which are part of Sysdig's product, to unite deep cloud visibility with familiar Wireshark functionality. Falco is an open source runtime security tool created by Sysdig that detects and alerts on unexpected behavior in cloud native environments such as Kubernetes. There's a quote here from Gerald Combs, who apparently works at Sysdig now, but also was a co creator of Stratoshark and Wireshark. Says Wireshark revolutionized network analysis by democratizing packet captures, a concept that Sysdig brought to cloud native workloads and Falco extended to cloud runtime security. Wireshark's users live by the phrase pcap or didn't happen. But until now, cloud packet capture hasn't been easy or even possible. And Starshark helps unlock this level of visibility, equipping network professionals with a familiar tool that makes system call and log analysis as accessible and transformative for the cloud as wireshark did for a network packet analysis. So yeah, that's the Strata thing. I played with it earlier today, just checking it out. I was slightly disappointed that it right now only supports AWS for the cloud trail integration, but the Linux connection where it will actually connect out to a Linux box running in your cloud environment and pull PCAP works on all clouds, but in general I'm very excited this exists. [00:25:50] Speaker C: Yeah, yeah, no, I mean I had the same reaction of like this is. I never really thought about a Wireshark type of analysis that brought in like CloudTrail, VPC flow logs and system calls on the OS. Right. And so I know that's not where it is today, but like the idea that you could then trace that all the way through and, you know, run it through the filtering and the same sort of display you have in wireshark, like that's kind of. It's just a fantastic idea. And so like I. This is something that I was like, this is, you know, I'm going to follow this very closely because it's neat and it will help me in my day to day. Also, I didn't know that C Systic was a thing, which is their open source sort of binary that you can use directly on host, which I thought was pretty neat. I watched the demo of that and, you know, it's a lot like other tools that you use, like Top and Htop. It's stuff. But it is neat. The filtering and sort of linking that they do with that tool. That's pretty. [00:26:54] Speaker A: Yeah. And the nice thing is this is both a plugin nature. So right now the first plugin is CloudTrail, but I assume that they'll start pulling in VPC flow logs, we'll get a plugin and you know, the Falco dump and the SSH dig, you know, they're able to do basically all the system call captures and log captures from the host directly back to the wireshark or stratoshark implementation. It does seem it's a different binary, which is fine, but once you get inside of it, it's the same wireshark, you know, and love. So, yeah, it's very interesting, it's very cool and I'm very excited to see where this continues to grow and get adoption, hopefully in the market, because if I can keep using Wireshark for another 20 years, be in my career and then retire. [00:27:36] Speaker C: So I remember before wireshark, the network at the packet level was just a black box. So if you're an application developer, you know, you have no idea what's going on in the wire. And the only way I ever learned it was through because of wireshark. And so like learning that, you know, the OSI model and how it all works with the column Schlotz and Synac, all of that stuff. And then thinking about that applying to the cloud holistically, because that's still something that, you know, is still a little, a bit of a mystery to a lot of cloud native developers, like how to trace this all the way through instead of just blaming DNS, which it's always DNS, but it is always DNS. [00:28:18] Speaker A: Brian, come on. [00:28:19] Speaker C: I know, I know. I just love the idea that, you know, something, a tool that can visualize sort of the flow between all those different elements. So super excited. [00:28:29] Speaker D: No, no, I come from that background too, but not quite as deep as you guys did. So, you know, every time I had built a Wireshark, I was like, oh, I'm not familiar enough with this tool. And okay, let me remember how to actually use it to get what I need. [00:28:45] Speaker A: Matt's not as excited about Wireshark. That's all I can see. He was like, you network nerds. [00:28:52] Speaker C: It's a magic trick. I've used Wireshark to, like, sort out issues that people were blaming. All kinds of different things. Like, I remember, you know, sorting through a Java heap problem because of Wireshark outputs and timing differences and a whole bunch of things. Like, it really is sort of like something I can break out. And it looks like, you know, the ancient times tool, but it really does help. [00:29:18] Speaker A: Well, it's one of those great things where you, you know, like you're in a troubleshooting session. They're like, oh, it's the network. I'm sure it's the network. And it's like, well, actually I see here where you know, we pass a packet into, you know, here's the packet return from the host back to you and then you basically close the connection. So why did you do that? And they're like, oh. [00:29:36] Speaker D: Most developers I know don't touch anywhere near that. You know, they're hitting an API at best and. Or, you know, some sort of driver and that's as far as they go. [00:29:46] Speaker A: But that also the problem. The developers are sometimes very obfuscated from their networking stack through other SDKs or drivers they're using. So that's how you find fun bugs. Yeah, in other people's components. [00:29:59] Speaker D: Or you're on something, there's a say. Or you're on four versions too old and we fixed that bug. That's definitely never happened before. [00:30:08] Speaker A: But at least, I mean, being able to point to something in your stack, then at least you can go to that driver's website and say, hey, I have this problem. And they can tell you it's because you're not doing something right or whatever. Or find documentation. Yeah, never. Never happened. All right, our final tool for the week, OpenVox, which I. I will talk about the name in a minute. But this, apparently the community driven fork of Puppet, it's finally arrived. That is called Open Box. This fork sprang from Puppet's owner, Perforce, which I actually know, owned Puppet, moving Puppets, binaries and packages to private harden and a controlled location. Additional community contributors would have limited access to the program and usage beyond 25 nodes would have required you to buy a commercial license from Perforce. And these changes have been resisted by longtime Puppet users and contributors who started this fork, initially referred to as the Open Puppet project. The community now known as Vox Pupuy has settled on OpenVox as the fork's name. They intend to continue Puppet's work while adhering to the open Source principles. A GitHub repository has been set up and discussions are ongoing regarding the project organizational structure and future direction. The intent is to be a soft fork with a desire to maintain downstream compatibility for as long as they possibly can. As well as the Puppet Standards Steering Committee will include seats representing the whole community, including offering a seat to Perforce whether they want to join it or not. They do say they don't fully plan to follow Puppet's future plans with Puppet, and they do plan to modernize the open box code base and ecosystem. In particular, the developers plan to support current OSes and Ruby versions rather than relying on 15 year old unmaintained Ruby gems. Thank you. Recentering and refocusing a community requirements actual usage patterns will drive development rather than which customers have the deepest pockets and democratizing platform support. Instead of waiting for Puppet to support the current Ubuntu Linux community members contribute to the projects themselves and maintaining an active and responsive open source community. Which means yes, your pull request will finally get reviewed, which is nice to hear. So, I mean, overall, I didn't know this was going on in the Puppet community because one time I tried to use Puppet. Then I learned that Chef existed and I pivoted. So I was never a big Puppet user in general, but I, you know, I'm sad to see another company just totally violating the entire idea of open source. And so I'm glad to see another fork to kind of get around that problem. But you know, if your original product's name is Puppet, OpenVox doesn't make any sense to me. Like, could you at least have gone like, you know, doll or you know, what are like a dummy, like, you know, dummies, something, something that kind of fits into the Puppet thing. Marionette, you know, I don't know. I don't know. There's all kinds of options that would have worked here. I mean, they ran a little long to type, but open Vox is not easy type either. [00:32:51] Speaker C: Well, Vox Papuli, the original name, right? Like it's, it's. [00:32:54] Speaker A: That would have been bad, that would have been awful. [00:32:56] Speaker C: And the only reason it's named that is like as an FU as well, right? Because it's a Latin phrase that means the voice of the people. So. Which I just looked up. It wasn't like I knew. [00:33:05] Speaker A: I. Yeah, yeah, I looked it up earlier too because I was curious, like that must mean something. And I was like, oh, okay. [00:33:12] Speaker C: And so it's, you know, it's very clear that they're, you know, sort of opinionated about what the current state is of Puppet development and the community is angry. [00:33:22] Speaker D: So we were talking about how Wireshark is old and I've been. I use Puppet. I think my, like one of my first jobs, like 2008, I was like, ooh, okay. I've been using this tool for too long to be fair. I haven't used it in many years because, you know, I've pivoted to other tools or like Chef for ansible or whatever else I've had to do. But I was like, oh God, this. This tool is still around and still widely used. From what I can tell, at least at some of my old contacts, they used it. [00:33:50] Speaker B: So since you're kind of running theme, you know, last search, Mongo Puppet, Terraform. [00:34:00] Speaker D: I assume what's one of them did? Kind of everyone's slowly doing it. Yeah, Chef did, I thought did it a while ago. A couple years ago. Chef did something a couple years ago that pissed off a lot of people. And I thought it was a license change. [00:34:14] Speaker B: They just exist. That's enough. They just built the worst possible, you know, inheritance system of what takes priority over what in Chef recipes. And that's just enough to turn anybody off. I don't know. Like, I mean, I guess the companies, the companies that are pissing off their, their customers and their users, they're going to get their ass handed to them with. With open source versions of the same things. But I think with AI as. As mature as it is and as mature as it's getting, it's not going to be long before you can. You can point out set of AI agents at any product you like and say, build me this thing that does exactly the same thing as this. And by the way, work around these patterns that they have and we'll be able to reproduce anything very cheaply, very quickly. I think I wouldn't want to be in SAS right now or any kind of software, to be honest. [00:35:10] Speaker A: Well, I mean, that's. I was. Some of the things that I'm reading about, you know, the idea of a CRM AI or AI CRM will like destroy Salesforce. I was like maybe. Yeah, but I mean like some of the extraneous things. Yeah, I guess, like, you know, there's a lot of things that are, that are more impactful to like the salesperson. Like, you know, reaching out to the client, responding to schedule a meeting, you know, doing demos like that. I'll probably be driven by AI pretty easily, but you still need a way to track the lead. You still need a way. And like, you know, does that become AI? I don't know. Like that still feels like forms and workflow to me. [00:35:43] Speaker B: Yeah, I think Salesforce would be pretty safe because they're huge integrators and you could build a standalone product, but unless you've got deals with everybody for access to data and APIs, you're not going to go anywhere. I'm thinking more. [00:35:57] Speaker C: And they're going to incorporate AI just like everyone else is into the product directly and so it'll just be their offering. [00:36:08] Speaker B: There are a lot of cloud cost management tools out there, but only Archera provides cloud commitment insurance. It sounds fancy, but it's really simple. Archera gives you the cost savings of a one or three year AWS savings plan with a commitment as short as 30 days. If you don't use all the cloud resources you've committed to, they will literally put the money back in your bank account to cover the difference. Other cost management tools may say they offer commitment insurance, but remember to ask, will you actually give me my money back? Our chair A will click the link in the show notes to check them out on the AWS marketplace. [00:36:48] Speaker A: All right, let's move on to AWS. CloudWatch provides execution plan capture for Aurora PostgreSQL. Now this basically collects the query execution plans of top SQL queries running on your Aurora PostgreSQL instance and stores them over time. And this feature helps you identify if a change in the query execution plan is the the cause of performance degradation or a stalled query question I ask all the time. Execution plans are available exclusively in the advanced mode of CloudWatch database insights, which I appreciate and I've used for RDS but not for Aurora. So I'm actually curious how they're differing at this point. [00:37:21] Speaker C: It must be at the engine level, right? I don't, you know, I, I mean. [00:37:24] Speaker A: I mean I'm CloudWatch database insights because I've seen the non advanced version. So I don't know what the advanced version gets me over the non advanced. [00:37:32] Speaker C: I mean, I don't know for all you suckers still using relational databases, I. [00:37:35] Speaker D: Guess he's using NoSQL with SQL on top of it. [00:37:40] Speaker C: Yeah. [00:37:41] Speaker A: What non relational database are you using? [00:37:42] Speaker D: He's using Route 53. [00:37:43] Speaker C: I just don't know how to use a relational database. [00:37:46] Speaker A: Well, that's fair. [00:37:48] Speaker C: That's right. [00:37:49] Speaker A: He's using NoSQL by not using SQL, but somehow, probably using DB2 is probably SQL somewhere. Yeah. SQL lights basically secretly in all of his apps. He just doesn't know it because he's using some abstraction. [00:38:01] Speaker C: Could be, it really could be Python import. [00:38:08] Speaker A: AWS Client VPN announces support for concurrent VPN connections Making all of your security people sad and everyone who actually needs to do real work very happy. This is basically ability to have concurrent VPN connections. This is going to be a theme this month with Amazon. We're going to do concurrency. This is the concurrency quarter where we do concurrent console access that can current VPN connections. The feature basically allows you to securely connect to multiple client VPN connections simultaneously, enabling access resources across the different environments that you have. [00:38:34] Speaker D: And now we have to use Wireshark to figure out where all of our. [00:38:36] Speaker C: Connections are going, correct? Yeah, but I mean we've all been part of like an integration or an acquisition or something where you have to log into one VPN to go do like your general office or your intranet or something like that. And then to log into a production network, you have to log off with that vpn, log into another one and then do that and you can't go back and forth. And I like, I like the model and I realize it's more complex and it's difficult, but you're, you're still running over private, you know, lines and traffic. [00:39:10] Speaker D: Security orgs will never understand that concept. [00:39:12] Speaker C: So I think it will sort itself out and I think it'll stop hopefully security orgs from just doing like full routing to everything through the VPN because it's just not sustainable anymore. [00:39:23] Speaker D: It will be extremely useful for the consultants because the amount of VPNs I had on my old laptop when I was a consultant was a new level. And like I would like, oh, let me go jump and help this other person forget that I log off this VPN. And it was always a nightmare of VPNs. At one point I had like different like parallel work machines for clients because I was trying that out and that just became its own nightmare too. [00:39:45] Speaker A: Well, we're going to talk about something that is really just lead into a rat hole. So AWS is announcing they're expanding an edge location to the Kingdom of Saudi Arabia. This comes on top of the fact that they're building a whole region in Saudi Arabia at some point. The new AWS Edge location brings the full suite of benefits provided by Amazon cloudfront to the region, which doesn't really solve the big problem of Saudi Arabia, which is that data sovereignty laws are basically so strict that you must have your data in Saudi Arabia, even though there's no viable doctor solutions there either. Which is just annoying. But you know, we're doing our own research on KSA for other reasons on the Google Cloud, and Ryan stumbled across this little tidbit from the documentation for the Kingdom of Saudi Arabia KSA location. You must use location specific URLs to access the jurisdictional Google Cloud console, as well as some of the methods and commands in the Google Cloud cli, the Google Cloud client libraries, and the Security Command center API. [00:40:39] Speaker B: Whoa, what a mess. Yeah. [00:40:42] Speaker A: Why? Like who, Why? I don't understand. [00:40:46] Speaker C: And I, I want to know, is it, is it that the, the compliance requirements are so asinine that it forced this, or was it, is this just a really terrible implementation? [00:40:58] Speaker A: I mean, if you've ever read the requirements from Saudi Arabia, they are pretty bad. Yeah, like they're, they're horrendously written and they are. There is no wiggle room on launch stuff. But I didn't remember seeing anything about the metadata for the. This is basically saying the metadata, if the configuration is stored in Saudi Arabia, but even the tooling to access the metadata has to be in Saudi Arabia, which just doesn't make any sense. [00:41:21] Speaker B: Yeah, it's like they have to host the DNS zone in Saudi Arabia as well. Everything is where does the. [00:41:29] Speaker C: Which is funny because that's exactly the type of research I was doing the day job is, you know, how to, how to enforce data sovereignty and how to, you know, you know, alert and those things. And so we're reading through these. I'm like, oh, that's gonna suck. I don't want to do that. [00:41:45] Speaker A: Yeah, so we're gonna have to do some more research on that one. They talk to our reps. We'll, we'll keep you posted. Because that was like, oh, interesting. And I wonder if that's the same thing good for Oracle if they have the same type of control limitations. Like, I gotta look at, I gotta look at a bunch of docs. But anyways, that's a future research rat hole that we were like, wow, that's unfortunate. AWS is announcing the general availability of AWS Managed Notifications, a new feature of AWS user notifications that Enhances how customers receive and manage AWS health notifications, which I never want, so please don't send them to me. This feature allows you to view and modify default AWS health notifications in the console notification center alongside your custom notifications, such as CloudWatch alarms. In addition, this also now works on the mobile app, which I do appreciate that because I can't tell you how many times I've got an alert from Amazon that's just spamming my mailbox and I just want to turn it off while I'm on the mobile and you can't do it, but now you can. So I appreciate that part of this feature, but in general, you know, this notification business that Amazon has been trying to refactor now for the last year or so, and this custom health dashboard bs, like, this is a muddy strategy I don't fully get. [00:42:53] Speaker C: I mean, they've been working towards this in a while, you know, for a long while. I remember previewing something that was similar to this. And so like, you know, the idea is that, you know, instead of blasting the, the email account that you associate with your AWS account, you, you can tune it to specific things and, and have specific, you can have multiple targets depending on the alert. [00:43:19] Speaker A: Right. [00:43:19] Speaker C: And that makes a lot more sense. But yeah, it still hasn't really reconciled itself into like something usable in a lot of ways. Like, it's. I don't know how to get, you know, anyone to read that, you know, their, their database engine is, you know, two versions out of support and they need to update and, but then also have the same list, you know, manage the, the outages that AWS might experience. And so like, it's, it's just sort of weird in order to, to configure this and deal with this. And it's a strange problem that I don't quite know the right solution. [00:43:56] Speaker A: Yeah, I mean, I, I hope that Amazon has thought this through, through more than I have, but what they're doing doesn't feel right to me either. So it's one of these confusion points I'm like, because typically, like when people talk about product ideas and what they're doing, like, it makes sense to me. I get it. Like, but like everything they've been doing about notifications and the health dashboards and all that, like, it's, it's like a problem searching for or it's a solution searching for a problem that I don't fully understand. [00:44:22] Speaker C: Well, and it's, it's not just AWS too. I have a similar sort of response to a lot of the Google notifications on Google Cloud. So it's, it's sort of like this weird like I think customers want too many different things and it's the private people are caught in the middle, that's my guess. [00:44:39] Speaker B: I hope you're reading Google notifications because I just delete them. [00:44:43] Speaker D: I have the same problem with Azure notifications though. Like they send us stuff but you know, because we have 15 subscriptions or whatever, you know, I get 15 notifications and I'm like, cool. I've now successfully ignored all of them and there probably was something useful in there followed by okay, can I set these ones that are like, hey, this thing's Getting deprecated in 2037 to a place somewhere that I can deal with later, not the main production ones, you know, and having some sort of granularity would be nice but you know, at one point that's more fine tuned for what I need than probably other customers. So I feel like it's probably what you guys are presenting where the product team's just stuck between too many requirements. [00:45:26] Speaker B: It makes sense to the native service like this though, because I guess in the beginning you had the root account, all the notifications got there and then IM users didn't have email addresses. We have a big shift from that to federated users who, you know, they have principles from Marktor or somebody else, readyfs and they don't always have email addresses, although usually they do. But I mean it kind of makes sense that they need a way of organizing which messages go where and people just subscribe to them because you might have an org with a thousand people all with eyes on a resources in a couple of different accounts and like you say, you know, you can't send all the notifications everywhere because you just delete them like I do. But yeah, there's like this weird gap, isn't there, between you've got infrastructure as code and we deploy the stuff, it goes in the cloud and then Amazon provide this, the maintenance notification API and you can check to see which resources are going to be rebooted next Thursday. But we're like missing that connection back to the SRE department in a way like either let me do a terraform plan and show me the warning, show me that this thing is going to reboot somewhere when I run my automation. But it's like there's this huge disconnect and I haven't seen any services whose sole purpose is to look for maintenance, look for deprecations, look for things like this and then alert on that in a sensible way. Anyway, Google Wave email isn't the solution. [00:46:54] Speaker A: Email is never the solution. [00:46:57] Speaker D: Just trying to make sure we all feel really old by the end of this episode. [00:47:02] Speaker A: Yeah, thank you. I mean, we already felt that way. [00:47:04] Speaker B: About Wireshark, so it was ethereal before it was Wireshark. And that goes back even further. [00:47:10] Speaker A: Oh it does, yeah. Ethereum. That's right. Well, Amazon has announced major changes to AWS Security Token Service Global Endpoint, which I was like oh no, what are they doing to my precious Global Endpoint? And it's as bad like it's a big change, but it has really no impact to you. So you can all calm down as I talk about this. AWS launched STS in August 2011 with a single global endpoint, which was sts.amazonaws.com because naturally hosted in the US east region because that's where the tire fire is. So it makes sense. And to reduce dependencies on a single region, STS launched AWS STS Regional Endpoints in February of 2015, realizing that that was gonna be a scale problem. These Regional endpoints allow you to use STS in the same region as your workloads, improving performance and reliability as well as reducing load. Hopefully on US East Region. However, customers and third party tools continue to call the STS Goal Endpoint because it's so darn easy. And as a result these customers don't get the benefit of the Regional Endpoints. And to help improve resiliency and performance, Amazon is going to make changes to the STS Global Endpoint with no action required for you today. As I mentioned, all requests go to the Global Endpoint are processed in US East Region. Starting in a few weeks, the STS Global Endpoint will be automatically served in the same region as your AWS deployed workloads. With some caveats I'll get to in a minute. For example, if your app calls sts.amazonaws.com from the US west region, your call will be served locally via the US West Region STS service. This will apply for all regions that are enabled by default, except for opt in regions. Or if you're using STS outside aws, those will still be handled by US East. This reminded me that oh yeah, regions. The new ones like KSA and all the other ones we're talking about, you have to opt into those, which is a weird idiosyncrasy if you are not aware of. So if you haven't opted into those and you hit the STS endpoint, which I don't know how you do that if you're not opted into the region because I don't know how you do anything in that region. If you somehow pull that off, you'll get basically passed back to US East 1 and then if you're calling it from the web, you'll get us East 1 as well. Now you might be thinking to yourself, well that's going to cause all kinds of problems. And I that's what my thought was. What about cloudtrail? But other things and Amazon has actually thought this through. So CloudTrail logs for global STS endpoints will still be sent to the U.S. east region. So even if you're calling STS, if you're calling the SDS global region and it's being served by a different region, it'll still forward those logs to us east. CloudTrail logs will have additional metadata fields added, including endpoint type and AWS serving region to clarify which endpoint and region served the request, which is a nice improvement. Requests made to the STS Amazon AWS endpoint will have a value of US East 1 for the requested region condition key so it doesn't break all of your IAM policies, which I also worried about. Regardless of which region serve the request and requests handled by the STS endpoint will not share a request quota with a regional STS endpoint, which is a nice side benefit. So overall I like this improvement quite a bit. I think it's a good move and welcome Amazon. This is cloud native architecture, so I appreciate using local latency based routing for cloud. [00:50:04] Speaker C: I mean it's it that you know, the fact that they're having to ship all the logs back from each region back to US east one, you know, really just it's your architectural designs are important and, and you know, I know scaling at the, the scope that Amazon had to was not part of the vision when they rolled out services like this, but it's. I can, you know this happens to companies every day with smaller, smaller things like you end up doing weird baits because of architectural choices in the past. [00:50:38] Speaker A: I'm actually half thought they were telling me they were deprecating the global STS endpoint because that seems like the logical choice you would make and luckily they didn't do that so. But they're doing a ton of heavy lifting on the back end to make this seamless for you as a user, which I appreciate that as well. But I assume at some point like there has to be a realistic conversation about like do you continue to manage a global endpoint of this scale and size, especially considering how many comp. You know or like do you start putting, you know, do you start making it noisy when you're using it so that it adds more logs? Or like, hey, this region isn't ideal. You should be using the local endpoint because I think it's easy for plugins and for, you know, for developers just to use the SDS one because they don't have to think about the region then they don't have to do it. They don't have to do a metadata look up, they just do it. So I, I suspect that at some point it would make logical sense that you would retire this. But still it's a good step in the right direction. Although I imagine if they retire this, it breaks all US east one forever. [00:51:36] Speaker C: I mean it'll break so many things, right? Like not even just US east one. Think about all the global influence that are in those. You know, like, you know, like it's just. [00:51:46] Speaker A: No, I know it's. I mean Amazon will be broken for sure and then all of every customer will be broken and you know, all the, all the deprecated extensions and things that people are still using that aren't supported will not get updated. Like it, like reality. This might be the, this might be the legacy architecture pain that you live with forever. Sort of like IAM and Amazon as well. But you know, if they're ever to actually replace the IAM system with like IAM version 2 and STS version 2, I would not offer this in the future if I have the choice not to. [00:52:20] Speaker C: Oh, I mean, I don't know if. [00:52:21] Speaker A: They'Re ever gonna build IAM too. I thought it would happen already because there's so many limitations to what they did with IAM that I thought they come up with a new paradigm for it. But they've yet to do that and I don't think they're going to. I think it's too embedded at this point. [00:52:33] Speaker C: I think it's too hard of a problem to solve and I don't know how you would reach consensus on how to solve it. [00:52:39] Speaker A: All right, let's move to next story here. The S3 metadata service announced at re invent is now generally available. Very quick, general available after reinvent.s3 metadata provides automated and easily queryable metadata that updates in near real time, simplifying business analytics, real time inference applications and more. S3 metadata supports object metadata which includes system defined details like size and source of the object, and custom metadata which allows you to use tags to annotate your object information like product sku, transaction ID or content rating, where has where. [00:53:08] Speaker C: Was this service, you know, or in my life. I've needed this for a long time and done some crazy workarounds. I'm glad to see that they're rolling it out there because it is super useful and you know, from trying to analyze cost per customer transactions and all those things, like it's, it can get really crazy. [00:53:27] Speaker A: So yeah, I mean, ability to put a customer ID directly into the metadata, et cetera. [00:53:35] Speaker B: Yeah, I mean just, just simply the ability to then query the, the metadata service, which is a whole lot cheaper than, than running ahead operation on 50 million items, let me tell you. Yeah, yeah. If anyone's going to tell us, Ryan's the guy. [00:53:53] Speaker A: Yeah, Ryan's familiar with this problem. Moving on to gcp, they see your metadata service and they raise you with the BigQuery Metastore, which they're releasing in public. Preview of BigQuery Metastore, a fully managed unified metadata service that provides processing Engine interoperability with oil, enabling consistent data governance. BigQuery metastore is a highly scalable runtime metadata service that works with multiple engines. For example BigQuery, Apache, Spark, Hive and Flink, and supports the Apache Iceberg table format. I would just want a good salad at this point. This allows analytics engines to query one copy of the data with a single schema, whether the data is stored in BigQuery storage tables, BigQuery tables for Apache Iceberg or Big Lake external tables. This is something that Amazon would love to be able to do, but their architecture sort of limits their ability. So things like the metadata service for S3 make some of these things more possible potentially in the future. But it's already built into Google. The way they built their storage engine. Their idea is you can write the data once and use it in lots of places. So this is just continuation of that story in the big data space, which I appreciate. [00:54:57] Speaker C: The alternative is a blunt or transformation. Right. [00:55:00] Speaker A: So which is why Amazon keeps trying to make glue work because you had to transfer all this data around all the time, which is not great for sustainability reasons either. So it's very nice that this is native in gcp. Well, cloud deploy, the feature that I always forget exists on Google Cloud has several new features to remind me that it exists and that it might be pretty cool. Three new features include repair rollouts which let you retry failed deployments, or automatically rollback to a previously successful release when an error occurs. This can come in any phase of the deployment from either a SQL migration, a misconfiguration detected, or when talking to a GKE cluster as part of a deployment verification step. That's good A deploy policies limit which is an automation or user that users can do initially they're launching. Time window policy is the first deploy policy which can for example inhibit developments during evenings, weekends or during important events. Which I love that they're examples where evenings weekends are important events, not during business day. Like I see the way that well thinks while an on caller with policy overrider role could break class to get around the policies, automate deployments won't be able to trigger during the middle of a big demo. For example, Time promotions is the third and final cloud deploy feature which is after your release is sefully rolled out, you may want to automatically deploy to the next environment. Previously auto promote features let you promote a release after a specific duration, for example moving it into product 12 hours after went to staging. But often you want promotion to happen on a schedule not based on a delay. And they pointed out in the article Google deploys the staging. They force a staging deploy on Thursdays but they don't want to deploy to production until Monday. And so they basically needed something that does time promotion for them which is appreciated so unavailable to you in cloud deployment. Definitely something to check out if you are trying to fix CD problems at your company. [00:56:46] Speaker D: I miss a good code deploy cloud deploy tool. That's all I have to say here. [00:56:51] Speaker A: They don't have one and Azure doesn't Azure DevOps do it? [00:56:55] Speaker D: Yeah, you have to use Azure DevOps Pipeline or GitHub Actions. But Azure was our DevOps pipelines I feel is a lot still lagging in there. Like it works for some newer technologies but a lot of it's still oh, just drop down to a shell and run this thing. And I'm like that's not native. [00:57:16] Speaker C: Yeah. And I kind of tire of GitHub Actions for continuous delivery type stuff just because it's not a great solution to that. Like I know. [00:57:27] Speaker A: You mean It's Jenkins. [00:57:28] Speaker B: Part 2 GitHub Actions, the new Jenkins. I made a shirt for that. [00:57:32] Speaker D: Why would you bring up the J word? [00:57:33] Speaker A: Yeah, sorry. It just. You know how you bring up deployments in a wrong tool. I immediately go to Jenkins and that's. You know. [00:57:40] Speaker C: And it. It's because it's. That's the. It is. It's exactly what's happening is that GitHub Actions, you know one wants to do Jenkins anymore so that. But they're going to abuse the CI tool, you know, which is what GitHub Action should be used for you know, building and testing and managing that for deployments. You want something that can be longer lived, that can look at your traffic and can make decisions based off of what's going on over long periods of time. Jenkins can't do that because it'll overrun the Java memory heap and Actions is too limited to really do that and you don't really want to run an executable like that. So tools like this are great solutions to that problem and I think they're underutilized in general. [00:58:21] Speaker D: I agree with the underutilization like CodeDeploy. Everyone wants to use, you know, every other tool in the world. But CodeDeploy on AWS, it's just so simple and works so well. Whether it's in scale sets, containers, wherever it is, it just works and it makes life easier. Like, you know, and I understand that maybe it's just because it's cloud native and you know, AWS has done a lot of the integrations already for you, but I've helped large companies and small companies and it's no easier or harder for either of them, which is nice. [00:58:56] Speaker C: Timed for motions. It's beautiful feature to have baked into the project. [00:59:01] Speaker D: Yeah, I like that one. [00:59:04] Speaker B: Just a time undeploy would be nice. Like bring this thing, bring this demo environment up for three days while I'm on site and then take it down again automatically. [00:59:12] Speaker A: Cool, I would buy that. Well, Google also has a feature from the redundancy department. Google is announcing Vertex AI gen AI. Yep, there's a redundancy evaluation service is now in preview. This new feature empowers developers to rigorously assess and understand their AI agents. It includes a powerful set of evaluation metrics specifically designed for agents, built with different frameworks and provides native agent inference capabilities to streamline the evaluation process to pick the best vertex AI gen AI agent that you want to use. [00:59:42] Speaker C: Every time I think I'm getting a handle on AI and I'm starting to get there. Like they do something like this where I'm like, okay, now I really don't understand the agentic AI thing and I gotta go figure out a workflow that lets me understand why this feature is important. Because right now I'm like, I don't understand what this, what's this for? [01:00:02] Speaker A: Oh, when you have a bunch of models like Claude and others, you would maybe ask a question like, which one works best? And so this gives you an abstraction layer so you don't have to code to that specific model and then you can say, do this juntek AI thing and Then compare the results in the tool. [01:00:16] Speaker C: But we've talked about in the past is really difficult to do. [01:00:20] Speaker A: And I don't know, I don't know how it works. I just know that's what they're doing. [01:00:23] Speaker D: It's a one to many. [01:00:25] Speaker B: It's like unit testing for your agent really. It's just a framework that lets you test the shit out of it because you may have 50 test cases and they all look good, but it's customer comes along with number 51 and all of a sudden it gives them a recipe that includes household bleach in their soup. And you know, all of a sudden you're out of business. [01:00:47] Speaker D: Remind me not to go to Jonathan's house. [01:00:49] Speaker A: But does it. [01:00:49] Speaker C: It doesn't stop that response. Right. It just allows you to find it during development. Right, okay. [01:00:56] Speaker D: Hopefully, you know, if you run the same thing 10 times though, you're most likely going to 10 different answers. So, you know. [01:01:02] Speaker A: Yeah, I mean I've seen that. I've asked the same question twice to some of these AI, the answers are different. I mean they're same conceptually, but the phrasing is different, the certain words are different. [01:01:13] Speaker B: You can change that if you change the temperature. Temperature is like the distribution of the random distribution of the tokens that get picked from the top tokens. So if you turn that right down to like 0.1, then it's almost deterministic. [01:01:26] Speaker A: That's cool. [01:01:27] Speaker B: And if you turn it up much higher like, like up to two, then starts to get a little psychosis. [01:01:33] Speaker A: Well, that explains how you actually do this then. Now I understand it better. Thank you, Jonathan. [01:01:38] Speaker B: Yeah. [01:01:40] Speaker A: All right, and then our final Google story. You can now get smaller A3 high VMs. You can get the A3 high VM powered by Nvidia's H180GB GPU and multiple machine types, including a 1, 2, 4 and 8 GPU option now as well as full support for the spot market pricing and more deeper integration into Vertex for all your spot management needs for your training and inference requirements. So apparently the H180 gig GPU was not as popular because now they're offering it to you in multiple flavors instead of just eight as well as spot market. So good to see. [01:02:11] Speaker D: Or availability is better. [01:02:12] Speaker A: One of the two I like to go with, it's not as popular, but that's probably not the case. [01:02:19] Speaker B: That's cool. I mean they're expensive to rent and if they didn't have a smaller option. [01:02:22] Speaker A: Then yeah, if you don't need eight, I mean like it was kind of crazy. That was the size before. I mean that's why it's an A3 high VM I guess because you got high number of GPUs but. And a high price tag. But I appreciate that you may not need all eight and so you can now buy less and if you're not buying all eight then the other ones get thrown to the spot market for you for a discount. So it's a win. [01:02:45] Speaker B: Cool. [01:02:47] Speaker A: We don't have any Azure stories. They were kind of. They had one thing about Coldplay and I knew Ryan would get really mad if I did that story, so I decided not to do that. And it had nothing to do with cloud. It was really about how they're using AI to make their new Coldplay album better with interactive experiences for users. And I was like Ryan will hate this article. And I, I almost did it and I was there and I just said I can't now. It was Eagles. We definitely. [01:03:10] Speaker C: Oh no, no. That's how you get me to. I'm not, I'm not showing up to that recording. [01:03:15] Speaker A: Yeah, well see I wouldn't, I wouldn't put Eagles. I would have dropped it in in mid record. You're actually record on. Yeah, that's how I know how to work this system. And so there are no AG stories. I have a new off topic topic. So cool. I created another new segment this week because I was in the new segment mood called off topic because there's always things I want to talk about that we don't ever get to talk about. And so there we talk about it in the after show. I guess it's after show but it's before the after show. So there you go. Details. But basically Chrome OS Flex, for those of you who have old laptops or computers hanging around, you can now deploy a no cost, easy deploy solution to breathe new life into that hardware for your children perhaps, or for your spouse who is not very technically literate. Chrome OS Flex is a USB stick. You can install Chrome OS Flex and breathe right into that hardware, transferring your aging laptops, kiosks and more into fast, secure and modern devices. Google says it's the perfect solution for businesses hoping to refresh devices, improve security and embrace sustainability. And going into 2025, they certified over 600 devices will be supported natively. Although it sounds like pretty much works everything as long as it can handle some standard drivers. And so that is available to you if you have some old hardware laying around. I have old Mac laptop that I've wanted to play with Chrome OS for a while and I'm going to plug in a usb, stick to it and play with it. Yeah, what else am I going to do with that hardware? [01:04:39] Speaker C: I'm a giant hoarder. So I have like laptops that go back 10 years, maybe longer. Actually this laptop that I'm recording on is at least 10 years, so I think no, I've got much older. And yeah, you start to boot them up and you realize that, you know, the, the software is so out of date that you know, the certificates have rolled and, and that kind of thing. And it's just really risky to connect the thing to a network like, you know, it's going to be full of exploits, like known, tried and true exploits. And so I love being able to do this and be able to make them usable for different things, whether it be small little projects or, you know, what have you. So I'm, I think it's pretty cool. [01:05:23] Speaker D: There was some system I booted up like a couple years ago and it was just like you were saying, I couldn't get online and I finally figured out the root certs expired and it had been updated and you couldn't update it because I couldn't get online to validate nasl. And it was like this, how do you get the certificate on the box and in order to get it? And it probably was one of the couple really old laptops that I used to have. And I was like, one of these days I'm going go through and clean up and make sure I have all my stuff from here. [01:05:51] Speaker C: And as much as I would love to force, you know, Linux command line, all my children, I don't want to support that reality. [01:06:02] Speaker B: I like the idea what they're doing. I think if it saves a bunch of stuff going in landfill or something and bring some new life into things for a few more years, that's great. Especially as Windows 11 is only supporting newer CPUs and TPMV2 and things like that. It's super annoying that an OS vendor would do that. [01:06:25] Speaker D: Well, it will help a lot of companies with their green initiatives like you're saying with, especially in the EU where there's a lot more standards around that. [01:06:34] Speaker B: I will say though, I don't like Chrome OS that much. I feel like it will. I give it to a parent. Absolutely. Because it just never breaks, it fixes itself, it updates itself, it does all the things. It's wonderful. [01:06:49] Speaker C: That's why you don't like it. [01:06:50] Speaker B: But yeah, that's why I don't like, it, it's not for me. No, it's not, no. But at the same time, they're really prevalent in schools. Chromebooks in schools now, just all over the place. And I think we're missing a huge opportunity and educating kids in how technology works and teach them to be, you know, to understand it rather than to be consumers of Google services. And I, that's, that's when my beef is ready. So I, I would suggest Ubuntu Live USB sticks work on almost every single thing on that same list of 600 devices. [01:07:22] Speaker C: No, I mean the challenge, really, even before you get to the kids, right. You've got to trade all the teachers to support it. And like, there's just so much subject matter expertise that it would take to, you know, where to just get to the environment, you know, where you can learn and you know, before you're learning it. So I don't know, like, I think that's why it's really taken off and, you know, they built a lot of, you know, central management capabilities into Chrome OS so that it's really easy to sort of gate things and put a lot of rules organizationally. And so, yeah, I completely agree. Like, I hate it. I really hate using Chrome OS as a user, but that's because I know how these things work. My kids have zero issue using Chrome os. They don't, they don't struggle with it, they don't mind it, they don't think it's too restrictive. [01:08:16] Speaker A: Well, I mean, they're doing all, Everything they do is in Google Apps. [01:08:18] Speaker C: Exactly. [01:08:19] Speaker A: So they don't care. Yeah, no, I mean, like, well, it's all web browser stuff and YouTube's web browser, all that stuff. Actually, it was funny because at the elementary and middle schools in our district where we live, they're one school district. Then the high school is a different school district, which makes really no sense. I don't really understand how this has happened. But the middle school and elementaries, they use Google, Google Apps, Google workspaces, all that. The high school district uses Office GC5. And so like, oh, no, my son's like, transition from middle school to high school. He was like, what is this? Like, why is it so bad? [01:08:53] Speaker C: Oh, that's wrong. [01:08:54] Speaker D: Welcome to the real world. Hey, what a job is. [01:08:58] Speaker A: Welcome to the next 50 years of your life. [01:09:01] Speaker C: Exactly. Outlook. Yeah. All those G Suite productivity tools. Yeah, you can't use those now. [01:09:07] Speaker A: Yeah. But yeah, I do miss Google tools quite often when I'm stuck using teams. Especially I miss Slack. So that one, that one still hurts every time. All right, gentlemen, it has been another fantastic week. Hopefully, Azure wakes up and we'll have stories for them next week. But we'll see what happens. Come on, Azure. Deliver for us. And we'll see you all next week here on the Cloud. [01:09:32] Speaker C: See you later, buddy. [01:09:34] Speaker D: Hey. [01:09:38] Speaker B: And that's all for this week in Cloud. We'd like to thank our sponsor, Archera. Be sure to click the link in our show notes to learn more about their services. While you're at it, head over to our [email protected] where you can subscribe to our newsletter, join our Slack community, send us your feedback, and ask any questions you might have. Thanks for listening, and we'll catch you on the next episode.