308: SCC: Security Command Center or Super Cool Capabilities?

[00:00:00] Speaker A: Foreign. [00:00:06] Speaker B: Welcome to the cloud pod where the forecast is always cloudy. We talk weekly about all things aws, GCP and Azure. [00:00:14] Speaker C: We are your hosts, Justin, Jonathan, Ryan and Matthew. [00:00:18] Speaker A: Episode 308 recorded for June 10, 2025 SEC Security Command center or super cool capabilities? You tell me, Ryan. [00:00:27] Speaker C: Oh, yeah, super cool for sure. [00:00:32] Speaker A: Okay. [00:00:35] Speaker C: I mean you're, you're, you've always questioned my ability to determine what is cool. So this is no different. [00:00:41] Speaker A: I mean, I, again, it's a security tool, so how cool can it be? Like, you know, I, it's one of the things I learned as I started forcing my way into demos of some security tools. Like, you know, people are like, oh, CrowdStrike's amazing. It does all those amazing magical things. And I'm like, cool. Like, I'd love to see what it actually does. And then like, I see it and I'm just like, I'm not as impressed as you were. [00:01:01] Speaker C: I'm underwhelmed. [00:01:02] Speaker A: Like, yes, that's neat. But how do I use that on a day to day basis? [00:01:08] Speaker C: Yeah, a lot of times you have to see the alternative. You know, it's like, what do we do before Crowdstrike? [00:01:12] Speaker A: Oh, we, we did everything. No, I, I, I managed antivirus before in my prior life, so mantech. And then when I created the edict that I'll never buy software for fuzzy creatures, I ran Panda antivirus back in 1998. [00:01:29] Speaker C: Wow. [00:01:30] Speaker A: So, yeah, that's when I created my rule steadfast, that I do not buy any software named after a fuzzy creature. And it's been a good rule. I've yet to violate it. And these companies come up all the time and they're named after a fuzzy creature. And I just look at them like, I can't support your product. And they're like, why? And I'm like, because you're a fuzzy creature. And then they think I'm ridiculous. But it's worked out. [00:01:51] Speaker C: But it's Panda. Yeah, yeah. [00:01:53] Speaker A: Those companies get bought or die or have terrible security breaches. I just don't know what it is. [00:01:57] Speaker D: But name a few of them because my brain is not processing fast enough to think of fuzzy feet. [00:02:04] Speaker A: Oh. I mean, well, there's Panda Antivirus. That was one. [00:02:07] Speaker D: Yeah, I know that one. [00:02:09] Speaker A: Cheetah Software is the one that's a fuzzy creature. I work there though, so that was a little weird, but I never bought it. There's others too, wasn't there? There's a flavor of Linux that's there. [00:02:25] Speaker C: I mean, I don't want to bash Any potential sponsor. But yeah, I'm not bashing. [00:02:29] Speaker A: I mean, I think I've said this before on the podcast. Those sponsors aren't coming anyway. [00:02:34] Speaker D: I'm just saying. According to Gemini, cheaters are not fuzzy. They just have relatively short hair, coarse fur coats. [00:02:41] Speaker C: Yeah, what does AI know? Do they know how aggressive they are going to be at sales? Because that's an important. [00:02:52] Speaker A: See here, I have a quick little. Let's see, we've got 66 rows of options here of names. That's a lot. There's a couple either. Like, I normally have a couple that I'm just like, yeah, that one. Oh, well, Mulesoft. That one's a fuzzy creature. And that one I didn't buy and then it got bought by Salesforce, where I went to die. So that one's a pass. Let's see what else I got. There's a couple others. I'm pretty sure I have to research this because Hipmunk was. I did buy Hitmonk, which technically was a play on Chipmunk, so. Oh, Red Panda. I never bought Red Panda is a Kafka tool. About that one, I guess I have technically been a customer of mailchimp. [00:03:33] Speaker C: I don't know if you count that. [00:03:35] Speaker A: One, but sure, I violated the rule once. [00:03:37] Speaker D: Sorry, sorry, I didn't mean to completely digress the first three minutes of the podcast, but now that we've achieved that. [00:03:44] Speaker A: Yeah, yeah, that's good. I did also say Seuss Linux is okay because Asus isn't an animal, but it's also a lizard, and that's fair per game. [00:03:56] Speaker D: Not fuzzy. [00:03:57] Speaker C: Not fuzzy. [00:03:57] Speaker A: Not fuzzy. Scaly, but not fuzzy. [00:04:00] Speaker D: Scaly. [00:04:00] Speaker A: All right, let's get into it. I was at FinOps X last week. We recorded the show from the hotel that I was staying at, which was the Marriott Marquis, Beautiful hotel in San Diego. I'm not supposed to write them either, but. But the conference was there this year, bigger than ever before. I mean, it felt bigger. At least they had moved the sponsors into a actual room now downstairs, which was nice, versus having them in the hallway where they would grab you rudely as you ran by trying to get to your session and trying to sell you on things. So now they're locked away where they belong in a contained space. That was good. Overall, it was a good conference. I will rant about finops tooling in the Cloud Journey section at the end of the show. Uh, so if you want to hear my controversial take on LinkedIn that I got a bunch of feedback on and that. And I have a blog post that someone else wrote that I also keyed off of. That was good too. But I definitely have some thoughts about finops tooling, but from the major cloud providers. You know this is one of the few places you'll see all the cloud providers on stage together in a roundtable as well as all of them get ample time to talk to you about their finops tooling. So that means that they typically either rehash things from Google Next Google or they actually announce finops things, which typically Azure and AWS have done in the past. And so it was always good to see kind of what they're up to. But first up, Focus is now supported by all the major cloud providers. The 1.0 spec they have now officially released the 1.2 spec AWS is generally available and Google Cloud launched a BigQuery export and private preview, all signaling industry wide standardization of cloud cost reporting formats. And Azure last year announced support for this. So now they all at least support the 1.0 standard. They are releasing a new version of the focus standard every six months. So the latest and greatest version is 1.2. If you're thinking it's a really major change, it's not. It's just adding new fields that have things like the 1.2 version I saw has some currency additions to put your cloud costs in different currencies. AWS introduced an AI powered cost optimization through Amazon Q Developer integration with Cost Optimization Hub, enabling automated recommendations across millions of your resources with detailed explanations and action plans for cost reductions. Azure launched AI agents for application modernization that can reduce migration efforts from months to hours by automating code assessments and remediation across thousands of files while introducing flexible PTU reservations that work across multiple AI models. I mean, if I'm modernizing my application, typically it's off net and Azure, but okay, appreciate the tool. And then Google Cloud unveiled FinOps Hub 2.0 with Gemini powered waste detection that identifies underutilized resources like VMS at 5% usage and provides AI generated optimization recognitions for Kubernetes, Cloud Run and Cloud SQL Services. And Oracle Cloud was there as well, adding carbon emission reporting with hourly power based calculations and GHGP compliance, plus new cost anomaly detection and rules based cost allocation features for improved financial governance. The most hilarious part of the whole conference for me was I did not go to either of the keynotes until the end of the keynote day when Oracle was on stage and I got there just in time to see them throw up the first Safe harbor slide and the entire audience cringe because Oracle can't talk about anything without giving a safe harbor statement because Larry Ellison likes to lie on stage. So you had to have that to not get sued in general. [00:07:17] Speaker C: That is funny. [00:07:19] Speaker A: A couple other things of note for those of you who've been in the cloud space as long as I have, which is maybe just me and Matt, maybe Ryan. [00:07:27] Speaker C: Not as long as you guys. [00:07:28] Speaker A: Yeah, well, not as long as us. But you were around when Cloud Health was kind of a big deal. [00:07:32] Speaker D: Yes, it was. Yeah. [00:07:33] Speaker A: So Cloud Health was kind of the. [00:07:34] Speaker D: I was certified on them at one point even too. [00:07:37] Speaker A: Oh, were you really? [00:07:38] Speaker D: Yeah, we needed one for work. It was special, special test. [00:07:42] Speaker A: So Cloud Health was the granddaddy of all like major finops tooling. And then they got bought by VMware right about the time Cloudability came onto Steam onto the scene. And I think Cloudability kind of ran away with their market as VMware didn't know what to do with Cloud health because no one know why the VMware company was buying a cloud tool. And then of course now VMware has been bought by Broadcom and so it's interesting to me that Broadcom actually was there with a cloud health booth that no one was visiting. Ironically, at least that I saw when I was in the show floor. They've redesigned Cloud Health with of course an AI powered features including intelligent assist for natural language queries and smart summary for explaining billing changes, making the platform's most significant Update since its 2012 launch, which I think is literally accurate. [00:08:23] Speaker C: Wow. [00:08:24] Speaker A: The update addresses a key finops challenge of making cloud cost data accessible to non technical teams through plain English interfaces Instead of requiring SQL knowledge as 44% of FinOps teams were created within the past year According to the FinOps foundations and CloudHealth claimed they're processing 10 petabytes of data for roughly 22,000 customers. I don't know who those customers are because I haven't ran to a CloudHealth shop in quite a while, but apparently they're now excited with their new AI features, which is great. So I'm glad to see Cloud Health getting some love. I thought it was just going to die inside of the Broadcom behemoth or get sold off or something, but apparently not. It's still kicking. [00:09:00] Speaker C: Yeah, heard of them in a while, which is kind of neat. I mean, maybe it's worth if they're making improvements to us, maybe it's worth looking again. [00:09:09] Speaker D: I'm just more confused by the fact that 44% of FinOps teams were created last year. And while I guess I can see that, it's one of those things that I'm like, ooh, I've been in this industry too long. Got it. [00:09:21] Speaker C: Yeah. Well, I don't think people really understand how powerful having a finops practitioner dedicated to saving you money can be. [00:09:29] Speaker A: Right. [00:09:29] Speaker C: Because a lot of companies are very quick to put that on whoever owns the cloud infrastructure or cloud owners service for dev teams if they're owned by engineering leaders. So just cut it by 20%. But there's so many intricacies and so many different things. You need to know that it does really require specialization, in my opinion. So it's. I get it. [00:09:52] Speaker A: I think a, until five years ago, we didn't really have the foundation. So the finance foundation is basically five years old at this point. So that's cool. It'll be five years old in August, it's almost here, which is cool. And it's been a great foundation. I think it's really helped build the standards around FinOps and it's really helped drive a lot of the discussion in the industry. But then the other part of it that I think really factors into why there are so many finops teams being created now is the upcoming pending recession that may or may not happen any day of the week at any given time, depending on who you're talking to. Everyone's very cost conscious right now. And so I think, you know, if you were living in a zero interest rate world where you didn't care about your cost model so much, now you do. And so I think this is driving a lot more finops conversations and practices. And then when you go to research, you find the FinOps foundation and then you create a FinOps team. So that's the recommendation from the book. [00:10:44] Speaker C: And it's a pathway, right? The foundation is laid out how you do it, right? How do you start a finops practice within your business does really help out. [00:10:52] Speaker D: So it's kind of like a COE from 10 years ago when AWS pushed that COE. Hey, this is the way you do it. This is the way you build it out, you know, and kind of structured it that way. I mean, it's definitely something you need. Speaking of people seeing my friends in the industry also that are not finops people, but people that are run a cloud or run a SaaS product like that, that are straddled with that dual role of the finops responsibility for the product, it's definitely a different mindset that you need to take on and it's a balancing Act a lot of times between, you know, between keeping the product up and running and moving and everything else and keeping the cost down. So there's a, there's a very fine line in there that at least I do. And I. And from talking to my friends in the industry, I've not had a dedicated FinOps team ever. And I would love to see kind of that different mindset when you look at it. But, you know, there's definitely a lot of things that, you know, I've taught, run by finance team, other finance. You know, I've talked to my friends where it's like, okay, savings plans or this thing, we're only using it, we're using a five, but we don't need it for two days a week. Is it so cost effective? And that's why I always assume a truly dedicated finance person would be able to really get that math behind those equations. [00:12:12] Speaker A: Ryan's had nothing free on that. [00:12:14] Speaker C: See, I already said my piece. [00:12:16] Speaker A: Yeah, yeah, he's got nothing. He's got. No, not those thoughts. [00:12:19] Speaker C: Calculations. [00:12:21] Speaker A: Leaving it out there. He didn't pick it up. And I was like, oh, that was lock. We're pause. [00:12:24] Speaker D: So it's security. It's a security person. It's fine. He just adds, he'll just buy a security tool. With the extra money that I spend, it's fine. [00:12:33] Speaker A: Exactly. That's exactly what security does. I say money and security spent it. Cool. Well, the other team that's spending all my money is the AI and big data teams. And so they all congregated in San Francisco. So I ran from San Francisco to San Diego, where I was hoping for better weather, which did not turn out to enjoy time with my FinOps friends and everyone else came to Snowflake Summit in San Francisco and they dropped a ton of stuff. So first of all, I'm not going to go through every one of these articles because A Ryan and Matt would both hang up on me. And there's also a lot of them. So I'm going to try and give you the highlights here. And there's one acquisition we'll save for the end. But basically they introduced Snowflake Intelligence and Cortex agents to enable natural language querying of structured and unstructured data, allowing your users to ask questions in plain English and receive governed answers, which makes sense. Cortex AI SQL brings AI capabilities directly into SQL syntax, enabling analysts to extract metadata, classify sentiment and process documents, images and other formats with 30 to 70% performance improvement over your traditional pipeline. They also include AI observability tools for monitoring generative AI applications and access to models such as OpenAI, Anthropic Meta and Mistral within Snowflake Security Perimeter and provision throughput for dedicated inference capabilities. New ML capabilities, including a data science agent that uses anthropic models. The new Snowflake Snow Convert AI now supports automated migrations from Green Plum, Nateza, Postgres, BigQuery, Sybase and Microsoft synapse with AI powered code verification and data validation to reduce migration complexity and timelines. [00:14:06] Speaker D: What's Greenplum? [00:14:08] Speaker A: It's a ETL tool. [00:14:09] Speaker D: Oh, okay. [00:14:11] Speaker A: Yeah, sorry. [00:14:11] Speaker D: See, I get to learn new things too. [00:14:13] Speaker A: Yeah. Natasa is also sort of a similar type tool. Standard Warehouse Generation 2 delivers 2.1x faster performance for core analytics workloads and 4.4x faster delete, update and merge operations, while new Adaptive Compute automatically selects optimal cluster sizes and routing without manual configurations. Thank you goodness yammer, because scaling your Snowflake up and down is a trick. Iceberg performance improvements include a 2.4x faster analytics on externally managed tables through search optimization, query acceleration and automatic compaction, and enhanced pruning capabilities for selective queries. The Snowflake has also introduced Adaptive Compute, which I just mentioned. [00:14:50] Speaker C: Sorry. [00:14:51] Speaker A: The platform adds comprehensive finops capabilities including cost based anomaly detection, tag based budgets, and joins the FinOps foundation as a premier enterprise member. And the tools help organizations track spending spikes, set resource limits by tags, and align with industry best practices for cloud cost management. New security features include anomaly detection using AI models, leaked password protection and disabled compromised credentials found on the Dark web, and bad IP blocking. And then finally Snowflakes. Converting existing warehouses to adaptive warehouses requires only a simple alter command with no downtime. So you get Adaptive Warehouses and then Snowflake Intelligent introducing again that natural language. Lots of repetitiveness in their articles as well, and I think that's pretty much it. I think. I think I covered all of it. Okay, that was a lot. [00:15:37] Speaker C: That was a lot. Yeah. No, I was just talking about Snowflake and how it's become sort of its own platform of the sorts. You know, like we've, we've moved into like running infrastructure, not being sort of the first principle of a lot of businesses. And now it seems like sort of hosting data in databases and large data warehouses is sort of go in that route too, which I think makes sense. I still sort of struggle to figure out where Snowflake would fit in to a lot of things, but I think that that's because of my specific workloads. But. [00:16:09] Speaker A: Well, I mean I could argue that Snowflake in some ways is kind of becoming aggregation layer of all hypervisors, right? Cause they're on aws, they're on Azure, they're on gcp. They provide you all the data services you need already, so all of your data and AI needs can be served by them. They have a serverless type technology as well, so you can run serverless functions there. And we'll just talk about this acquisition here in a second. But they have database technology coming on too. So at some point if you could just get some basic general servers that could run web and APPT here, they could completely compete with the cloud providers that they're running on top of. So I mean you could say, look, I'm not going to use GCP or Azure AWS native, I'm going to use Snowflake and I can put your data in any place you want it in any region and you this full abstraction layer that could be really powerful. And so I don't know what they're thinking they're thinking about that way, but that's how I think about it. I'm like, it's kind of interesting, I. [00:17:05] Speaker C: Mean the fact that it makes sense from a monetary value. Snowflake is expensive, but it's also like it's not too expensive where the value isn't seen by a lot of companies, which is proven by their success and their growth in the last few years. And so it's, it's nuts to me that they can pull that off on top of these hyperpectures. But it's great, I mean, good for them. [00:17:29] Speaker A: And they did kick off the conference with announcing announcement of an acquisition. They have apparently purchased Crunchy Data to create a Snowflake postgres service bringing enterprise grade security compliance and operational standards to postgres within the Snowflake platform. This addresses the gap between developer preference for postgres and enterprise requirements for production workloads, acquisition targets, organizations that need advanced security features like customer managed encryption keys and compliance certifications for regulated industries. Crunchy Data brings proven expertise in enterprise postgres deployments across cloud kubernetes and on premise environments. Snowflake postgres will enable developers to run existing postgres apps on Snowflake without code rewrites while gaining access to built in connection pooling performance metrics and logging support. And this consolidates transactional and analytical workloads into a single platform. And the offering complements Snowflake's existing unistore solution by providing native postgres compatibility for transactional applications and early Customers like Blue Yonder and Landing AI see opportunities to simplify their app stacks and accelerate AI development. This move positions Snowflake to capture more enterprise workloads by eliminating the need for separate database management while maintaining full postgres compatibility. So curious to see how that pans out the next year, how that integrates. But the idea of being able to run your OLTP workload and your analytical workload basically in the same place, and I imagine, you know, with having postgres in the backend, they'll be able to feed into the rest of the Snowflake cloud much faster. Less ETL work, less connector work you have to do between these different systems. And so if you can feed your analytical system much faster, directly from the OLTP natively integrated, like, that's a huge benefit. And I could see this becoming MySQL as well as other database technologies in the future, if they can make this one to work. [00:19:05] Speaker C: Absolutely. [00:19:05] Speaker A: Like, kind of. [00:19:06] Speaker C: Because that, you know, I had the same idea as you were reading this out, like, if the data set is presented as a single data source that I can run analytical and transactional workloads against. Right. Like, that would be amazing value to develop on and to simplify the application architecture. So that would be super cool. [00:19:31] Speaker A: Brian or Matt just wishes he had postgres use a SQL shot. [00:19:38] Speaker D: Don't wear my dirty laundry. [00:19:40] Speaker A: Hey, I have a SQL and Mongo and postgres, I got it all. [00:19:45] Speaker D: I've dealt with all of them in the past. So they all have their own sharp edges. That's what you have to know. It's a matter of finding which sharp edges you're willing to deal with right now. There's some sharp edges I don't want anymore in life. [00:19:58] Speaker A: Postgres is one of Those. Or a SQL server. [00:20:00] Speaker D: No comment. [00:20:03] Speaker A: So you're on the NoSQL camp. Got it. Okay. Well, an interesting announcement came out this morning. Thank you for dropping that press release before we cut off the show for the day. OpenAI is adding Google Cloud infrastructure to its compute resources, despite being direct competitors and AI marking a shift from its exclusive reliance on Microsoft Azure for Data center initiative infrastructure since January 2020. 2025. The deal centers on Google's Tensor Processing Unit, or the tpu, which were historically reserved for internal use, but are now being offered to external customers, including Apple Anthropic and safe superintelligence. OpenAI's compute demands are driven by both training large language models and running inference at scale. And with the company reporting $10 billion annualized revenue as of June 2025. The partnership adds to OpenAI's infrastructure diversification strategy, including the 500 billion Stargate project with SoftBank and Oracle, plus billions in compute contracts with core weave for cloud providers. The deal demonstrates how AI workloads are reshaping competitive dynamics with Google Cloud generating 43 billion in 2024 revenue and positions itself as a neutral compute provider despite competing directly with other customers through their DeepMind AI programs. So yeah, that's Microsoft divorce is getting real. This one. [00:21:13] Speaker C: That's a lot of money. There's definitely been a move on both sides to, to separate. This feels a lot more than that, right? Like this is. That's a lot of money. And on constrained resources like TPUs and access to TPUs is problem industry wide and globally. [00:21:32] Speaker D: So I mean it also is probably the first true multicloud workload that there's out there. They can train across multiple clouds and if they do it right, they can, you know, in theory actually leverage spot markets and things like that. Which would be interesting to see how they destroy spot markets real fast when they start training everything. [00:21:52] Speaker C: I don't know. This seems like private, private deal like the TPUs on offer here are very much not. Not available to the public. [00:22:01] Speaker A: I can very much tell you that a spot market on Google would be laughable based on the amount of capacity planning I have to do. [00:22:09] Speaker D: Yeah, I get the question all the time on security questionnaires and like on ISO, I think it was ISO or SOC asking me about capacity planning. I'm like, I'm in the cloud, I just tell it I need more. There's definitely things I appreciate about that and there is definitely a little bit capacity planning I have to do in smaller regions with Azure. But the larger reasons are just like, great, let's change CPU or change that. [00:22:39] Speaker A: I don't have a problem with unique instances and I don't have a problem with GPUs because I understand the national international storage of GPUs that exists because AI. But I do get really annoyed when general compute type instances are not available like that. That's where I draw the line. I'm like, you can tell me, you know, my big massive six terabyte memory boxes aren't available. You can tell me that I can't get GPUs, but I want a basic, you know, N2 instance or N3 or a C class or an E class on. On the cloud. I'm getting cranky real quick. [00:23:13] Speaker C: Yeah. [00:23:13] Speaker A: And unfortunately that is part of my reality in the cool world although there. [00:23:19] Speaker C: Is a lot of customization in your day job that I think is well. [00:23:22] Speaker A: Yeah, we do have some of that too, but I mean it's extracted away that I don't have to deal with it other than I hear my team complain about it as well. But still, it's not very cloudy in my opinion, which bugs me. And every time I say that my Google reps all cringe Mistral AI has released Magistral, their first reasoning model, available in two versions, a Magistraw Small 24 billion parameters open source under Apache 2.0 and Magistral Medium the enterprise version with medium version scoring 73.6% on the AMY 2024 benchmark and 90% with majority voting. The model introduces transparent traceable reasoning chains that work natively across multiple languages including English, French, Spanish, German, Italian, Arabic, Russian and simplified Chinese, making it suitable for global enterprise deployments requiring auditable AI decisions. Magical medium achieves 10x faster token throughput than competitors through Flash Answers and Lechat, enabling real time reasoning for cloud based applications and regulated industries, software development and data engineering workflows. Enterprise availability includes deployment options on SageMaker with upcoming support for IBM, Watson x Azure AI and Google Cloud Marketplace, positioning it as a multi cloud solution for businesses marketing needing domain specific reasoning capabilities. The open source Magistraw Small enables developers to build custom reasoning applications with the community already creating specialized models like Ether Zero or Chemistry and Deep Hermes 3 expanding the ecosystem for thinking language models. [00:24:46] Speaker D: The multiple languages day one and the quantity of languages always impressed me. Like those languages, you know, it's not like all, you know, Latin based languages but getting like Russian and Chinese in their day one in Arabic they're different alphabets, different completely different speech patterns and having anything handled that in general in my brain is just ridiculously hard. Plus then you know, doing it day one in all of them at once is always impressive to me. [00:25:16] Speaker C: I wonder if that comes from the fact that they're they're not a US based company, you know, they're a French company. So I wonder if it's something that's already like a, like a business initiative for them. It'll be kind of cool. I don't know. [00:25:30] Speaker A: I mean I do think it's helpful in the fact that they are foreign and they do not speak English as a primary language and it's a way for them to differentiate between other language models because they do have the support at this level. So I, I think it gives them a niche that others aren't playing in quite as aggressively you know, other than potentially deep seek, who's probably going to be pretty strong at Chinese, you know, based on their source created as well. But I, I do appreciate that that's kind of their big focus is, you know, languages. And so I think if I were to be building, you know, an app that needed to do localization, I think I'd probably be looking at Mistral to potentially be my part of my localization engine or QA process or something to help do those things. All right, let's move on to AWS. AWS is launching its 37th global region in Taipei. This is AP East 2 with three availability zones, making the 15th region in Asia Pacific and bringing the total to 117 availability zones worldwide. This addresses data residency requirements for Taiwan's regulated industries including finance and healthcare. The region builds on AWS's decade long presence in Taiwan which included two Cloudfront Edge locations, three Direct Connect locations, AWS Outpost Support and a local zone in Taipei for single digit millisecond latency applications. Major Taiwan enterprises are leveraging AWS including Cathay Financial holdings for compliance focused cloud environments, Gamania Group's Vian AI platform for celebrity digital identities and Chunghua Telecom using Amazon bedrock for generative AI applications. AWS has trained over 200,000 people in Taiwan through AWS Academy, AWS Educate and AWS Skill builder programs supporting the local E system that includes four AWS heroes and 17 community builders. The region supports Taiwan's 2050 net zero emissions global with customers like ACE Energy achieving 65% steam consumption reduction and Taiwan Power Company implementing smart grid technologies, the drones and robotics for infrastructure management. [00:27:19] Speaker D: I remember when there were seven regions and there's like 37. This whole podcast is gonna be me. Well, yeah, I, I, I never realized how long I've been on the cloud till you said like Ryan, like I've probably been on the cloud longer than Ryan and I'm like oh God, you have, you know, pretty sure thinking through it, I'm like oh God, like you know, like local zones. I remember getting released, you know, outpost. I was like, ooh, you know, now I'm like oh God, 37 regions. Yeah, that's a lot of regions. And you know, originally the idea with regions was like there was only going to be like the 7 or 8 and you know, obviously over time data res, data residency and all the other things have come up, but still an impressive number of regions. Multiply that by at least three or four on average, I'd say it's a lot of data centers. They're running. [00:28:11] Speaker C: Oh for sure. Like and you know, 15th in the Asia Pacific region, you know, sort of. Yeah, part of the globe. You know, it just speaks to the density that, you know, of the market there and how much potential there is for compute and the businesses that are hosted in that area. [00:28:29] Speaker A: So yeah, I mean, I'm sure it helps too that TSMC is then Taiwan and they have a pretty big partnership with TSMC and yeah, they build all those graviton chips for them. And I'm sure, I'm sure there's reasons commercially that makes sense. [00:28:43] Speaker C: That does seem like a pretty good reason. Yeah. [00:28:46] Speaker A: I mean it is also a risk though, if the Chinese attack Taiwan, which has been a threat for a while, that this become the next Chinese region. If you're not careful, there's a little. [00:28:58] Speaker D: Bomb at the end bottom of every rack that they just push about and blow it up. It's fine. Don't worry about those details. [00:29:03] Speaker A: Make sure your Dr. Strategy is. [00:29:07] Speaker D: I wonder if they convert the local zone into a zone, like into that becomes a zone or on the back end the infrastructure person is like, do they just take a zone and then that stays a local zone and a re and availability zone, or do they build it out from scratch? How would they kind of do that? [00:29:24] Speaker A: I think even when local zones first came out, we talked about that. Is this just a. A way for them to kind of test the waters and regions they're not sure about yet and then at a certain point they grow out of it? But I assume all the, you know, the. Typically the local zones are tied to a global region somewhere else. So unless you swung the region to the local zone to the new Taiwan zone, I assume it's still attached to the original parent or region. [00:29:50] Speaker D: Well, that's just where the control layer. Control plane is. [00:29:53] Speaker A: Right. But I mean, if you're. If that's. I guess it could still be different. Compute in the same data center that's handling the local zone needs versus the region needs. Needs with isolation. [00:30:02] Speaker C: Yeah, but so many things about the AWS like infrastructure and how you configure that would be tied to that. Like it'd be too. I don't know, like that'd be very difficult to move a workload from a local zone tied to a different region to the same region, even if it's the same compute in the end, I. [00:30:18] Speaker D: Wonder if they just slowly. The plan is to deprecate the local zone to get the region up because there's core services when you launch a region that are always available in each region and then there's the secondary services they add on. So like IAM is obviously an easy one. IAM will work in every region that they launch, you know, EC2, EPS, S3, et cetera. Like those are core and then they, you know, that's why they always add the other stuff later on because they're not the core AWS services like at the heart that have to launch when a region launches. But so I wonder if like the complaint is. Or people just naturally shift over and they just don't expand the local zone anymore. [00:30:59] Speaker C: I bet that I would guess that the latter but just because it seems like an easier migration path because it's net new and working with customers and using incentive like incentives like voluntary discounts in order to get them to migrate to workloads seems easier to me than trying to get everyone to change their conditional IAM policies. That mentioned region like all the different things that you can tie into a region. [00:31:23] Speaker D: Yeah. [00:31:25] Speaker A: All right, the next one I'm hoping maybe you guys know more about than I do because AWS is now publishing Smithy API models daily to Maven Central and GitHub, providing developers with definitive, up to date sources of AWS service interface definitions and behaviors that have been used internally since 2018 to generate AWS SDKs and CLI tools. Developers can use these models to generate custom SDKs for unsupported languages, build server stubs for testing, create developer tools like IAM policy generators, or even generate MCP server configurations for AI agents. The repository structure organizes models by service, SDK ID and version, with each model containing detailed API contracts including operations protocols, authentication methods, requests, response types, and comprehensive documentation with examples, this release enables developers to build purpose built integrations without waiting for official SDK support. Particularly valuable for niche programming languages or specialized use cases where existing SDKs don't meet your specific requirement. The models are available at no cost through the GitHub repository and Maven Central with Smithy CLI and build tools providing immediate access to code generation capabilities. [00:32:28] Speaker C: I mean, I guess I know a little, but I mean that's, it's a. It's one of those things that has been a differentiator between Google and AWS for a long time, which is like Google automatically generates all their their SDK based off of API documentation and so all of that is available for reference. And I think this is a similar thing, although they're without the sort of the other side of that, where they're providing all those SDKs and other things. [00:32:59] Speaker A: Based off of I guess is Smithy just kind of like a slightly different version of Swagger Spec. [00:33:06] Speaker D: That's what I understood it as. [00:33:08] Speaker C: I thought it was more of like a serverless hosting thing, but maybe I've got it wrong too. [00:33:12] Speaker D: No, no, it looks like model a service that should be easy no matter what. Smithy is an extensible type, safe protocol agnostic and power services such as aws. [00:33:25] Speaker C: Yeah, it does look more like. [00:33:28] Speaker D: So I read it as more of like here's the definitions and then you can interact with the APIs kind of are generated from that. So not more of. So kind of like the swagger doc. It's a way to ingest it. And I see this being useful because there's definitely times that when I was like, hey, there's this new feature I've been waiting for on insert service here on Alas, you know, and I want. I need day one and I'm yelling at Terraform. I immediately go Terraform and open the issue saying hey, somebody that knows, go, go program this for me. Because I didn't have, I have, you know, Claude years ago to go auto generate the code for me and I would then, you know, go deal with it that way. So here in theory it was. This is kind of a way for your. For you to kind of abstract that logic out. And in theory it's just the definitions they keep up to date. [00:34:21] Speaker A: Yeah, my, my half ass Internet research is that this Reddit thread says basically there are different levels of abstraction. So Smithy is protocol agnostic. While Open API, which is what Swagger was rebranded to, is tied to HTTP and JSON schemas. Smithy is more useful for defining domain models and operations regardless of underlying technology. It can also generate open APIs as one of its targets, they said. But take a look at typespec as well, which is Microsoft's version on Smithy as they think it's more seriously taken care of than AWS takes care of Smithy. So apparently AWS created Smithy, which I didn't know. Yeah, okay, so I understand this now better. So I understand Swagger. So I have the context, the mental model now to understand this abstraction layer of it. So that. Okay, I get why this is a cool thank you for bringing me to. [00:35:09] Speaker C: Yeah. [00:35:09] Speaker A: Yes, yes, yes. [00:35:11] Speaker D: I didn't realize AWS made it though. [00:35:14] Speaker C: Yeah, I didn't know that either. Well, I mean today I'm learning about it for the first time, so I don't. I didn't know anything about it. [00:35:18] Speaker D: We definitely do a lot of pre. [00:35:20] Speaker C: Show research, of course no, we're well, well researched and not learning on the fly at all. [00:35:24] Speaker A: That would be wrong. [00:35:26] Speaker C: I, you know, like thinking about MCP services and stuff, like, I can see this being hugely beneficial, right, for generating that kind of thing and creating your own MCP servers that does all kinds of things across the Amazon ecosystem. So that's neat. It'd be a lot easier. [00:35:44] Speaker D: I view this almost as like the contract between two teams. Like, here's what my API is going to do and here's the. For obviously a lack of a better term, the swagger map of, you know, or the spicky doc of how it's going to work. And I will just keep this up to date. And as long as you're referencing this document, you don't care what the API looks like. So, you know, it's that extra abstraction layer. And I probably assume that this was like the contract framework they used between teams internally. You know, that they just were like, this is cool, let's make it public. [00:36:16] Speaker C: That is the Amazon way. So. [00:36:20] Speaker A: Yeah, I guess this is also interesting when you get into like GRPC and Protobuf as protocols versus using. Again, I do most of my APIs with JSON and HTTP, so yeah, I guess it's just one of those because I'm not doing a lot of that. I guess I haven't mixed with this much, but. [00:36:37] Speaker D: Have you had to deal with GraphQL yet? Like GraphQL APIs a little bit craziness of them that my brain still struggles to wrap around. [00:36:44] Speaker A: I mean, my brain struggles around GraphQL in general. [00:36:47] Speaker D: Yeah, like the mutations and everything else. My brain's like, ah, yeah. [00:36:51] Speaker A: And then you get it and then you get into the. I mean, it was, it was kind of like when the first time someone explained Mongo to me, I was like, I don't, I don't, I don't understand. You call it a document, but it's not a document. [00:36:59] Speaker D: Document. [00:36:59] Speaker C: Yeah. [00:37:00] Speaker A: And then, then people think it actually is still a document store, even today in 2025. Cracks me up because I'm like, no, it doesn't actually store documents, people. It stores JSON objects anyways. [00:37:10] Speaker C: It's just JSON objects on a, on a page. So it's document, right? [00:37:16] Speaker A: Yeah, yeah, that's why it's a document store. But people, people think it. You put your Excel file in there or your Word doc and like it's a doc story, it's where you store it, right? I'm like, no, it' no, stop, stop yourself. You can put a You can put a representation of those documents into the thing as long as it's in JSON format. So. All right. AWS is reducing prices by up to 45% for Nvidia GPU accelerated EC2 instances including the P4 and the P5 families with on demand pricing effective June 1st and savings plan pricing after June 4th, addressing the industry wide GPU shortage that has driven up costs for AI workloads. The price cuts apply across all regions where these instances are available. With AWS expanding at scale on demand capacity to additional regions including Asia, pac, Europe and South America, making GPU resources more accessible for distributed AI training and inference workloads. AWS is now offering the new P6B 200 instances powered by Nvidia Blackwell GPUs through savings plans for large scale deployments previously only available through EC2 capacity blocks, providing customers with more flexible purchasing options for next generation GPU capacity. Compute customers can choose between EC2 instance savings plans for the lowest price on specific instance families in a region, or Compute savings plan for maximum flexibility across instance types and regions with both one year and three year commitment options. The pricing reduction represents AWS passing operational efficiencies from scale back to customers, making advanced GPU computing more economically viable for generative AI apps, employee productivity tools and customer experience improvements. So I guess they were trying to get OpenAI to go with AWS and then they chose GCP and they're like, oh, we have all this capacity available Or I don't know. [00:38:49] Speaker C: Yeah, I think I took issue with the way that this blog post was written and was just squinting all the way through it because it feels like the shortages are lightening up and so they can offer this, which I like because they are really passing down that savings and maybe it's extra capacity, but I don't think so. I think it's because the capacity is available that they can, via supply and demand lower the prices for it. [00:39:20] Speaker A: I mean, to give a 45% reduction, I mean I can see like, oh, we're, you know, we have economies of scale. We're going to give you a 5 or 10% deduction. I could see that, but like 45% tells me that you maybe are struggling to sell this and it is slightly older, you know, P4, P4 and P5 hardware, that's the A100 and the H100 chips. But the fact that you're also even offering sense and savings plans and compute savings plans, like you're, you're definitely giving a lot more purchase options and ability for customers to lock in these compute needs versus before. You were very much like, no, no, you need to use these fleet constructs and we need to be able to re, you know, retake these things away if you're not using it. And it'll save you money in that way because you're not always buying. And customers didn't like that. That's how I read this. [00:40:01] Speaker C: And it might just be how volatile the GPU market is, right, because everyone's going to just migrate on mass to the newest and greatest and. [00:40:10] Speaker A: Well, I think that's really where the sweet spot of most companies AI stories is going to be is let the big models go after the biggest, fastest, baddest training compute and then use last year or the year before's hardware to run your inference workloads at a lower cost and a better bang for the buck. And you still get decent performance out of it. And it sort of mirrors the gaming industry, right? No one needs the best and greatest GPU on launch day because no one's built games for it yet. And so all the gaming get in, they say, okay, well this is the new baseline for all the new fancy games I want to build. And then in two years from now those games come out and I'm like, oh man, my video card doesn't play that game anymore. Then I would force myself to buy the next video card. It's kind of the same thing, but in AI world, yeah, it's interesting for sure. All right. AWS is open sourcing PG Active, which is a postgres SQL extension that enables asynchronous active active replication between database instances, allowing multiple writers across different regions and maintain data consistency and availability. Extension builds on Postgres 16's bidirectional replication feature, simplifying management of active active scenarios for use cases like regional failover, geographic data distribution and zero downtime migrations between your instances. This addresses a common PostgreSQL limitation where traditional replication only supports a single writer architecture, making it difficult to achieve the true multi region active deployments without complex third party solutions. Organizations can now implement disaster recovery strategies with multiple active databases instances, reducing recovery time objective and enabling seamless traffic switching during maintenance or outages. The open Source release on GitHub allows community collaboration on improving PostgreSQL's active active capabilities, providing ADS customers with a supported path for multi writer database architecture without vendor lock in. [00:41:55] Speaker D: I mean, I think it's great. This is clearly something they use under the hood, you know, in RDS or the RDS proxy and probably even in the thought it's on postgres. Finally the feature where you can send a read a write request to a reader and it auto routes like. I'm sure it's in the PG bouncer stuff. They're. They're open sourcing but this has definitely been a pain point where I've dealt with postgres and migrations and managing, you know, how you manage it. And not every developer is always going to be great about sending all the reads to the reader and all the rights to the writer and everything along those lines. So it's a great feature that they definitely happy that they are open sourcing and I hope it gains some traction. [00:42:42] Speaker A: It's also interesting that they announced this just after Snowflake announced the purchase of Crunchy Data. [00:42:48] Speaker C: Yeah. [00:42:48] Speaker A: Which I believe also offered a active, active solution as well as there are a couple other commercial versions you can buy for lots of money. So interesting as well on that part. [00:42:58] Speaker D: But I would assume this had to be in the realm Aria something they were working on because I doubt they're open sourcing it just. [00:43:11] Speaker A: I think Amazon. [00:43:11] Speaker D: Get cloned the code. [00:43:13] Speaker C: Yeah, yeah. [00:43:14] Speaker A: I think Amazon already had this internally. You know, probably using similar technology or engineers they hired away from Crunchy data or left Amazon and went to found crunchy data. You know, somehow there's. [00:43:24] Speaker D: They knew it was coming. [00:43:25] Speaker C: Yeah, yeah. [00:43:26] Speaker D: I mean but I assume this is two months old deal. Looking at the. The repo, it's been committed. The last commit there was two months ago. So they clearly knew it was coming. So the first commit was 11 years ago that I see on the main page. So that's kind of impressive. [00:43:42] Speaker A: Yeah. [00:43:42] Speaker D: Cells were updated 11 years ago. [00:43:45] Speaker A: Yep. I mean cloud seek. I mean how. When did RDS get Postgres support? I bet it ties closely to that period of time. [00:43:53] Speaker D: Don't know. That feels like. [00:43:55] Speaker A: Hold on, I thought you were going to Google it, so I was just waiting for you to Google. [00:43:59] Speaker D: I was trying to think if I could know and then I was like. [00:44:02] Speaker A: No, I mean if you knew that I would call you a total nerd. Like who. Who maintains that data? All right, I barely remember. [00:44:11] Speaker D: Community Release of Postgres 9.6 was on September 26th. RDS release. Oh, sorry. 2016. RDS was November 2016. [00:44:25] Speaker C: It's five years later. [00:44:26] Speaker D: No, no, no. One month later. September 2016 to November 2016. But I'm trying to see like when they. [00:44:33] Speaker A: I mean that's nine years there then. If you think it took them two years to develop Cloud SQL for Postgres 11 years works out just about right. [00:44:41] Speaker C: Yeah. [00:44:43] Speaker A: All right. AWS Network Firewall now includes a built in monitoring dashboard that provides visibility into network traffic patterns including top traffic flows, TLS, SNI and HTTP host headers. Without additional changes beyond standard CloudWatch and Athena costs, the dashboard helps identify long lived TCP connections and failed TCP handshakes, making it easier to troubleshoot network issues and spot potential security concerns that previously required manual log analysis. This addresses a common pain point where customers had to build custom dashboards or use third party tools to visualize network firewall data. Now providing out of the box insights for faster than incident response. Setup requires enabling flow logs and alert logs in network firewall, then activating the monitoring dashboard and a straightforward process that immediately provides actionable network intelligence and starts costing you more money. So awesome. [00:45:29] Speaker C: Well, so yeah, I mean I've set this up like in a build your own, just sort of using Athena, which is the easy button, which I say with my tongue firmly planted inside of my jeep. [00:45:42] Speaker D: Yeah, I was gonna wonder. I think with easy and Athena in. [00:45:45] Speaker A: The same sentence, felt my experience with Athena has never been easier. [00:45:48] Speaker C: Yeah, Athena is a tool I love to hate. [00:45:50] Speaker D: I feel like 50% of the time I get to work and the other 50% of the time I just swear at it and walk away. [00:45:55] Speaker A: And if I can get the index right, it's all beautiful. But if the index is wrong and most of the Amazon documentation is wrong, it's a nightmare. [00:46:04] Speaker C: Yeah, anytime you have to partition your data into subsets and then query across them, it's like, oh cool, can't wait to do that. But I mean, that said, the ability to be able to query that data once you get it working, I think it's the greatest tool in the world, except for when I have to use it. And so being able to sort of write this or not have to write that right, it's just built in. Because that was a very common misconception when this feature was released was like, okay, cool, let me see where the firewall is blocking my connection. No, you have to build out all this like scaffolding and run a query and parse the query and it's in, it's ugly and you know, you can't really just make sense of it easily. And so like this is, you know, it feels like a long time coming that they've, they're, they're offering this. But it's cool, I do like it. [00:46:59] Speaker D: So I, all I just also heard was Ryan's hundreds of hours of Work just got replaced with a built in feature. [00:47:05] Speaker C: You know, I live for that though. Like that's exactly what I want, you. [00:47:08] Speaker A: Know, being Sherlocked as a badge of honor because A, I was smart enough to come up with it before Amazon and number two, they now killed it so I don't have to maintain it. Right. [00:47:15] Speaker D: Yeah, it is, it's really the second one. But you got to do the handle the migration from your system, which probably most of Sharp Edge is over. [00:47:22] Speaker C: Have you ever migrated from Rye Tools? You unplug it, that's fine. [00:47:26] Speaker D: I've helped you build your tools and. [00:47:28] Speaker A: Typically the availability goes up right away. I mean, most of Ryan's tools live in his GitHub shitty repo. [00:47:33] Speaker C: Yeah, that's right. [00:47:33] Speaker A: You know, so, you know, it's always, always a mark of quality, first of all. And then, you know, but you know, one of the challenges typically is that Ryan will be like 80% of the way there on it and then they'll sure lock it. But they won't Sherlock. What? He's the one feature I want. They'll sure, like. No, they'll only show like, like 20 to 30% of it. [00:47:55] Speaker C: Yeah. [00:47:55] Speaker A: And then like, so he still has to complete that last 20%. Yeah. Which annoys him. [00:48:00] Speaker C: Yeah. [00:48:00] Speaker A: For them to eventually kill it like two years later when they finally get to feature parody with him. Yeah, that's. That's the more real realistic Sherlocking that's. [00:48:07] Speaker C: Happened so many times. [00:48:09] Speaker D: And the two of us have been in there when they released specific features and we were like, are you guys serious? [00:48:15] Speaker A: Yeah. [00:48:16] Speaker C: And I can't use this tool because it doesn't do the one thing I really wanted. [00:48:20] Speaker D: Yeah. Security, day one. Why would we need that? [00:48:23] Speaker C: Yeah. [00:48:23] Speaker A: Who needs encryption at rest, right? [00:48:25] Speaker D: Yeah. Yeah. Day one. Why would you want encryption? Don't worry about encrypting everything. [00:48:30] Speaker C: Still better to about managed Kafka. [00:48:33] Speaker A: Still better. Well, let me give you something. Some new features for Site to Site vpn, which is one of those things like they're still developing features for this service. AWS Site to Site VPN now will integrate with Secrets Manager to automatically redact pre shared keys and API responses, displaying only the ARN instead of exposing sensitive credentials using the new Get Active VPN tunnel status API eliminates the need to enable VPN logs just to track negotiate secure parameters like ikeversion, DN groups and encryption algorithms, reducing operational hybrid. Thank you. [00:49:04] Speaker D: Seriously, so many times. [00:49:06] Speaker A: I've done this so many times. And the third one, AWS has added a recommended parameter to the git VPN Connection Device Sample Configuration API that provides automatically configures best practice security settings, including IKEV2, DH Group 20, SHA or SHA384 and AAS GCM256 also. Thank you for that. Finally, these security enhancements come to you at no additional cost. Thank you. Address common VPN configuration challenges where customers often struggle selecting appropriate cryptographic parameters or accidentally exposing PSK and logs. It's really that I just don't like reading the docs. Like, to be honest, it's every time I had to go look at the doc and hey, what is SHA384 again? Is that the one I need or is it the other one? And so just being able to query default parameters that are best practice is super nice. [00:49:51] Speaker D: I was going to go with the logs the amount of times back in the day before they really even publicly expose the logs. I've opened support tickets, you know, and was like, okay, we need to set this up to the corporate Palo Alto or the corporate whatever, Cisco, how do we do this? Okay, great. It's not working. Amazon, we can't see these logs. Can you tell us what we need to do? Like where is it failing on the site to say encryption? And it was always phase two. And I've just given up a train to understand that stuff. Yep, it's done three times. [00:50:22] Speaker C: And you know, like if you're a small company and you're or you're a single developer, like once you get to this aspect of networking, like if you're setting up a VPN between these two, two workloads or two environments or something, like you're out of your depth very fast. So like having these tools be much easier to consume for like a, just a, you know, typical human is great. You don't have to have, you know, your, your network degree to understand how these things actually work. It's just great. [00:50:52] Speaker A: I do have to apologize to our friends in the Milan region. You're the only region that didn't get this feature. And I don't know why, but literally it says these gateways are available in all AWS commercial regions where AWS site to site VPN is available, except Europe Milan region. And I'm like, I'm. Why? [00:51:09] Speaker C: Why the subjects, you know what you. [00:51:11] Speaker A: Did you like, who did you make mad? What did you do like at the Milan, you know, Amazon summit? [00:51:20] Speaker D: Like, are they like, like a change freeze that like whole region. Like something be like is blowing up here, the seeds. So they're not pushing new features there. [00:51:27] Speaker A: Like, yeah, so weird. And that's a. [00:51:31] Speaker C: That's a good one. Yeah. [00:51:33] Speaker A: I'm used to. Not. I'm used to saying, you know, Gov cloud not included, China regions not included. But Milan, that one's new to me. [00:51:39] Speaker C: Yeah. Yeah. [00:51:41] Speaker A: All right. Moving on to gcp. They've got a lot of good goodies for us this week. First up, Google Pub sub now supports JavaScript user defined functions or UDFs for in stream message transformations, eliminating the need for separate services like dataflow or cloud run for simple data modifications. This reduces latency and operational overhead for common tasks like format conversion, PI redaction and data filtering. The feature allows up to five JavaScript transforms for a topic or subscription transformations happening directly within Pub Sub before messages persistence or delivery. This positions GCP competitively against AWS event bridges, input transformers and Azure service buses. Message and restrict capabilities. Key use cases include data masking for compliance, format conversion for multi system integration, and enhanced filtering based on message content rather than just attributes. Industries handling sensitive data like healthcare and finance will benefit from built in PI redaction capabilities. Service integrates seamlessly with existing Pub Sub features like import topics and export subscriptions, continuing Google strategy of simplifying streaming architectures. Additionally, transforms include schema validation and AI inference are planned for future releases available now and general availability through Google Cloud Console and gcloud CLI with standard Pub Sub pricing applying to transfer messages. The JavaScript runtime limitations and performance characteristics aren't specified, which may be important for latency sensitive apps. So sharp edges. Yeah, but awesome. [00:52:57] Speaker C: Yeah, I mean the fact that this happens before resilience, the OR persistence layer is key right? Because it's difficult to undo anything you introduce once that happens and stuff like be careful test. [00:53:09] Speaker A: It's also bad if you just corrupted all your data with this transform because you've messed up the UDF and now it's not stored anyway you can recover it, but there's that other side of it too. [00:53:17] Speaker C: Yeah, I mean there's so many things you can shoot yourself in the foot with any kind of messaging, but I can't wait to struggle with JavaScript and get frustrated with that only to overload this and try to make it do something that's too much. [00:53:33] Speaker A: Five transforms is all you get butter yourself to two and then you will never have a problem. [00:53:39] Speaker C: Yeah, that's not how I develop. You know, could do this one thing. [00:53:43] Speaker A: And then it's so when you're in chat GPT and you're like right, I can only have five transforms like write it multiple times in your prompt and that way call. [00:53:52] Speaker C: Yeah, that's probably the only way I'd make this to work. But I do like this for things like masking data and those simple transformations like you got that one data source that doesn't publish strings or whatever. So I do like that. [00:54:12] Speaker B: There are a lot of cloud cost management tools out there, but only Achera provides cloud commitment insurance. It sounds fancy, but it's really simple. Achera gives you the cost savings of a one or three year AWS savings plan with a commitment as short as 30 days. If you don't use all the cloud resources you've committed to, they will literally put the money back in your bank account to cover the difference. Other cost management tools may say they offer commitment insurance, but remember to ask will you actually give me my money back? Achero will click the link in the Show Notes to check them out on the AWS Marketplace. [00:54:51] Speaker A: Well, for those of you who are running SAP 4 HANA and the M3VM class is just not enough for you. Google has the new M4 VMs with up to 224VCPUs and 6 TB of DDR5 memory, targeting memory intensive workloads like SAP HANA and SQL Server with 66% better price performance than the previous M3 generation and full SAP certification across all shapes. It's built on the Intel's 5th gen Xeon processor. Emerald Rapids NIM4 offers two memory to VCPU ratios 13.31 and 26.61 and delivers up to 2.25 times more SAP's compared to M3s, making it the first memory optimized instance among hyperscalers to use these processors. Infor leverages Google's Titanium offload technology for 200 gigabytes per second. Networking bandwidth integrates with hybrid disk storage support for up to 500,000 IOPS and 10,000 Mbps with dynamic tuning capabilities and storage pooling for cost optimization. The instances are backed by a 99.95% single instance SLA and support hitless upgrades and live migrations for minimal disruption during maintenance, with initial availability in five regions US East 4, Europe West 4, Europe West 3 and US Central 1. Infor completes Google's memory optimized portfolio alongside X4 with up to 32 terabytes of memory positioning GCP competitively for large scale in memory databases and analytics workloads with both on demand and committed use discount pricing options. [00:56:12] Speaker C: This is a conspiracy by two companies to just milk the entire marketplace of money. [00:56:18] Speaker A: SAP hana yeah here certify a Hana and I'll print money for you. [00:56:24] Speaker C: Seriously, it goes against all my developer instincts of like. Yeah, no, scaling horizontally is probably the a better way to do this, but nope. [00:56:35] Speaker A: Unfortunately when you're running SAP4HANA and SQL Server and Oracle, you need all the. [00:56:40] Speaker C: Memories and it's all in one place. And sweet Jesus, if that thing gets. [00:56:43] Speaker D: Reboot, God forbid you have to cross the network layer to talk to your database or anything else every time. [00:56:50] Speaker A: Have you ever tried to shar a database like at scale? [00:56:53] Speaker C: Yes, actually, I've done that successfully a number of times. [00:56:58] Speaker A: Okay, you should tell us more about this. [00:57:01] Speaker D: Wait, wait, wait, wait, wait, wait, wait, wait, wait, wait. We're not moving past that comment. Tell us more about how you. [00:57:07] Speaker C: So before I got into financial services, I used to do meaningful work. And you know, one of the things last projects I was working on was using open source CEPH to sort of build a competitive object store. And it was back before I really knew what AWS was and I knew that S3 was a thing and that I had to compete with, but that's all I need. And so we had large data sets, primarily photos and mail attachments that we had to try to serve. And one of the things we had to do was when we replaced a single cluster, we typically had to add on another cluster and then reshard that data across all its replication points. If you're not already asleep. I've just done this successfully and I found it very challenging and very fun. [00:57:52] Speaker A: So you picked the correct shard key in your model? It sounds like. [00:57:56] Speaker D: Yeah. [00:57:59] Speaker C: So I did not pick a key. I created a key which was key. Yeah, you're right. That is usually the challenge. [00:58:09] Speaker A: Yeah. So most sharding disasters occur because you chose the wrong shard key and everything is ben for their end. And I know this because I inherited a 300 node Mongo database one time because they sharded incorrectly. Correctly. And the first thing you got to remember about Mongo is Mongo says always have the rule of three. So you had to have three nodes minimum per shard. And so when you take your shard set and you say, I'm going to do 100 shards, that now means you have 300 servers and had to be managed and synchronized. That didn't go well. [00:58:44] Speaker C: Sounds a little chatty. [00:58:46] Speaker A: Real chatty, real bad. Not great. Very early days of Mongo. And then. So it was all bad. It was all bad. Luckily never got out of dev. [00:58:59] Speaker C: It was web scale, right? [00:59:00] Speaker A: It was web scale, right? Yes. [00:59:03] Speaker D: This was still in dev. That's the way it was designed. [00:59:06] Speaker A: So you either had a really it hit the 300 node. We hit 300 when we were in low test because we had to scale up to handle the load that we were going to do. And that's when we realized this is terrible. Yeah, dev was only like, you know, 12 nodes. It was. [00:59:21] Speaker D: Well, that's what I was thinking. Like, I was like, wow, your dev environment is massive. So what was the production workload going to be like? [00:59:28] Speaker A: 3,000 Low test what production would have been like. But it was very quickly you realized that the data center could not handle this. Yeah, we did not have the networking throughput required. All right, moving on to Google Cloud, releasing optimized deployment recipes for Meta's Llama 4 Scout 17B 16 and Maverick 17B 128e and Deepseek's V3R1 models on AI hypercomputer providing step by step guides for running these open source LLMs on Trillium TPUs and A3 Mega Ultra GPUs. The recipes leverage ZStream for TPU inference and VLMs SGLang for GPU deployments with pathways enabling multi host serving across TPU slices. The same system Google uses internally for Gemini model training and serving Max Text now includes architectural innovations from Deep SEQ like Multi Head, Latent Attention, MOE Shared routed experts and yarn rope embeddings, allowing developers to experiment with these newer model architectures on Google Cloud infrastructure. I think they made this article up. [01:00:27] Speaker D: I'm like listening to it and I'm like, I think you're making up half these words. [01:00:32] Speaker C: I don't even know what they are. [01:00:33] Speaker D: What does a jet stream have to do with a TPU in the yard? [01:00:37] Speaker A: Little rope where shots, Yarn rope embeddings. Yeah, I know mo shared routed experts. I'm like, what? [01:00:45] Speaker D: I feel like we should have called this story before we started this podcast. [01:00:49] Speaker A: I blame you both for not telling me to kill this. [01:00:51] Speaker D: This is fair. [01:00:51] Speaker C: We, we should have done our diligence on this one. [01:00:54] Speaker D: Thought somebody smarter than me on this podcast was going to know more about this and I was. [01:00:59] Speaker A: See, the smart guy is not here. That's the problem. [01:01:01] Speaker D: Yeah, British people not being here. [01:01:03] Speaker C: I did open this article because when we were going through the pre read I was like, this seems suspect and I think I got distracted by something. [01:01:12] Speaker A: Shaq. Yeah, Mo is mixer of experts. That's weird. [01:01:16] Speaker D: Okay, moving on. [01:01:18] Speaker A: Jetstream Google throughput and memory optimized engine for LM inference on XLA devices. Oh, batchette stream. Okay, there you go. I don't know what that is. [01:01:27] Speaker C: Yeah. [01:01:27] Speaker A: Okay, let's move on to BigQuery Query introduces I understand. Yeah, at least slightly better. BigQuery introduces reservation fairness and predictability features that allow organizations to set absolute maximum slot consumption limits and distribute idle capacity equally across reservations rather than projects providing more granular control over resource allocation and costs. In enterprise editions, the new runtime reservation specification feature enables users to override default reservation assignments via the cli, the ui, SQL Server or, sorry, SQL Command or API at query execution time with role based access controls for improved security and flexibility in multi team environments. Autoscaler improvements deliver 50 slot increment granularity down from 100 with near instant scale up and faster scale down capabilities, allowing more responsive resource adjustments to workload demands compared to previous iterations. Reservation labels now integrate with cloud billing data for analysis. Slot attribution askew Nailing detailed cross tracking and optimizations by workload or team, addressing a common enterprise requirement for chargeback and showback scenarios, These updates position BigQuery's workload management closer to dedicated resource pools found in Snowflake's multicluster warehouse or AWS Redshift's workload management queues, both more dynamic allocation options suited for variable analytics workloads. Now, if you're going to use reservation fairness and you're not going to honor the project boundary, I will cut you Ryan, when you take my BigQuery slots. [01:02:48] Speaker C: Yeah, no, I mean it is weird to have isolation construct which is what a project is and then be like yeah, we're going to ignore it, but I think this is more akin to, you know, being, you know, negotiating your savings plan at an organization level. But you know, for, for BigQuery. So I think it still does make sense because it allows reuse and that kind of thing. But it is sort of funny that way. But. And you know, if, if anyone doubted my comment at the top of the show that finops is incredibly complicated and having a dedicated resource is super important, this is the article that really should highlight that. [01:03:34] Speaker A: Yeah, I mean I do think you probably want to think about your reservation specification. You want reservations for production, reservations for dev, and then maybe you want to share your dev ones but not your production ones. There's definitely thought that needs to go into this. Or again, you're going to have a sad day. Especially because it did tell you that it was going to auto scale up by 50 slot increments, but that'll only work until you hit the quota limit and then all bets are off. So it's great. It, it's good to have this, but definitely keep it in check and think through before you implement. [01:04:04] Speaker C: Yeah. And it's a very difficult workload to forecast. [01:04:06] Speaker A: Right. [01:04:07] Speaker C: Because it's, it's based off of, you know, your query and it's based off of the data set. And so it's, you know, if you have something where you're inputting a lot of customer data and storing that, that data changes format or something like that, it can be really challenging. So this is one of those areas where, you know, it's easy to save money if you look at it from like, it looks like a low hanging fruit, but it's actually very difficult to implement. [01:04:30] Speaker D: It sounds difficult just to start off. So I don't think it's a low hanging fruit. [01:04:34] Speaker C: No, it's such a big, it's such a big number in one place. [01:04:37] Speaker A: Right. [01:04:38] Speaker C: Where it's like, that's what makes it a target. Like wait, if we just change that one thing from the executive level, Justin will come in and be like, hey, why don't we introduce reservations in this one data set? And we're like, it could save, you know, 50 grand a month. And, and he'll wonder why it's not done in two weeks. [01:04:57] Speaker A: I just, do I ever wonder why it's not done in two weeks? [01:05:00] Speaker D: Yes. Only why is spot instances not done in two weeks? On a ECS cluster, maybe. [01:05:05] Speaker A: Yeah, well, that was different. We were, we were under pressure. [01:05:11] Speaker C: Yeah. I will tell you it's four weeks and you'll immediately cut it down to two weeks. And then so. [01:05:19] Speaker A: Because I learned you just cut things down when the, like, you just double them. And so I know, like, let's go your system. [01:05:24] Speaker C: I don't just double though. I think about all the use cases. [01:05:27] Speaker A: I mostly take your, your, your estimate as face value. Just, you don't care what the business needs. So. [01:05:35] Speaker C: That is fair. That is fair. [01:05:37] Speaker A: Yeah. You know, there's, I had the technical versus the business. That's where I live. You know, I'm like just, you know. [01:05:45] Speaker D: Okay, so back in the day, Peter had this principle that whenever anybody given s estimate for a statement of work, he just took the number, immediately doubled it and that was the number used all statements of work. So two today I still follow the Peter principle and maybe he's listening to it. I just double it in my head. So I talk to my team, there's a few people on my team. I4x it still. Because I'm like, yeah, it's not gonna happen then. But like double is like my bare minimum. [01:06:10] Speaker A: You're not performing at a high level. [01:06:12] Speaker C: Yeah. [01:06:13] Speaker D: You have not embraced cloud enough, sir. So you are Forex. [01:06:19] Speaker A: All right, well, Google becomes bearing gifts for Ryan. Four new security Command center capabilities. The first one is the security Command center now offers you an agentless vulnerability scanning for compute engine and GKE at no additional charge. Eliminating the need to deploy and manage scanning agents on each asset while providing coverage even for unauthorized VMs provisioned by adversaries. And if I get you to kill Qualys for this, I'd be super happy. [01:06:40] Speaker C: Ah, no, it's not going to kill Qualus. It's going to kill the Ops agent. Okay, fine. [01:06:48] Speaker A: Number two, container image vulnerability scanning is now integrated through artifact analysis with scans included at no extra cost for SEC Enterprise customers. When images are deployed to gke, cloud run or app engine. Consolidating security findings in one dashboard. [01:07:01] Speaker D: That sounds nice. [01:07:03] Speaker C: It's interesting that it's not at the artifact registry. [01:07:07] Speaker A: Yeah, that's a weird. [01:07:07] Speaker D: Well, it's what's running, so it's a little bit easier. But also, how much is SCC Enterprise? [01:07:13] Speaker C: Oh yeah. [01:07:15] Speaker D: Is it like a big step to get into there? [01:07:17] Speaker C: Yeah, yeah it is. [01:07:18] Speaker D: All right. [01:07:18] Speaker A: Have you ever bought Splunk for sim? [01:07:20] Speaker C: No, it's not that bad. That's second. [01:07:23] Speaker A: That's what I'm saying. Have you bought that? If you are familiar with that pricing, it's not as bad as that, but it's not that far off. [01:07:29] Speaker D: So it's my left kidney, not my right one. [01:07:32] Speaker A: Not both kidneys, just one. [01:07:34] Speaker D: Yeah. Okay. [01:07:35] Speaker A: Got it. The third item for you for SEC is Cloud Run Threat Detection introduces 16 specialized detectors that analyze serverless deployments for malicious activity, including behavioral analysis, NLP powered code analysis and control Plane monitoring capability is not available in third party products. [01:07:52] Speaker C: Yeah, I mean the thing I love about this most is it's a detector I didn't have to create. So that's. That's usually. [01:07:57] Speaker A: Are you going to create it before though? [01:07:58] Speaker C: No, of course no. [01:07:59] Speaker D: Okay. [01:07:59] Speaker C: I'm going to think I'm going to lie awake at night thinking I should have done that. But you know, it's one of those things that never gets to. And it's. You know, for me to write this would be very difficult. [01:08:09] Speaker A: Right? [01:08:09] Speaker C: Like thinking about the service technologies and what. What the malicious activities would look like from a SIM perspective. [01:08:16] Speaker A: Like it's crazy hard to write some. [01:08:17] Speaker C: Of these detections and so I love this so much. [01:08:22] Speaker A: And the last one, I kind of like SCC automatically detects connections to known malicious IPs by analyzing internal network traffic without requiring customers to purchase, ingest and analyze VPC flow logs separately. Unlike third party security tools. You bastards that charge extra for this capability. [01:08:39] Speaker C: This is interesting because I'm like wait, but you do have to have flow logs. [01:08:44] Speaker D: Yeah, I was going to say, don't you have to have flow logs enabled for them to ingest them to. Therefore. So you're just now sending it to a third party to process them a second time? [01:08:53] Speaker A: Yeah, yeah, basically. [01:08:55] Speaker C: I mean, or I'm spending money uselessly. I gotta go look at that now. [01:08:59] Speaker A: And maybe, maybe they're gonna give you just enough of the VPC full logs that you don't have to actually set it up. You know, be cool. [01:09:05] Speaker D: But your security. You don't care about the cost ride. [01:09:07] Speaker C: Nah, you know, I'm his heart is still cloud. Still I'm true to my roots. I am bringing the cloud to security and security to the cloud like always. And I used to try to do that with finops too and I don't do that anymore because I'm out of capacity. [01:09:21] Speaker D: So your old man yelling at cloud. [01:09:23] Speaker C: Got it. I've always been that. [01:09:26] Speaker A: That's not new. [01:09:28] Speaker D: Still old man. Yellow cloud. [01:09:29] Speaker A: Yes, yes. Google continuing to roll out MCP toolbox features, now enabling AI coding assistants like Cloud Code Cursor and Windserve to directly query and modify Google Cloud databases, including Cloud, SQL Alloy, DB Spanner and BigQuery through natural language commands. In your idea, developers can skip writing complex SQL queries and instead use plain English to explore database schemas, create tables, modify structures, and generate test data. Tasks that previously took hours or days can now be completed in just a few minutes. The tool implements Anthropic's MCP Model Context Protocol, an emerging open standard that replaces fragmented custom integrations between AI systems and data sources with a unified protocol approach. This positions Google competitively against AWS, CodeWhisper and GitHub Copilot by offering deeper database integration capabilities, though those services don't yet support direct database manipulation through natural language because no one wants to write the incident they deleted the database. AI drop tables. [01:10:23] Speaker C: Yeah, you know, like I will find out firsthand pretty soon how great or not great this is because it's something that I'm working on my day to day, which is a big bigQuery data set and I want to query it and I don't like writing queries. I never have. I've complained about SQL I don't know how many times. [01:10:45] Speaker A: On this podcast. [01:10:47] Speaker C: So it's like if, if I can just make the. The NCP server do what I want and return the data. It's amazing. I can't wait. [01:10:57] Speaker A: Today I learned you don't like SQL queries. [01:10:59] Speaker D: I don't like them. [01:11:00] Speaker C: Today you've learned this. [01:11:02] Speaker A: No, I know. [01:11:02] Speaker C: Okay, that was a joke. [01:11:05] Speaker A: He's like, you complained multiple times on the show and I was like, have you? I don't recall that. [01:11:10] Speaker C: Okay, good. For a second I thought I was living alternate universe. [01:11:15] Speaker A: It's Berenstein Bears, not Berenstein Bears. All right. [01:11:18] Speaker D: Okay. Right. I think at my day job there's a rule, if I log into SQL, we should all quit. So, you know, that's where I am. Like, don't. Yeah, I'm in your po. [01:11:31] Speaker A: Well, Google is marrying its expensive vertex AI engine with an expensive observatory tool called datadog, making two very extensive services that now measure each other through its new AI agent console, providing unified visibility into autonomous agents, actions, permissions and business impacts across third party and Google Orchard agents. Datadog can now provide you insight into how those things are working through the application layer, the model layer, and the Azure layer and data layer, all being monitored for cost. Datadog has implemented KubeCloud's Active Metrics APIs to reduce monitoring costs by only calling APIs when new data exists. Sure. Colonizing their private service connect support to minimize data transfer expenses. And the expanded BigQuery monitoring helps teams identify top spenders, slow queries and fail jobs of flagging data quality issues. Addressing a key pain point for organizations. Using BigQuery for AI data insights, customers can purchase data directly through Google Cloud Marketplace with deployment in minutes, making it straightforward for GCP users to add comprehensive AI observability to their existing infrastructure. Also, if you have a Google Cloud spend commit, you're going to miss buying your datadog purchase through Google Cloud Marketplace is a great choice and I highly recommend it. And I don't mean to be ill of them because they did send us a lovely bottle of whiskey that I'll be sharing with the three of us when we all get together again as a congratulations for our 300 episode. So thank you to Datadog for that. [01:12:48] Speaker C: But you know, datadog only has some of the responsibility because a lot of it is for all of these managed monitoring solutions. It's what you send to it and they're just charging by ingestion rates points. And so like if you're in control of your data, your spend is not going like crazy big. But no one, including Me is in control of the data. So. [01:13:10] Speaker A: Yeah. [01:13:12] Speaker C: But I mean, I do like the, you know, the Active Metrics API that someone is finally taking a different approach from the cron job that runs against every API every five minutes to scrape data. So I'm, I'm, I look forward to this sort of expanding and I hope this works out for people so that it does gain popularity. [01:13:33] Speaker D: I just want to know what job I need to take that I can burn this much money on. I need to. Need a look at that. Where do I get to play with more of these tools? [01:13:44] Speaker C: For sure. Yeah. Any logging and monitoring. Splunk, datadog, New Relic. They're all expensive and they all are invaluable. [01:13:53] Speaker A: They're amazing. I mean, they're great tools. Datadog is great. I mean, I always like their dashboards and the way they do integrations. I mean, we're a new relic shop. We like New Relic as well. You can't really go wrong with any of these tools other than with your cfo, right? [01:14:06] Speaker D: Yeah. [01:14:08] Speaker C: And we've also built our own internally and it was. [01:14:10] Speaker A: Yeah, we don't do that. [01:14:11] Speaker C: Right? [01:14:11] Speaker A: Yeah, don't do that. Please don't buy friends. Don't let friends build elasticsearch clusters. We should make, we should make stickers for that. [01:14:17] Speaker B: We should. [01:14:17] Speaker C: That's a good idea. [01:14:18] Speaker A: Write that down. All right. Our next and final story for Google Cloud is Google Cloud Serverless for Apache Spark is now Generally available within BigQuery, eliminating cluster management overhead and charging only for job runtime rather than idle infrastructure. This integration provides a unified developer experience in BigQuery Studio with seamless interoperability between Spark and BigQuery SQL engines on the same dataset. Service includes Lightning Engine in Preview, which levers up to 3 1/2 faster query performance through vectorized execution and intelligent caching. Prepackaged ML libraries like Pytorch and Transformers can come standard with Google certified Spark images, plus GPU acceleration support for distributed AI workloads. Big Lake Metastore enables Spark and BigQuery to operate on a single copy of data, whether in BigQuery managed tables or open formats like Apache, Iceberg and Delta Lake. All data access is Unified through the BigQuery storage Read API with no additional cost for reads from serverless Spark Jobs and BigQuery. Spend based codes now apply to serverless Spark usage as well. And the service supports full OSS compatibility with existing Spark code across Python, Java, Scala and R Enterprise. Features include job isolation, CMEC encryption or Customer managed encryption keys and custom. Org policies and End user credential support for data access traceability and the Gemini powered features include PySpark cogeneration with data context awareness and cloud assist for troubleshooting recommendations. Both of those are in preview to you. [01:15:36] Speaker C: I mean the fact that you can run a Spark job or just do like a SQL query on the same data set without having to reformat and restore your data is pretty rad. I didn't know that that existed. So that sounds awesome. [01:15:50] Speaker A: How do you feel about writing Spark queries versus SQL queries? [01:15:52] Speaker C: Oh yeah, I love it. No, it's also. [01:15:56] Speaker A: Is it just query? [01:15:58] Speaker C: No, no, no. [01:15:59] Speaker A: It's. [01:15:59] Speaker C: Yeah. No, it's all hard and it's all thinking about your data structures. Although like I. I guess I don't know. Yeah, you know, so it's. These things are neat. I do like that you'd be able to take a lot more sort of open source, machine learning sort of notebooks and blogs and then apply it directly to your BigQuery data set, which is neat. That's a uniformity that I think users didn't have before. It's kind of cool. Okay. You know, and it's interesting that where the announcement is as part of the serverless, you know, running of it, which is, you know, like a lot of bigquery execution and job execution is serverless and so it is kind of neat that they're making that even bigger. [01:16:40] Speaker A: All right, let's move on to Azure with two great features from this week. First is Azure Prompt Shields provides real time protection against prompt injection attacks which owaspizes the top threat to LLMs. No duh. Finalizing inputs to detect both a direct jailbreak attempts and indirect attacks that have added in documents or emails. The service integrates directly with Azure OpenAI content filters and Azure AI Foundry offering contextual awareness to reduce false positives and a new spotlighting capability that distinguishes between trusted and untrusted inputs and generative AI apps. Microsoft Defender now integrates with Azure AI Foundry to surface AI security recommendations and runtime threat alerts directly in development environment, helping developers identify prompt injection risks early in the development process. Enterprise customers like AXA and WRTN Technologies are using Prop Shield to secure their AI deployments. The acts of preventing prompt action and their secure GPT solution and WRTN leveraging customizable content filters for their Korean AI companion platform. Azure OpenAI customers can enable prompt shield through built in content filters while Azure AI Content Safety customers can activate it for non OpenAI models, positioning Microsoft ahead of AWS and GCP and offering integrated prompt injection defense capabilities. [01:17:47] Speaker C: I don't think that last bullet point's true, but since you can do that in gcp, maybe. I don't know about AWS specifically, but these types of tools are invaluable. [01:17:59] Speaker A: Right? [01:17:59] Speaker C: Because it's AI is such a changing landscape. You're writing an AI app taking inputs from a customer and the risk of, you know, that there's responsible AI is built into all the larger models. But if you're trying to use a custom model or you're trying to use a lot of different models and trying to really sort of like merge together the responses, like having this as like a. An overall layer aspect on it is super key from protecting yourself from writing a chatbot that talks a whole bunch of trash about your business and your competitors. [01:18:33] Speaker D: Yeah, I mean, it's a good feature. It's an extra fee on top of what you're already spending on Azure OpenAI. So your finops person is going to have more fun trying to calculate what the cost of every single query is for. If you know, whatever you're doing, it does vary a little bit of how they charge and what they charge. And then I just actually was curious and I saw they have committed tiers, which is only like $200,000 a year for tech. So, you know, you're definitely leveraging a lot of Azure OpenAI at that point, but it's definitely a great feature. And it's just also slightly terrifying to hear OAuth top 10 is now on AI, not just on, you know, websites. So every time I hear os, I'm like, God, why is it an AI now too? Well, we were done with that. [01:19:21] Speaker C: I mean, it's the, it's the, it's just new rules. The top 10 for websites still exists. You just got 10 more to worry about now. [01:19:30] Speaker A: Yeah, and you are correct. Both Amazon and GCP have this capability. Yeah, that was cloud hallucinating. [01:19:40] Speaker C: You know, it's neat. I got to play around with these today and talk about that next week. But yeah, I mean, I like these. I think you're right, Matt. The challenge is that extra fee is problematic and everything's built in tokens, which are just magic money on the Internet. No one knows how to calculate, so it's sort of like, hope this works for forecasting, hopefully your usage is somewhat standard and predictable. [01:20:08] Speaker A: And then finally, Microsoft is introducing Jazz, a JVM launcher that automatically optimizes Java applications for Azure cloud environments by detecting container limits and selecting appropriate heap sizing, garbage collection and diagnostic settings without Manual configurations The tool addresses a significant problem where over 30% of developers deploy Java workloads with default open JDK settings. Never seen that before that are too conservative for cloud environments leading to underutilized resources and higher operations operational costs Currently in private preview for Linux containers using Microsoft build of OpenJDK and Eclipse Tamarind Java 8 JAWS simplifies deployments for replacing complex Java ops configuration of a single command Jazz Jar My app jar the robot includes app CDS support for improved startup times future project latent integration and continuous tuning capabilities with Prometheus telemetry sharing positioning it as a cloud native alternative to manual JVM tuning or tools like Paquito build package I do believe is it Google has this or was it aws? Someone has something very similar to this to help you optimize your Java JVM boot time. Because yes, this is a terrible problem that all developers screw up. Yeah. [01:21:15] Speaker D: When are we getting rid of heap sizes? Just like when can we get rid of that code? [01:21:19] Speaker A: Why doesn't Java just make it part of the freaking thing? [01:21:22] Speaker D: Right? Like it's not that difficult and it causes so many outages. [01:21:26] Speaker A: Yeah, so many. [01:21:29] Speaker C: Yeah. No, I've been burned up all sides of these default configurations. Like on one one hand it's not big enough. On the other hand it's always too, too large for smaller stuff. And so it's. When you're in your troubleshooting it's. It's now one of my. If it's a Java app, it's the first thing I look at and I think a lot of Java people look at that very much first because garbage collection sucks. [01:21:50] Speaker D: I mean I remember even. [01:21:51] Speaker A: But even. I mean this is just one of those like Linux things kind of in general is that the batteries aren't included and a lot of things and so even like Apache configurations and Java configurations and Tomcat configurations, like they're all like the one thing I do in most of those systems now when I get them is I write code that automatically detects the memory. [01:22:09] Speaker D: The amount of times I've done that. [01:22:11] Speaker A: Yeah, I mean I literally have it in my Justin shitty script rebuild. Yeah literally boot on boot. I let it run and then it takes care of it because manually. Manually dealing with that is such a nightmare. [01:22:23] Speaker D: Well then if you're running spot and you're running a spot fleece or changes sizes every time, you know, do you do the small end? [01:22:29] Speaker A: You know, but yeah, no, but if you do the dynamic then it doesn't matter. It just figures it out and it's beautiful. I love it. [01:22:34] Speaker D: Right. It's just, but like it just feels like these things should be things out of the box at this point and then you could tweak them if you want to override them. Not default to 128 or 256 and then you're like I have a 20 terabytes RAM system. Why am I using 250 megabytes? Hey, by the way, the AI that earlier from the FinOps will tell you scaled down which will be good for you. [01:22:58] Speaker C: I wonder like, I wonder which way this goes and can it do it accurately. Yeah, be funny Michael. [01:23:03] Speaker A: The other way I was laughing because cloudability was telling. It was like you should is recommended action for these M3s that were just sitting around not being used yet was just terminate. I was like, well that's bold because you don't actually know anything about that box. Yeah. [01:23:17] Speaker C: What is it doing? I don't care. [01:23:19] Speaker A: Turn it off. It's not using all of its CPU. So terminate. I'm like that has 25 databases on it. You can't just turn that off. [01:23:26] Speaker D: Well it's not going to affect them and the bill will go down. So clearly it was good life choice, Justin. [01:23:31] Speaker C: Yeah, yeah. I mean there's nothing for, for you know, sandbagging your utilization metrics. So utilizations automatically. 40%. Perfect. Got it. Yeah, perfect. [01:23:44] Speaker A: Awesome. All right, well let's. I'm going to rant about finops tooling because I promised the listeners that we would and they just made it through that show which was long. Yes. To get to here. So yeah, this is the third year I've been at FinOps where I've been sort of looking for the next generation of FinOps tooling of like what are you guys going to do to not just sell dashboarding and RI recommendations and. Right sizing because. Right sizing and dashboarding and all that is basically what everyone does and so every tool has those features and I was talking to somebody he mentioned in this blog post called the coming downfall of Cloud finops Tools Market and who falls first by blog author Will Kelly. I don't know who he is but his I agree with everything he had to say but you know, he basically says FinOps tools market is heading for a massive shakeout by 2027. I think it's sooner than that. With native cloud providers tools like AWS Cost Explorer and Azure Cost management finally catching up to third party vendors by offering free built in features like tagging, enforcement, anomaly detection and savings plan recommendations. They used to be the bread and butter of standalone FedOps platforms. He then goes on to say AI is fundamentally changing the game by automating what FinOps vendors use to charge premium prices for. Instead of manually reviewing cost anomalies or building reservation coverage charts, AI can now generate execute observation plans on real time, making dashboards only look only tools look like expensive relics from a bygone era. The article called out specific vendors who are in trouble, including Kion's desperate pivot to partner with Prosper Ops for Kubernetes visibility Appdio credibility, which despite IBM's backing remains bloated and tied to legacy enterprise reporting models. And there's a brutal reality check for some of the vendors disguised as managed SaaS services companies like Cloudkeeper that promise guaranteed savings but are really just offshore NLS preparing manual reports behind a sleek UI and charging enterprise SaaS prices for it amounts to templated spreadsheets and consulting work as well as lack of deep cloud provider alignment is becoming a death sentence for finops vendors as enterprises increasingly want tools that integrate directly with their CSP contracts, procurement flows and enterprise discount programs. And if you're not one of the big three, with proper building integration, you're essentially invisible to enterprise buyer. So in general I have to agree with him. I think this is a big problem. I think I have some solutions for it, but I don't think any of the vendors want to do this. So on the show floor there was a few vendors who I was really quite interested in because they were doing something different. So one of them is a company I've talked about, I think maybe last year, Adaptive 6. Basically they solve one of the two problems you just talked about, which is if you have a finops tool and it tells you this thing should be deleted and you go delete it in the finops tool. Well now my infrastructure's code is screwed, so they will automatically provide the pull request. They'll tell you they'll push that to the repo, they'll do all of that for you. So if you're making remediations inside your FrontOps tool, it gets tied back to IAC, which you definitely need to happen. Otherwise those drives just come right back next time Terraform Apply happens as well as they do things like it seems like you intended to deploy 10 of these and you deployed 100 of them because you typoed it. It detects certain rules and things like that, which is good. Another one I liked was Stackalet, which is basically the commercialized version of cloud custodian so basically it's a company now built around cloud custodian and making that a commercial product. But in both cases, both those products are really interesting to me. But they both offer you dashboarding and ri recommendation of things because when you're trying to sell to the enterprise, enterprise buyer is looking for a FinOps tool that has those capabilities. And so I think the real solution here is for vendors to either figure out how to build on top of the hyperscalers or and just take advantage of their native tooling or for one of these legacy partners like cloud health or cloudability or, you know, any of these vendors to basically create a platform that these companies can actually build their products on top of. We'll provide you the standardized dashboarding, multi cloud viewpoint. You just plug into us through the marketplace, you sell through us, you get a small cut of whatever you sell through. And basically we're providing all these value added services on top of the data that we already have. And that way you don't have to build those features that aren't differentiators. But they keep these dinosaur companies from becoming true dinosaurs. Give them an option. If one of those partners is really good, maybe they just buy those partners and say, I'm going to buy that and now pick it part of my product. But I think it gives an opportunity for these vendors to start playing in the space and being innovative. Because I was shocked how many vendors, despite AI becoming a much bigger deal this year. I think on the main stage JR said it went from 31% interest in AI last year to 65% interest in AI this year. Which no, duh, I could have told him that was going to happen, but. [01:28:19] Speaker C: Surprised it's not higher. [01:28:21] Speaker A: I was a little surprised I wasn't higher too. But you know, the reality is that there's two things on AI. One is how do I use AI to help with my FinOps workload and what I need to do. And number two is how do I measure my AI inference and learning and training. And so, you know, a lot of the vendors I think have like, oh well, the cloud vendors and focus standard. They're giving me the data so I can visualize it like, okay, cool, but that's not enough. I want more than that. I need, you know, like, hey, how do I optimize my spend? How do I do different workflows to like determine which models are the best for my process? And so there's vendors who are specializing in those things, we talked about them here on the show. Or even hyperscaler tools that Let you test different models of the same prompt to see which one provides the best outputs. But if you could do that in real time based on the query coming into it through the prompt and say, oh, that's a very simple prompt, I could pass that to a lower end model versus this is a very complicated prompt that's going to require thought or reasoning. I'm going to pass over here. Those are things that a FinOps tool could potentially be a provider of or could be a partner of a different company that makes that the whole company. Is that. So I think there's a ton of opportunity. I just think it's going to take someone being bold in the FinOps community in the vendor space to say, hey, stop building the stuff that we already got and just focus on the innovation and let us. And then, you know, figure that out. And so either these companies need to figure out how to sell to the enterprise in a different way that hey, you don't need all the dashboarding and our recommendations. You're getting that already from a hyperscaler. Here's what we can provide to you and we just integrate or like I said, someone like cloud health or cloudability who's got a massive scale, become a platform company, sell platform. And I think that's another way you could tackle this market. But I think they've got to figure this out because there's so much more interesting things to me in the finop space than dashboarding and RI recommendations. I mean, the fact is, once you get through those things, you negotiate your enterprise discount. The next steps are always about architecture. How do I start rearchitecting my application to take advantage of this? Oh, hey, I see you're using netapps. You should move to Object Storage. Here's what your cost savings could be. Yeah, you know, that's something that a tool and AI could help you determine a path and a roadmap for. You know, there's a bunch of stuff around security and being able to do things in a more secure manner that has a direct impact. You get into green, you know, green sustainability initiatives and like, how do you start thinking about data and like, oh, well, you're copying the same data between six different data stores. That's not very sustainable. You know, these are the things that are actually more interesting and have kind of the next generational idea to me of what finops tooling is going to have to start going to do. But no one's thinking about it. They're all just stuck in dashboarding and ris. And I'm just so frustrated about it. And I posted on LinkedIn as I was leaving the conference about this and on Twitter and on Macedon and on Bluesky and the amount of people who reach out and agreed with me, either privately in DM or publicly on my post was pretty impressive. Probably one of my best trending LinkedIn articles in a while. So I definitely, I hit a thing that's I think people are thinking as well. Clearly Will Kelly feels these vendors are screwed. I don't know him, but I'd love to talk to him more about this. But definitely very interesting and I think this is a problem that the Finhops foundation is fine, they'll be around forever. They've got a great model, they're building a bunch of ton of really good stuff on AI this year, AI frameworks, AI platforms, et cetera. But but the vendors have to figure out how to compete. And when you walk around the floor and you see the same dashboard, the same chart, the same ROI recommendation, and you go to 12 or 15 different booths, you're like, okay, this is done. Why do I need to be here? [01:32:02] Speaker D: Yeah, it's all the same data, just slightly spun differently. And I think while I think a tool can do it, I think it's going to have to be a tool with a human behind it, at least for several years. Because sure, moving from one architecture to the other is in theory an easy Solution. Moving from NetApps to S3 or to FSX or whatever, you know, it be there, there are steps along the way. And yes, while this might save you $50,000 a day here, you know, what's the implication of it on the flip side? Or you know, do you need to then set up Dr. So it becomes more complicated. Like those are the business questions that then have to come into account. So maybe just even pointing out, hey, there are other places to spend it, maybe SQL isn't the best solution, you know, but understanding what's in that SQL database then becomes the important part. Stone was like pointing the big red finger, which should be fairly obvious to a fair ops person of saying, hey, there is this is 30% of your spend. This is an area to focus on. If you take a 5% cut here, that's great, you know, that's going to do a lot more than a 5% cut on 2% of your bill. And I understand that sounds obvious, but you know, so many times people get stuck on the reservations. Are we fully utilized? Sure, we can optimize this and gain an extra 1% of our bill, but there's that 50% over there that you're, that no one's looking at and what can we do over there to really make this thing be more effective? So you need then the business and the, you know, not just the general business but like your specific business and your application team and your everything else to really focus on that aspect of it. And I don't know that necessarily tools going to show you that, but it maybe would help start the conversation internally. [01:33:59] Speaker C: Yeah, I think it's one of those things where it's like if I see another announcement about a compute recommender, I'm going to lose my mind because I know and, and it's AI. AI powered, right. And it's a. [01:34:12] Speaker D: To have 5% less CPU. How is that AI powered? [01:34:15] Speaker A: And. [01:34:15] Speaker C: Exactly. And so it's like, you know, that was the running joke for our, you know, our AI title which was, you know, AI is how ML makes money, which has become less true over time. But the reality is that this still is nothing new. It's going to have the context that it has and the AI is going to make a determination based on whatever context it has. And that context likely will not be what's important to the business because it's very difficult to feed that kind of context without custom models and a lot of tuning and fine tuning. So it's you know, like being able to sort of take action and be that cost saving tool I think is a, is definitely a very good opportunity. And yeah, I'm surprised Cloud Custodian, like there's even a new thing for that because I feel like that's gone away of the dinosaur with all the built in stuff that most of the cloud providers have for managing these things at scale. They didn't used to, right? They used to, yeah. And so it was a problem. But that's a bygone era. [01:35:14] Speaker D: I mean in my opinion the cloud tools, you know, in general, choose a tool, whatever it is, are good to get the system to get you up and running. Hey, I walked into an environment, I have a new job. Cloud costs are the key things. How do we drive down the cloud costs? What's all this stuff? Look, there's always going to be that EBS hard drive, whatever it's called in your cloud, you know, thing sitting there that, yeah, it will save you five or ten bucks. So there's never again, you can never say you're 100% complete, but any decent cloud slash finance person will look and be like, okay, what's our reservation count? What's our savings plan count. Let's start there. Are we doing general scaling? [01:36:01] Speaker C: Cool. [01:36:02] Speaker D: Are our instance types correct? Great. Any blob storage? Are we doing lifecycle policies like these are just 101 that all they did was just take the general things that, you know, consultants, whoever have been doing for years put into a product, sell it. And that's great for the first year or two of your journey. But it's that year three, four and five that Justin was talking about here of like, how do you then re architect your platform to continue to leverage those cost savings? And that's not something to me a tool could straight up do. Maybe it can. And I'll eat my own words, I'm sure in three years when an AI. [01:36:40] Speaker C: Is able to do it, I haven't been able to do it like trying to convince teams after three or four years of arguing with them that they need to re architect to not have the stateful thing burning money in the corner. So. [01:36:52] Speaker D: But that's where you need your FinOps team and your executives to say, guys, this is not. You're costing us $2 million a year, but you're bringing $1 million of revenue in. But we need this product. So figure out how to make it work. And that's the business aspect of it. [01:37:08] Speaker A: But I do think there is an opportunity there. I mean, I think the being able to track the cost of that thing over time, because that's one of the things is you're doing these big transformations. Okay, I'm going to transition 50 petabytes of storage from NetApp to object storage. It's going to take me three years. And then basically people lose interest in that project in six months. They're like, what have you done for me lately? But if you're able to show through this type of tooling that, hey, this was the architecture we came up with. And here's how it's saving money over time. Here's what we're estimating to complete that, that helps with forecasting. There's a bunch of things that can help you do that are also beneficial too. And again, as you get into these tools, it's very common for a vendor to sell you cloud health or cloudability. You get all the recommendations and then three years from now the renewal comes up and you're like, well, what value are we getting out of that tool? [01:37:57] Speaker D: That's what I vote for. [01:37:59] Speaker C: We email a report once a month that no one looks at. And it's just like this. [01:38:04] Speaker D: Especially when they're all 1% of bill 1% of saving, whatever they're, whatever, you know, it's 1, 2, whatever the percentage of your bill is to me they are, hey, use this tool as an analysis. It's a project tool, not a sticky tool. Like you buy this tool, do this project, now this, you're going to build your entire process around tool. [01:38:28] Speaker C: But they're incredibly complex to roll out. It's really difficult to get it deployed everywhere and then build in the context you need. Like in a lot of cases you're doing ETL data sets and you're doing custom dashboarding. Like a lot of coding behind the scenes and queries like they're painful to roll out and to get them to work. Well, because you have to teach these things business context. And I think a tool that could do that much more automatically would be a huge benefit, you know, and I think that's where these finops providers should focus. You know, like, you know, tell me the business context of how your opera, your application operates and then set AI loose on, you know, the different logs and the different things going back and forth in your cloud resources to build that pricing model for, you know, so that you can now it's not, you know, how do you save money? It's like, oh, this is how much we're spending per transaction. This is a per customer. I got cost. And they will forecast that tool like they need to really add more value. [01:39:32] Speaker D: There was a tool that I've talked to at my day job recently that focused a little bit more not on cost savings but on what each customer. So if you have 50,000 customers, you in theory could figure out what each customer was costing you. [01:39:49] Speaker C: Yeah, yeah. [01:39:50] Speaker D: And that to me had a lot more value because your 500 person customer probably costs you different than what your 50,000 person customer versus what your 500,000 person customer is. Hopefully you have some economy and scales in there, but like it was more focused on that aspect and then driving costs in that way versus Pierce your RI solution. And it was an interesting tool still kind of looking at it but, but you know, it needs to my, my day job it would have to work a little bit differently or figure out a few pieces of it. But there's pieces of it that are fairly interesting. The underlying concept, which is what we're saying here, which is you got to have the business context of it is key. And so any of these tools you're going to spend a lot of time building that business context into. So how do you get that business context automatically? [01:40:43] Speaker A: Yeah, well, especially in this, in this Era of profitability and efficiency. Right. Margin now becomes important and you're talking about like, hey, I have 50 petabytes of storage and it's costing me money and I could move it to object storage and save you 90% of the cost. Yeah, it sounds like big numbers and people get excited about it. But if you say hey, it increased the margin of the primary product by X percent, now suddenly people are like, oh actually that's really good. You know, that's something we can tell our investors, that's something we can tie to that is tangible and has business value. Yeah. So getting down to cost, unit economics, that's a big area. So there's definitely a couple of vendors who have focused on that. Cloud Zero is one that comes to mind that I talked to, I was at FinOps X about that exact topic. But again, a lot of these are tied to the FinOps framework. So if you look at the FinOps framework, there's different aspects to the domains. Right. And so one of the domains is unit economics. And so, so what you do see in the differentiator between the dashboard and our acquisitions is that some of them have picked up different boxes inside of the FinOps framework for differentiation. And unit economics is one of those that I've seen a couple times. Anomaly management was a big one a couple years ago, but then everyone got anomaly management. So again they're all racing to beat each other in the FinOps framework. But again, I think there's more interesting use cases like you said in the business side of it and tying it to the business value and the business strategy and how do you get to the bigger part of that? And I think, you know, again the foundation will continue to evolve and they'll continue to adopt more and more things that make sense to it. You know, that's why they do on prem cost management now and they do SaaS cost management because it's again related to costs and these things. But again, I think there's a foundational set of technology built on top of focus that is common place and then everyone needs to be able to play in the innovation space and that's where they need to compete, I think. And that's really my key message to all these guys who are maybe listening from my pot my LinkedIn post, like, you know, you guys had to figure this out because like, you know, I remember last year talking to some vendor and they were showing me this like super complicated JSON configuration YAML file and I was like, just give me AI for that. I don't Want to like learn your freaking, you know, language, like give me an AI that can do that and you know, use those tools, use the tools that are coming out for everybody. Like, you know, you can if with AI tools now you can go look at the source code and you can actually go probably figure out some of these things that, you know, Matt says aren't possible. But I bet it's actually more possible knowing what I can do with AI. You know, I'm like, you know, I can ask it like, hey, what do you recommend for doing this thing? And it'll give you a whole recommendation architecture guideline of like, here's what a migration could take over six months. It'll tell you these things. So that's one of the fun things I've been doing, you know, and talking about certain concepts. Like I'll just go talk to the architect mode and say, hey, let's architect together. And like here's what I'm thinking. I want a system that does this and this and I want to have these fastbacks and then ask me a bunch of questions to make my idea better. Then produces five markdown files that have basically a full architecture, enterprise architecture put together for this thing. So the tooling is coming. And so I think that's the call out to the FinOps vendors is AI is here, take advantage of it and compete there. Stop competing on dashboards or get consumed. I mean the point that Will makes in his article, they're all going to get bought in the next two years because it's going to be massive consolidation and IBM instead of having three vendors on the floor, will have 12 vendors on the floor selling me FedOps tools and dashboards. Oh great. [01:44:06] Speaker C: I feel like that already went through a phase too where all the, all the traditional data center cloud cost reporters bought all the cloud native ones. And now there's the marketplace has shrank down and they try to marry them together. It didn't work. [01:44:21] Speaker A: Yeah, well, I mean it cost management has always been ITBM which apptio created the space and then they bought cloudability. At some point they basically get into cloud cost manager because they weren't in that space. ITBM space was only for the largest of mega enterprises who really could invest in it at that level to really tie back to assets and depreciation and things like. And so that product, which was a good product and I use it in the past, was always interesting to me because it is so reliant on data and there was no standardizing the data. So at least now in the cloud space because again, it's all API driven and they've built this Focus standard and customers are adopting like vendors are adopting the standard. It makes a lot of the date, the basic data hygiene much better. [01:45:07] Speaker C: Yeah, no, I mean, it's night and day and it just allows. It's. It really is game changing. Right. Because you can. Yeah. Now because that's where you're, you know, like the savings on a single cloud and stuff are, you know, there. You can figure that out and you can use native tools and, you know, getting things like Focus lets you run your business and get that visibility across and it's not specific to a certain type of technology. [01:45:30] Speaker A: Well, this was a long one. We gotta let folks get back to their other shows, but we'll see you guys next week here in the Cloud. Have a good one, Matt and Ryan. [01:45:40] Speaker C: Bye, everybody. [01:45:40] Speaker D: Bye, everyone. [01:45:44] Speaker B: And that's all for this week in Cloud. We'd like to thank our sponsor, Archera. Be sure to click the link in our show notes to learn more about their services. While you're at it, head over to our [email protected] where you can subscribe to our newsletter, join our Slack community, send us your feedback and ask any questions you might have. Thanks for listening and we'll catch you on the next episode.