[00:00:07] Speaker A: Welcome to the cloud pod, where the forecast is always cloudy. We talk weekly about all things aWs, GCP, and Azure.
[00:00:14] Speaker B: We are your hosts, Justin, Jonathan, Ryan and Matthew.
[00:00:18] Speaker C: Episode 277, recorded for the week of October 1, 2024. Class Eips. So now you can procrastinate ipv six even longer. Good evening, Ryan and Matt. How you doing?
[00:00:30] Speaker B: Doing good.
[00:00:31] Speaker C: Doing good.
[00:00:32] Speaker D: Doing well on a Monday.
[00:00:33] Speaker C: Yeah, we're technically recording a day earlier than we normally do because I'm out on vacation, and now you guys only have to worry about getting one show in while I'm out, so that'll be a be good. But good luck to you all on your show next week.
[00:00:48] Speaker D: Try to wrangle the cats.
[00:00:50] Speaker C: Yeah, if Matt doesn't get you guys to do it, then I have no hope for Jonathan and Ryan to get this done. So it's all on you, Matt. Zero hope.
[00:00:56] Speaker B: Me and Jonathan have been useless lately.
[00:00:58] Speaker D: All right, I'll start tomorrow for next week's show, and we might have a prayer.
It'll just be messaging. Hey, Tuesday. Hey, guys. Tuesday. Okay, Tuesday.
[00:01:09] Speaker C: How about Tuesday?
[00:01:10] Speaker D: Tuesday.
[00:01:10] Speaker C: Be great to record.
Well, let's get into some news that has happened here. First up, OpenAI chief technology officer Mira Maradi is leaving. She was also the CEO for just a few days when Sam Alden Washington ousted. And then within a couple hours of her resignation, apparently two additional people had resigned from the OpenAI technology leadership team.
Altman. Sam Altman said, it's hard to overstate how much Mira has meant to open AI our mission and to us all. Personally, I feel tremendous gratitude towards her for what she has helped us build and accomplish. But most of all, I feel personally gratitude towards her for her support and love during all the hard times. I'm excited for what she will do next. Mira, during her time as CTO, oversaw the development of chat, JPT, and image generator Dial E. She was also a pretty public face for the company, appearing in its videos and interviewing journalists. And I said pretty, but not like pretty as she looks nice, but isn't like she was the public face. Now, wording the other two departures were Barrett Zoff, who was the company's vice president of research, and chief research officer Bob McGrew, which, when you lose two research officers same time your CTO does, you have to think that they had an idea for a startup that they wanted to do. But, yeah, we'll see where they all end up. But. Or maybe they're all. Maybe they're just like, I don't want to work for that guy. And so they just quit on principle with their chat GPT money. I don't know, I don't know, click.
[00:02:33] Speaker B: It through the paywall. Her reason for leaving is, you know, to take some time and space to explore and you know, be more creative, like, yeah, okay, so they're starting.
[00:02:44] Speaker C: Yeah, that's, that's executive speak for where.
[00:02:47] Speaker B: You quit and yeah, yeah, yeah. Leaving for health reasons, you got fired. You know, just taking some personal time to explore your creativity. Yeah, you've got another idea and you know, more power to them. There's, you know, there's a billion AI company starting up right now.
[00:03:06] Speaker C: Then you get some funding, you can be successful or not beef, they're going.
[00:03:09] Speaker B: To have a lot of advantage.
[00:03:10] Speaker C: Yeah, yeah, I mean there's always so many people who can do foundational models of size too. I mean, I also wonder, you know, she, she's a former Tesla employee. Does she have connections to Elon and what he's doing with exit out AI? And so again, I, I suspect we'll see, you know, her pop up somewhere either with a new startup or maybe potentially stolen away by Elon.
[00:03:32] Speaker B: I don't know, that would be interesting.
[00:03:34] Speaker C: Yeah, I'm just speculating because I don't know, I have no inside baseball on this, but seems there's some connective tissue there. Well, Meta had their developer day last week and they announced Llama 3.2, which includes a small and medium sized vision LLM, eleven b and 90 b, and a lightweight text only model, one b and three b that fits on edge and mobile devices, including pre trained and instructed tuned versions. The one b and three b models support context, length of 128,000 tokens, and create and state of the art in their class of on device use cases like summarization, instruction, following and rewriting tasks. Running locally. At the edge models are enabled on Qualcomm and MediaTek hardware and optimized for ArM processors. The 3.211 b and 90 b vision models are drop and replacements for their text model equivalents while exceeding an image understanding tasks compared to closed models such as cloud three haiku. Unlike other multimodal models, both pre trained and aligned models are available to be fine tuned for custom applications using torch tune and deployed locally using torch chat. In addition, they are launching llama stack distributions, which greatly simplify the way developers work with the llama models in different environments from single node, on premise cloud and on device enabling turnkey rag and tooling enabled applications with integrated safety models are available to you on llama.com and hugging face and the various partner platforms which we will talk about as they come up.
[00:04:55] Speaker B: That's funny, like a true infrastructure nerd. Like ooh, cool model, whatever.
For me, I'm excited about the stack distributions just because it makes using these things a lot easier. I love the idea of having a turnkey rag and being able to create that more dynamically without going too deep into AI and knowing how the sausage is made. Then the fact that they're making models small enough to fit on edge and mobile devices is great.
[00:05:27] Speaker D: Thats part I thought was really cool was like youre here, Apple and Google, everyone say theyre running but now these generalized models that can run on any edge device. So now im thinking more people embedding them into apps in the stores and being able to target different arm processors to get the very fine tuned speed that youre going to need out of a mobile device without killing the cpu and battery life of them. So it's going to be interesting to see how that aspect now evolves for all the edge computing devices as they have to be very small to run there, very efficient but also very accurate. Otherwise people are going to just throw them out.
[00:06:06] Speaker C: It's funny in the llama stack, which I also was most interested in too, because I'm an infrastructure nerd.
I cringe though, when I see agentic apps. I'm like whenever you decide that's going to be the name for these things.
But it is kind of interesting the way that they compartmentalize basically the data, the models, the hardware components into the stack, and then they give you a model toolchain API and they're giving you an agentic system API that your end applications can leverage. And those use cases are important. And making it easy to do that on a mobile device is going to be really important for performance reasons and for how do you want to get them on the box and how do you want to provide the data and how do you manage privacy, which is a big thing you're looking for in the prompt world. So overall, yeah, glad to see Lambda 3.2. I have not downloaded it yet to play with it, but I look forward to doing it.
And first up, databricks now supports Metalama 3.2. Amazing, amazing.
[00:07:05] Speaker B: They've been first to the last couple, right?
[00:07:09] Speaker C: They all dropped their pressure dances on the same day after this gets announced. So I just happens to be there the first section because we covered databricks and snowflake and AI is going great segment of the show. That's why they're first on the list, not because of any other order of operations that occurred.
So don't, don't get that excited that they reversed.
Moving on to AWS this week. Last week we talked about the new c eight g instances, which are the graviton four powered compute optimized instances. But they also announced Graviton four powered m eight g instances a day after we cut off the show notes. So appreciate that. But these come with even more cpu and memory than what we talked about last week. The M eight gncs can have up to 192 vcpus with 768 gigs of memory and 50 gigabits of network bandwidth and 40 gigabits of bandwidth for EBS adjust. Graviton four processors offer enhanced security with always on encryption, dedicated caches for every VCPU, and support for pointer authentication.
[00:08:05] Speaker D: Does that mean there's a 50 gig, 250 gigabit cards in these boxes? And so 240 gigabit Ethernet ports and then combination of five or ten, there's.
[00:08:17] Speaker C: 250 gigabits because I assume they're redundant, and there's 240gb because they're redundant. So I think.
[00:08:25] Speaker D: It goes ten and then 40.
[00:08:27] Speaker C: So according to the press release, it's a 50 gigabits of network throughput for front end applications going through your.
[00:08:33] Speaker D: Oh no, I was thinking more of like, just physically what's in there, because.
[00:08:36] Speaker C: I know there's 40 gigabitous, a 50 gigabit network card. Now I don't know anything about that. Like this is a level of infrastructure I wish I still played in, but I also don't want to still play in because that's what I have clapped writers for.
[00:08:47] Speaker B: Yeah, I don't know why you guys are more concerned about the headline, because I was like, what is a sustainable workload when you're talking about 192 VCPU and all the gobs of memory and you go through the entire blog post, they don't mention it. They don't mention anything about the power or the CO2, anything. And so you're just less to assume that because it's graviton, it's more energy efficient. But I, I am claiming clickbait. I call both.
[00:09:13] Speaker C: Well, I mean, if you put 192 intel vcpus into a box versus these graviton, I'm pretty sure that intel box is going to use way more power.
[00:09:22] Speaker B: Sure, sure.
[00:09:23] Speaker C: So that's why it's more sustainable.
Hey, someone had a quota of how many sustainable articles you need to put it in the title and they check the quota, all right, that's how it works.
[00:09:34] Speaker D: I'm also just impressed that I read the entire article looking for that use.
[00:09:37] Speaker B: Of information and I was so angry when I got to the end.
[00:09:41] Speaker C: I like that you don't have time though, tonight that you actually did the show notes like the homework. That's amazing.
[00:09:46] Speaker B: It's amazing what happens when I show up on time.
[00:09:48] Speaker C: Yeah. So weird.
[00:09:49] Speaker D: It was going to go much, much higher today.
[00:09:52] Speaker C: Yeah. I mean, the listeners are going to have this new expectation of your insights that you're.
Good luck to you on that.
[00:10:00] Speaker B: Yeah, that could go both ways for me.
[00:10:03] Speaker C: Let's see, Amazon bedrock now supports, you guessed it, llama 3.2.
[00:10:09] Speaker D: Wow.
[00:10:10] Speaker B: Woo.
[00:10:12] Speaker C: Well, if you are a big fan of AWS Appmesh, I'm sorry to tell you that effective September 30, 2026, Amazon will be killing it. Until this date, AWS AppMesh customers will be able to use the services as normal, including creating new resources and onboarding new accounts via the AWS CLI and AWS cloudformation. However, if you're a new customer who never uses service, you are already out of luck, because even before they published this article, they had stopped taking you onboarding you. As of September 24, 2024, no new customers may access AWS Appmesh. If you don't know how to move from Appmesh to ECS service Connect, this blog post will walk you through all of the steps, including the differences between the two solutions, how to migrate to the new solution, and all the ways that these things should be done in the future. Thank you, Amazon for writing a thorough blog post detailing how to get this done versus just silently canceling a service in the community post. I appreciate that.
[00:11:05] Speaker B: Yeah, I understand this announcement. I'm a little bummed though, because that mesh was not as tightly coupled to the container ecosystem, and so it gave, I mean, when it came out, I had this, you know, grand designs of having creating mesh networks with multiple workloads, whether it be containerized workloads or on prem workloads or vm based workloads.
It really gave the promise of that because it was outside the ecosystem versus the ECS service. Connect is very much in the container ecosystem.
It's still envoy, but it's just running in a sidecar. Cluster routing of traffic, which is still great, but those things, it gets difficult to integrate meshes across clusters, and it becomes a little bit more overhead when you're trying to orchestrate at a organizational layer versus just a series of services running on a single cluster.
[00:12:09] Speaker C: Yeah, I hadn't experienced either one of these, but mostly cause I've moved on from Amazon world before we got to meshes. But I do like that they've also eliminated some of the confusion because how do you choose between the two of these? What are your big decision points? And the problem with Amazon and all the choices sometimes is that you get overly complicated decision making trying to pick these different things. And so I appreciate that they have given clear answer now, which I do appreciate, even though they are deprecating services, which is kind of a bummer.
[00:12:46] Speaker D: I mean, I do think it's good that they're deprecating, like you said, kind of trying to make life easier for people on the same side. You know, they ruined Ryan's dream of this magical world where everything is in one location, you know, but they are kind of forcing people down what I hope is better patterns. So you're not getting people that do crazy things like Ryan just talked about, and then they get all mad at you when, you know, when you get mad at Amazon, when they don't actually work for you. So, yeah, it has its pros and cons of just seeing how many times I can make fun of Ryan in one sentence. So I think it did well.
[00:13:17] Speaker B: I mean, I'd have to start my own podcast to go on and on about service mesh and the way I feel about it, you know, like, I just really wish get it off the ground anywhere.
[00:13:31] Speaker D: It's a complicated concept that everyone wants to just do. And it's, you know, to really do a proper service mess, you need the app teams, infrastructure team, you need everyone to really be working together to make it work. And when it works, it's great. Debugging a service mesh can take some effort.
[00:13:52] Speaker B: It does, yeah, no, that, that is, that is the rough edge. Definitely on service mesh.
[00:13:57] Speaker D: That was my nice way. I'll say.
[00:14:00] Speaker C: Well, so I feel Appmesh is a little bit of a big deal because it was kind of the first solution they had, and I imagine some customers adopted it. I've been very happy using it.
This deprecation feels a little bit more like a yawn to me personally. So this is the Amazon FX file gateway is being deprecated, and you're basically being asked to move to Amazon FSX for Windows File Server. On October 28, 2024, new customers will no longer be able to deploy Amazon FSX file gateways. So if you need one, you have just under 30 days to go get one. FSX File Gateways is a type of AWS storage gateway with local caching designed to deploy on premises. FX file Gateway optimizes on premise access to fully managed file shares in Amazon FSX for Windows File Server. Basically, Amazon says with the drop in bandwidth costs and increasing availability, many clients can access FSX for Windows File server in the cloud from their on premise location without the need for a gateway or local cache. And for those of you who still need local cache, you can pay way more money to get Amazon FSX for Netapp on tap using flex cache or global file cache, which can serve your needs better. This one, I don't think a lot of people are really using this gateway. I don't think a lot of people use the Amazon s three gateways either. So I get why this one is going the way of the dodo. But using NetApp on tap as your alternative for caching locally is a bummer because that definitely has some costs for you. But overall this one doesn't feel quite as disruptive as Appmesh did. But what do you guys think?
[00:15:28] Speaker D: It's more interesting that this is the first one they decided to kill off, not the other services that been around. Because years ago when they first had all the storage gateway, there was like the three types they had, and obviously they had the fourth, but like they didn't kill off any of the s three ones that were related. If you're talking about things that network latency and everything else for blob storage is meant to kind of handle that. Where Samba shares, Sif shares don't really handle that latency as well. So it's interesting they chose kill this one off first, but I also assume one that probably had the least customers.
[00:16:03] Speaker B: Yeah, I was gonna say the dozens of customers are really upset about this one. Yeah, they don't. Yeah, I mean, there's a lot of different reasons why. I think that probably adoption of this was low and, you know, like, and I kind of agree, like in modern, modern practices, you don't have such a strong reliance on that, on having that edge cache.
You're still only releasing the file handle after the complete write. And so caching can be better, but only if in certain circumstances. And I feel like a lot of these workloads are going to be. The storage is going to be attracted to a net app cluster. So that's already going to be there.
[00:16:44] Speaker C: Well, even your Netapp, if you had a flex cache, you don't have to have any disk locally, you just have to have, basically, I think even a virtual appliance you can run now to do that.
It's probably not a huge cost. It's probably Amazon looked at it and said look, Flexcache exists. Netapp on tap exists. You can do that path if you want to, if you really need a caching option. It's interesting to think that the Internet's fast enough though to give me access because sifs over the Internet is not fast.
I don't know about that part so much, but overall, yeah. If you're really serious about doing sort of a gateway caching, I think the s three one is the better way to go anyways. That way you're converting from file store to object, unless you have reasons why that doesn't work for your use case. I think that's a better solution anyways.
All right. Moving to GCP. Google has filed an antitrust complaint against Microsoft within the European Commission. The move has to do with Windows Server, of course, for Google, a set of licensing terms that Microsoft applied to the OS in 2019 harmed competition and raised costs for its customers. Under the revised use of terms, customers must pay additional fees if they wish to move their Windows server licenses from Azure to rival platforms such as Google Cloud. Google claimed that this can result in a 400% increase to run Windows on rival clouds. And Google wasn't done just complaining about Windows. They also complained that companies that run Windows servers on third party cloud offers get limited access to security patches compared to Azure users. And the search giant argues that there are other interoperability barriers. This complaint comes two years after CisSP, which is a conglomerate of small european cloud providers, filed a similar complaint. They withdrew it after reaching an agreement with Microsoft. So I assume Google is hoping to force Microsoft's hand to sign a similar type of agreement.
[00:18:26] Speaker B: Yep. Yeah, I don't think they're concerned about the antitrust ability of this at all. And you know the other way to the Microsoft press releases for this have been worded very differently in the sense of it's features built into the Azure workloads. And so it's like while you say that they're not granting the ability to Windows servers to get security patches onto their clouds, the reality is it's only because they have the workloads running in Azure that they can offer the enhanced security patches. Or at least I presume that I guess, I don't know. But yeah, and then, you know, the Windows licensing, like it's, it's a, it's a service. Your licensing fees are built into using the service.
So. Yeah, yep. I mean competitive advantage, like use it?
[00:19:17] Speaker C: Yeah, well you can't even bring Microsoft licensing to, you know, to the cloud on Google. I mean, maybe you used to be able to, which is why then this being able to move licensing from Azure to Google make sense. But yeah, it's sort of a limited number of customers who are allowed to even do that before. I think Microsoft really kicked everyone down in 2019 and said you can't do that anymore because I think even Amazon used to be able to do that to some extent and that stopped as well.
[00:19:42] Speaker D: You definitely used to be able to, yeah.
[00:19:45] Speaker C: All right, well, Bigquery vector search is now generally available, setting the stage for a new class of AI powered analytics, says Google. This functionally also commonly referred to as approximate nearest neighbor search, is the key to empowering numerous new data and AI use cases such as semantic search, similarity detection and retrieval, augmented generation or Rag, the large language model initially announced back in February, bigquery vector search integrates generation, management and search of embeddings of the data platform to provide a serverless and integrated vector and analytics solution for use cases such as anomaly detection, multimodal searching and product recommendations, drug discovery and much more. In addition, IVF, or inverted file index for bigquery vector search is also generally available. This index uses a k means algorithm to cluster the vector data and combine it with an inverted road locator and a two piece index in order to efficiently search similar embeddings representing your data. IVF includes several new enhancements including improved scalability, manage index of guaranteed correctness, stored columns and pre filters. And don't tell the GOP it may get banned.
[00:20:48] Speaker B: Yeah, and watch your bill like a hawk if you're using this, because like, I haven't used the vector search aspect of it. But Bigquery is nothing cheat either to host data or to query data. So it's definitely something you want to use intelligently and not just sort of turn it on and just use it in lieu of doing, you know, a vector search on a different type or more, even more dedicated sort of database or data database engine.
I think it's a really powerful option and it's cool, but I think this is going to be cost prohibitive for a lot of use cases.
[00:21:26] Speaker D: I mean, I feel like a lot of these new features and technologies are extremely cost prohibitive for a small business to use.
You know, this definitely feels like one of them. Azure's definitely released some of them, AWS definitely released some of them where you're like, the barrier to entry is so high that some companies still aren't even willing to riot, which to me kind of defeats a little bit of the purpose of the cloud where the barrier entries pay as you go should be low enough. But now I know enough about pricing for this.
[00:21:57] Speaker C: I mean, I. So my experience so far with costing of AI things is that it's not as expensive as people fear it is. If you're building a foundational model, 100% it's expensive. You need lots of Nvidia GPU's, you know, that kind of stuff. But you know, if you're using like inference nodes and you're doing, you know, you're using an LLM to respond or using rag to augment it isn't as expensive as you might think it is to do those things, at least at some scale, not as much as you might fear, unless you have super crazy high volume of AI calls, if you're just trying to plug it in as a feature of an existing application and you have to. Jira, for example, has an AI capability where you can basically ask it free form questions and it'll generate you JQL code. That type of use case, that's not going to cost you much at all for the API call. But people have been conditioned that everything AI is super expensive.
Everyone's a little bit nervous if you're already running bigquery, I don't know that this use case will increase that much of your cost of bigquery. Now, if you're not running bigquery, yes, this is a very expensive use case.
I think it really depends on what's your maturity of bigquery on its own and if this is just something you're adding to it, I don't know, that adds a ton. But if it's new to you completely. Yeah. Cost of bigquery alone can be very pricey.
[00:23:17] Speaker B: Yeah, yeah.
[00:23:18] Speaker D: I was talking about the general platform of like bigquery, your redshift, like these large, expensive services that have large potential. Just the hard barrier to entry sometimes to swallow.
[00:23:31] Speaker C: Yeah.
[00:23:32] Speaker B: And the pricing models are built for a different type of workload.
[00:23:34] Speaker C: Right.
[00:23:35] Speaker B: It's not supposed to be a queryable data store. It's a large sort of reporting engine, if you will, where you have more or less dynamic queries hitting the workload.
I think it's neat, but we'll see, because I do love using bigquery. It's just not cheap.
[00:23:59] Speaker C: Yeah, bigquery on its own is an expensive tool, very powerful tool, but expensive.
And so, yeah, be careful with bigquery. My hesitation, but if you're already using it and you're using at scale, adding rag to it, I don't think it's a huge cost.
All right, Google Cloud is summarizing a busy month of announcements here in September for Google Cloud databases. Many of these we did talk about, but a couple of them we did not. First of all, Oracle databases, GA and Google talked about it. Check it out in a prior episode. New Spanner edition is now generally available across standard enterprise Enterprise. Plus we did talk about that as well. Cloud SQL has three new features that improve the cloud SQL, enterprise postgres and MySQL capabilities, including addition upgrades, MySQL miner version enhancements, and a zonal standalone instance that has no redundancy. You don't recall us talking about those three specifically? Mostly because I thought it was boring.
AlloyDB now supports postgres SQL 16 or in preview. We did not talk about that on the show. Node level metrics on memory store for Redis customers and val key support for node level metrics. That's great. Observability improvement and k and n vector searches for Firestore is only available. We definitely did not talk about that because I typically never talk about Firestore if I don't have to. But a lot of the time we covered a bunch of these stories. So hopefully you checked out those last month. But if not, you can check out these articles in this link in the show notes and get to the deep, deepest technical document from Google directly so you can read up on those things if you missed our show or just go back and listen to the prior episodes. That's my recommendation.
[00:25:29] Speaker B: Yeah, I like that zonal standalone instance because I'm cheap and I just, I build backup and doctor, so like if you know, and typically I don't have high uptime requirements, so I have time to rehydrate when I need.
It's cool. I like it.
[00:25:48] Speaker A: There are a lot of cloud cost management tools out there, but only Archero provides cloud commitment insurance. It sounds fancy, but it's really simple. Archero gives you the cost savings of a one or three year AWS savings plan with a commitment as short as 30 days. If you don't use all the cloud resources you've committed to, they will literally put the money back in your bank account to cover the difference. Other cost management tools may say they offer commitment insurance, but remember to ask, will you actually give me my money back? Archer will click the link in the show notes to check them out on the AWS marketplace.
[00:26:28] Speaker C: Well, if you struggle with making prompts for AI, which sometimes I have, I've gotten better. I think it's definitely something I'm learning. I think maybe a year ago, we're all learning. I complained about how hard it was to change my brain from doing Google search box to doing prompts. I've come a long way in a year, so I'm very proud of myself. But if you're looking for help announcing the public preview of Vertex AI prompt Optimizer and Google says prompt design and engineering stands out as one of the most approachable methods to driving meaningful output from an LLM. However, prompting large language models can feel like navigating a complex maze, and you must experiment with various combinations of instructions and examples to achieve the desired output. Taking a prompt, moving it from one LLM to another, say Gemini to chat, is challenging because different language models behave very differently from each other. Simply reusing a prompt is ineffective, although that's how I always test it. So users need an intelligent prompt optimizer to generate useful outputs. To help solve this problem, Google is announcing a Vertex AI prompt optimizer in public preview. The prompt optimizers make it easy to optimize, handles versatile tasks, and expanded support for multimodal tasks. The comprehensive evaluations and flexible and customizable outputs of built for data driven optimization and built for Gemini up first. So basically you're getting the ability to take a prompt, put it in the Vertex AI prompt optimizer, and it'll rewrite the prompt and test it against your Gemini models in different ways to see which ones come up with best outcomes to help you tune and provide the best response.
[00:27:58] Speaker B: I feel like I'm ahead of my time because I have not retrained my brain, but what I have learned to do is just ask aihdeme how I should ask it.
I feel like this is basically just service by my normal use case, which is like, hey, I want to do a thing. How do I ask you to do a thing? And then it asks itself much better than I would have.
[00:28:20] Speaker C: Yeah, well, it's actually one of the fun things to do is like, or you give it a prompt and you ask it how to make this prompt better. Or you say like, hey, I'm trying to make a meme image based on this thing I've seen. And then like help me do. And you say help me do that. And then it'll ask you a bunch of follow up questions and then basically produce at the end the statement and like, okay, cool, draw that for me or create it and it'll generate that image. And then I post it into our slack and we laugh at it. It's either hilariously bad or it's awesome one of the two.
[00:28:48] Speaker B: Yeah, yeah, it's cool. I mean, it's interesting. It's the like, I also will copy and paste the same prompt into multiple models just to see the differences. And so it's interesting to me that they're trying, there's a use case where they're trying to craft the prompt different per model to get the same results, I guess. I don't know, that part confuses me.
[00:29:10] Speaker C: But yeah, I don't. I don't know if you're trying to get the same result or trying to, but again, like, asking the model differently, I guess. And Jonathan probably has a good, you know, with all his playing with LMS he's doing, he probably could tell us why that's important, but he's not here, so save it for next week. You can ask him.
All right, if you're going from millions to billions, you can now announce leverage vector search and memory store for Valkey and redis cluster basically, vector search on both the memory store for Valkyrie memorystore and Redis clusters combines ultra low latency in memory vector search with zero downtime scalability and high performance vector search across millions or billions of vectors currently in preview vector supports for these memory store offerings means you can now scale out your cluster by scaling out to 250 shards, storing billions of vectors in a single instance. Vector search with redis can produce single millisecond latency on over a billion vectors with a greater than 99% recall.
I don't know if I would say that Redis or Valkyrie is zero downtime, but sure, okay.
As long as you store the data in the database and you didn't do the temporary caching version of the data in the database, or you don't have to hydrate your redis cluster on startup. Sure, okay, I can see it.
[00:30:28] Speaker B: But again, it's all say zero downtime, or did it say high resiliency?
[00:30:32] Speaker C: It says zero downtime scalability.
[00:30:33] Speaker D: I believe it's probably just adding it because that's what Microsoft has, where you can add notes to the cluster and it just doesn't lose zero downtime as they just add for adding to it versus, you know, reorganizing, resharding, or anything else along the lines.
[00:30:51] Speaker C: Yeah, so I did to confirm, because I do sometimes translate things differently when I read the article and put in the show note, zero downtime scalability in or out for Valkyrie.
[00:31:04] Speaker D: In would be interesting, because if you're literally getting rid of data that's cached, then you're losing.
[00:31:08] Speaker C: Data or you don't need it anymore because it was temporary.
[00:31:12] Speaker B: I mean, you're just evicting shards from nodes, right?
[00:31:16] Speaker D: Yeah.
[00:31:18] Speaker C: So yeah, this is cool. I could see this being again, talking about that bigquery thing earlier. If you put redis cache in front of it to do this all in memory, you could get a lot of performance out of something like this.
Using value as a vector search cache in front of your data, I can see being a pretty common pattern in the future.
[00:31:38] Speaker B: Yes, this would be what I would recommend, because you don't need to do the vector search directly on bigquery, but if you have your data set in bigquery and populate register that and then you can query the vector, that's pretty awesome. That'd be super quick.
[00:31:55] Speaker C: All right, so this one took me some research because I didn't know this existed until this article, which apparently I missed out on a memo because Ryan was like we talked about a prior company of ours.
So basically this article is about leveraging class e ipv four addresses to mitigate ipv four exhaustion issues in Kubernetes as most of you probably know, we are rapidly running out ipv four space, and the number of applications and services hosted on Google Kubernetes engine or any Kubernetes engine continues to grow, consuming even more private IPV four address space. For many large organizations, the RFC 1918 address space is becoming increasingly scarce, leading to the IP address exhaustion challenges that impact their application at scale. IPV six, of course, solves this attack issue by providing more addresses, but not enterprises or apps are ready for IPV six. Bringing class E IPV four address space, which is two 40 dot zero dot zero dot zero four can address the challenge as you continue to grow, class E addresses are reserved for future use, as noted in RFC 5735 and RFC 11,112. However, that doesn't mean you can't use them today in certain circumstances. This blog post goes into the details of how to do this, which I find pretty interesting. Some of the common objections or misconceptions about using classy addresses classy addresses do not work with Google services. This is false. Google Cloud VPC includes classy addresses as part of its valid address ranges for IPV four. Further, many Google managed services can be accessed using private config methods with class C addresses. Using classy addresses limits communications with services outside of Google. Misleading is the answer, given that the classy addresses are non routable and not advertised over the Internet or outside of the Google cloud. You can use a nat or IP masquerading to translate classy addresses to public or private IP, four addresses, as well as a load balancer to bring the traffic back in.
Class C addresses have performance scale limitations, they say. This is not true. There's no performance difference for class C versus any other type of things that go through matting, IP masquerading, etcetera. And so while the class E addresses are reserved for future use not routable over the Internet and should not be advertised over the public Internet, you can use them for private use with Google Cloud VPC, both compute engine instances and Kubernetes pod services and GKE. To give you some idea of the benefits of this, the RFC 1918 address space has 17.9 million addresses and the class E has 268.4 million addresses. Gives you lots of scalability and growth, efficient resource utilization, and future proofing. There are some sharp edges, though. Not all oses will support class C addressing, and network equipment and software such as routers and firewalls need to be able to support classy addresses properly. And transitioning from an RFC 1918 to class C requires careful planning and execution, as any ip migration should and will require.
[00:34:37] Speaker B: When it says not all os is it looking at, it's probably windows.
I've used this for years in Linux workloads and had no issue. I didn't even realize that it was different from RFC name 1918, I just thought it was another range.
Cool. I mean these have worked, I know, in AWS network layers for forever. I'm glad to hear they work in Google.
I love having a ginormous ip space to just give to developer teams who don't know what their requirements are. I'm like here, if you ever want to talk out of here, it's going to get complicated.
[00:35:15] Speaker C: They just let them go.
[00:35:17] Speaker D: 268 million IP addresses for even the largest companies in the world feel successive, like I feel like you're Joey slipping rogue if you need that many IP addresses.
[00:35:33] Speaker B: I mean, it's more about just having a room to grow because it's and people don't like planning. You know, used to give, you know, 200 ips for a single microservice and sometimes that was an abuse because they'd use eight and sometimes they're like no, we need more than this. But we didn't factor it in because our application architecture needs to be contiguous. You're screwed.
[00:35:57] Speaker C: I did do a quick Google search. Does Windows support classy addresses? And no, it does not. Yep, windows blocks classy addresses and doesn't allow them to be assigned to a NIC through DHCP. Apparently though you can set one up in Azure as your VPC virtual network. But they say it will not work for your Windows boxes and it may have compatibility issues with your Linux boxes which. Yeah, cool, cool, cool.
[00:36:18] Speaker B: Yeah.
[00:36:19] Speaker D: What's funny is it says AWS has unofficially been using it but doesn't support it.
[00:36:25] Speaker C: Yeah, I imagine this is a way that you solve a lot of problems with serverless technology if you're Amazon.
Yeah, I mean, you know, the reality is that the reason why a lot of companies run out of IP addresses in the RFC 19 space because they did dumb things with them. So like, hey, this office is never going to have more than 300 people, let's give it a slash 16.
They just allocated huge blocks of IP space to places that didn't need it. And then because they thought oh, we're never going to be that big. And then something comes in like kubernetes and like oh well, we only had a class c available to give it. Well, that's not enough for that use case, so. Yeah, and then the contiguous problems comes into play, as Ryan pointed out, et cetera. But I mean, the work to go to class e versus ipv six, I don't, you know, if you're going to do all that work to plan it. Like what are those apps? Mainframes, I think mainframes probably the only thing left really that doesn't support IPV six. I mean, almost everything else is at least trying to some, at least at the os level. Maybe not the app side, maybe the.
[00:37:25] Speaker B: App side I think is still complex, but I mean, I think it's all abstracted away in karate.
[00:37:31] Speaker C: Yeah. So I, you know, old network guy never use classy, because back in my day we had, you know, VMware was our most IP addresses we could possibly use and even then, you know, we weren't. They were close to user RFC 1918. But yeah, new problems, new era.
Guess what? Llama 302 available in Google Cloud.
[00:37:52] Speaker B: No way.
[00:37:53] Speaker C: Yeah, amazing, right?
[00:37:54] Speaker D: Shocking.
[00:37:55] Speaker C: Shock. Shock. They do want you to point out that you can use it to craft intelligent agents, which no one else talked about, and you can deploy without overhead, which I don't know what that means, but sure. Okay. And it operates inside your enterprise guardrails, which means it fits inside of the Vertex AI model garden VSP policy. So you're welcome.
Thanks for that.
[00:38:14] Speaker B: I mean, the agent is really just a Vertex AI feature, which is super cool. And it's, you know, it's a, a long way to making an AI interface for the novice user. So it's pretty cool. Even I did it.
[00:38:32] Speaker C: It's chatbot DM's or SQL server databases is now generally available. Data misgradations are of course often challenging and require scarce expertise and lack automation of the modern era. Database migration service has a unique approach to SQL server database migrations, allowing you to do minimal downtime and system overhead. Serverless simplicity. You don't have to install any pesky agents in the middle security at the design with encryption and no additional charges for you to leverage the Google database migration service.
[00:39:01] Speaker B: Yeah, they're just going to charge you for the ginormous cloud SQL instance all the time.
[00:39:06] Speaker C: They're going to get their money anyways. Yeah, exactly.
[00:39:09] Speaker B: I like the service. I really just wish it would work server to server in the cloud because then I could use it.
[00:39:16] Speaker C: But technically why couldn't it work server to server in the cloud?
[00:39:20] Speaker B: It just, it doesn't because they restricted it so that you have to define your endpoint as a cloud SQL box.
[00:39:28] Speaker C: That's silly.
I guess if you're going cloud SQL to cloud SQL.
[00:39:32] Speaker B: Yeah, no, I mean because that's like I want something magical for database server to server. Yeah, but uh. Cause, but it's, you know, it's a difficult space because you know, it's built into Microsoft SQL Server itself and dbas like to use that kind of tool, but I want something a little bit more dynamic.
[00:39:53] Speaker C: Yeah, it's interesting because the Amazon doesn't have that limitation. You can use it for going to EC two instance database servers or non EC two. You just basically connect it servicely as well and then it just does it. So.
[00:40:03] Speaker B: Yep. I guess how your, this ex AWS fanboy learned the limitation of this one.
[00:40:12] Speaker C: All right, let's move to Azure developer insights building resilient end to end security. I'm only highlighting this because it's a new series that apparently Azure blog is starting on their end to end approach to security. The purpose of the series is to highlight how Microsoft Security is transforming security platforms with practical end to end security solutions for developers. This one's a little bit fluffy. It's a fluffy overview in the first of the series, but I will keep an eye out for these, see if they get more meat. We had a really good one back on resilience. If you remember we talked about the tire guard service they built and a bunch of things they were doing to make azure more reliable and automated. So I'm hoping we're going to say a little depth. But if you're not familiar with the platform approach to security, then maybe this first one is a good article to check out as it does talk about the platform concepts of security.
[00:40:57] Speaker B: This is just content for when they're doing the apology tour for all of the year's security, things were like, but, but, but we've got all these things on our secure infrastructure. It's end.
Yay.
[00:41:11] Speaker D: I think it's a good start to try to get people to think about security. Day one, there's so many people think about security, they were ready to go to production. Oh wait, this thing has to be.com gDpr, whatever it is, you know. So I feel like it's a good way to try to get developers to think security at the beginning versus security at the end. If I have to say shift left, I might vomit a little bit.
[00:41:35] Speaker B: Yeah, yeah.
[00:41:36] Speaker C: But you don't like shift left.
[00:41:37] Speaker B: I don't understand.
[00:41:39] Speaker C: It is overdue at this point.
[00:41:41] Speaker D: Yeah, somebody tried to tell me, I think it was like a year ago. There's this new term called shift left. I was like, wait, what? You know, I couldn't understand him at first. It took me like three tries to figure out that he was like actually being serious. Like, yeah, I just heard about this new terminal. It's like, what are you talking about?
[00:41:58] Speaker C: Crazy. They just heard about it. But I mean, things morph too. Like shift left is really build it. You run it, which has been around for a while and then that came out of DevOps practices, which is all about getting developers and operations close. If you didn't abuse the process, you actually shifted left. But most people didn't do that. They just created a team on and on and on.
[00:42:19] Speaker D: That's the after show conversation.
[00:42:23] Speaker C: All right, for those of you who are paying VMware's extortionist pricing and getting a VCF cloud foundation bundles from Broadcom, you can now port those descriptions to Microsoft Azure's vmware solution. AV's any fast and easy way to seamlessly migrate vmware based workloads to Azure using familiar vmware tools and skills. If you don't have a VCF subscription but want to take advantage of VCF and AV's, you can buy your solution directly from Microsoft, who'd be happy to charge you lots of money for that service.
This may be a benefit for you if it includes the fully managed and maintained cloud and vmware infrastructure, as well as you get access to things like VSphere, VSAN, NSX, and HCX, as well as VCF operations and VCF automation, formerly known as the Aria suite. You also get extended security updates for Windows Server 2012 and SQL Server 2012 and 2014. For those of you who are stuck in the legacy, great way to get extended support and move your VMware workload out of your data center all at the same time.
[00:43:17] Speaker B: So Google makes it illegal in the EU and they have to retroactively do that, right?
[00:43:26] Speaker D: How would you spend money quickly on your legacy hardware? Legacy infrastructure?
[00:43:30] Speaker C: Oh yeah, keep paying that payday lender called vmware in three easy steps.
All right, and then our last Microsoft article today is Microsoft trustworthy AI? Unlocking human potential starts with Trust Microsoft is focused on helping customers use and build AI that is trustworthy, meaning that is secure, safe and private security is a top priority, and their expanded, secure future initiatives underscores the company's commitment and responsibility to making sure customers are even more secure to hand security with AI. They are launching evaluations in Azure AI studio to support proactive risk assessments, and Microsoft 365 Copilot will provide transparency into web queries to help admins and users better understand how web search enhances the code pilot response. In terms of safety, they have several new features to ensure that the AI is safe and several new capabilities to mitigate the risk of it not being safe correction capabilities and Azure AI content safety groundedness detection feature that helps fix hallucination issues in real time before users even see the hallucination. Embedded content safety allows customers to embed Azure AI content safety on devices. This is important for on device scenarios where connectivity could be unavailable or intermittent. New evaluations in the Azure AI studio to help customers assess the quality and relevancy of outputs and how often their AI applications output. Protected material and protected material detection for code is now in preview in Azure AI content safety to help detect preexisting content and code. This feature helps developers explore public source code in GitHub repos, fostering collaboration and transparency while enabling more informed coding decisions. And on the privacy side, they're giving you confidential interference and preview and Azure OpenAI service whisper model so customers can develop generative AI applications support verifiable end to end privacy general availability of confident vms with the Nvidia H 110 SR core GPU and the Azure open data zone for which it says privacy for sure, open data zone for the EU and US are coming soon and build on existing data residency provided by Azure OpenAI service by making it easier to manage data processing and storage of generative AI apps. This new functionality offers customers the flexibility of scaling generative AI applications across all Azure regions. The geography while giving them control over data processing and storage within the EU or the US.
[00:45:38] Speaker B: That's an interesting wrinkle that I hadn't thought of before.
You know, the computation of these AI models and having that all be within specific regions for I guess GDPR reasons.
[00:45:52] Speaker C: Yeah. So I'm going to use the globe to process it then as long as I store it back in the EU I'm okay.
[00:45:56] Speaker B: Yeah.
[00:45:57] Speaker C: I don't know, I don't know how the GDPR will feel about that.
[00:46:00] Speaker B: Yeah.
[00:46:01] Speaker D: Not, well, let me tell you.
[00:46:03] Speaker C: Yeah.
[00:46:03] Speaker D: The german work Council also will not feel very happy about that.
[00:46:07] Speaker C: Yeah, they would review that, you know, since they're now the micro, they're trustworthy AI, you know, or the AI.
[00:46:13] Speaker D: They said their trust.
[00:46:15] Speaker B: Yeah.
[00:46:16] Speaker C: I appreciate these things though. I mean like I appreciate you avoiding hallucinations. I appreciate you providing more tooling to help me detect them. I appreciate the embedded content safety. Confidential inference is just the expansion of confidential VM. So you're giving me encrypted data for OpenAI. Cool, cool. I preach all these things. Just, you know, the buzz or the marketing speak is a bit thick.
[00:46:37] Speaker B: It's a, it's a bit thick, yeah. I think the evaluation is going to be really cool in the studio just because it's so, you know, it's just so upfront and prevalent to the end users who are playing with these things. And so I think that'll, that's a good place to put it. So I like developer feedback, although you know, like it has to, it still has to be used, it still has to be like looked at. But you know, I think these are good things.
[00:47:01] Speaker C: Yep, I got Oracle stories. Even though we had so much oracle content last week, I still have more. You missed out on Oracle open world recap last week.
[00:47:09] Speaker B: No I didn't.
[00:47:10] Speaker D: You didn't miss much.
[00:47:13] Speaker C: Although we did, we did talk about a lot of hardware. So like as a cloud infrastructure guy, like you probably would have jazzed out of it. I was kind of excited, like yes, there were some AI things but there are a lot of actual hardware there.
And then Matt pointed out because they're so woefully behind, but I still appreciated it.
Details? Yeah, details. First up, Oracle is launching Java 23.
I'm still not really sure how we got from eight to 23 because in my mind Oracle eight is still the most cutting edge thing out there, but apparently I'm out of touch with Java. Java 23 is supported by the recent general availability of Java management service nine, or JMS nine on OCI native service that provides a unified console to help organizations manage Java runtimes and applications on premise or in the cloud. Of course, JSM nine includes usability improvements and JDK 23 provides more options for fine tune and improved peak performance with additional Graal compiler. A dynamic just in time compilation written in Java that transforms bytecode into optimized machine code, which I think is also one of the requirements for using like the fast start capabilities of Java. And serverless is using that bytecode to optimize machine code changes. But yeah, if you're paying Oracle's ridiculous Java fees and not using Coretto or any of the other numerous Java ports that have happened, you can get this from Oracle. For Java 23, there are this that.
[00:48:32] Speaker D: I don't at all understand why I care. I guess it's like millisecond speed differences of the byte code. Like I understand the boot up times and stuff like that, but I just want things to run and go run in the background. As Ryan said, like most of the stuff that I run as back end services are, do not have that strict of an SLA. I just don't know that I care as much about some of these extremely detailed features now. Yeah, I'm not a Java developer exactly.
[00:49:04] Speaker B: Just enough Java to go, it's garbage collection and we on.
[00:49:07] Speaker C: Yeah, and if you watch a JVM startup, you know, that takes forever. And so if you're trying to do a serverless function that you have a thing that takes forever, that's a problem. So the more you can optimize it to start and return a response to your customer, the better off you're going to be.
[00:49:21] Speaker D: Yes.
[00:49:22] Speaker B: In general, I literally thought that Java 23 was like a miss and like they just skipped a whole bunch of numbers, you know, like, no, no, they.
[00:49:30] Speaker C: I checked that too because I was like, maybe they just jumped sequences. They were like, yeah, or Java eight, we'll just, you know, jump to, you know, 20 years. Yeah, but no, they've had a version.
[00:49:40] Speaker B: They'Ve been releasing two versions a year for a while.
[00:49:43] Speaker C: Yeah, that's just how I've touched. I've been in this whole space.
[00:49:46] Speaker B: Yeah, me too.
[00:49:47] Speaker C: I mean, I think Oracle eight was when the lawsuits happened and everyone realized that Oracle was going to sue you for having Java eight. And I think that's probably why it's marked in my brain is like, that's the last Java, don't touch it after that.
Just go to Coretto or OpenJDK or any of the other numerous options that are out there for Java.
[00:50:06] Speaker B: Well, Java eight is also the latest supported or yeah, the oldest supported version too. So like they haven't kicked everyone off kicking and screaming yet.
[00:50:15] Speaker C: It'll happen.
[00:50:16] Speaker B: It will come 2030, it says with the BD.
[00:50:19] Speaker D: Oh wow.
[00:50:20] Speaker B: I know.
[00:50:21] Speaker C: Is that because we're getting too close to Linux epoch time? And they're like, yeah, we don't want to start doing the mathematic.
[00:50:29] Speaker D: I guess they figured they can delay it by a year, still be within that window.
[00:50:33] Speaker C: Yeah, I like look forward to epoch time. That's going to be the next y two k. It's going to be fun times.
Another day show.
Oracle stock has had strong earnings with a strong earning beat, driven by cloud growth and new partnerships. Their quarter was good, with shares of $1.39 versus the target of $1.32 per share. Revenue for the quarter rose 8% from a year before to 13.31 billion, better than Wall street estimates. While net income rose to 2.93 billion, up from 2.42 billion in the same period a year prior. Cloud service and licensing support revenue rose 10% from a year from the year earlier to 10.52 billion, whereas cloud infrastructure grew 45% to 2.2 billion, up from 2.42 billion in the same period a year before. And I don't really understand if cloud service and licensing is like oracle licensing and cloud OCI revenue shoved together, and then they also break out cloud infrastructure into its own number. But like, 2.2 billion is not a lot of money for cloud.
[00:51:31] Speaker D: Yeah, I know, I was just thinking.
[00:51:33] Speaker C: What'S AWS is way much bigger than that.
[00:51:37] Speaker B: Yeah.
[00:51:38] Speaker C: Katz, who's the CEO of Oracle, said that demand for the oracle cloud is outstripping supply, and she is okay with that problem, which, yeah, sure, people, you're in demand, people will pay for it.
[00:51:50] Speaker B: Well. And when you're last on the market. Yeah, you want the overflow.
[00:51:54] Speaker C: Exactly.
[00:51:56] Speaker D: Okay, so AWS is 575 billion, approximately.
[00:52:01] Speaker C: Is that all of Amazon or just.
[00:52:03] Speaker D: Oh, sorry.
AWS is approximately 100 billion.
[00:52:09] Speaker B: Yeah, there you go.
[00:52:11] Speaker C: So like 50 times like 25 billion per quarter versus 2.2.
[00:52:18] Speaker D: They forgot a zero somewhere. That's fine.
[00:52:21] Speaker B: Yeah.
[00:52:22] Speaker C: And then our last answer from OCI is they are announcing the general availability of bare metal instances with the new AMD Instinct Mi 300 x gpu. OCI supercluster with AMD instinct Mi 300 x accelerators provides high throughput ultra latency RDMA cluster network architecture up to 16,384 mi 300 x GPU's. And a single instance will be $6 per hour and include eight AMD instinct Mi 300 x accelerators with 1.5 terabytes of memory and an Intel Sapphire Rapids CPU and two terabytes of DDR five memory and eight three by 3.84 terabyte NVMe drives with front end network support of 100 gigs. So if you thought 50 and 40 were fast, 100 gigs available on this.
[00:53:06] Speaker D: One, they'll say, you're doing the cloud run.
[00:53:11] Speaker C: This is all for. This is model training, baby.
[00:53:14] Speaker D: Yeah, it's true.
[00:53:17] Speaker C: All right, you guys aren't impressed with this one.
[00:53:18] Speaker B: It's fine. No, no.
[00:53:21] Speaker D: $6 an hour. I'm like, ooh, that adds up quickly.
[00:53:24] Speaker B: Yeah.
[00:53:26] Speaker C: Yeah. $6 an hour. I was like, whoa, that's a, that's a pricey box.
What is that? It's time. 727, 20 times 6720 below and 4320. I like, I have a bunch of different, like, 766 I've used, like, I don't know. Depends how many days of the month you want to calculate it by.
[00:53:43] Speaker D: 750 is 30.
Hold on.
[00:53:47] Speaker C: So 750 times 631. $4,500 a month for that certain, you know, for one. Yeah, it's expensive.
All right, we'll see you guys next week here at the cloud. Well, you won't see me, but you'll see these guys, hopefully.
[00:54:00] Speaker D: Maybe, maybe, maybe bring out the web. Sorry, no.
[00:54:05] Speaker C: Oh, you get a matt.
[00:54:10] Speaker A: And that's all for this week in cloud. We'd like to thank our sponsor, Archera. Be sure to click the link in our show notes to learn more about their services.
While you're at it, head over to our
[email protected], where you can subscribe to our newsletter, join our slack community, send us your feedback, and ask any questions you might have. Thanks for listening, and we'll catch you on the next episode.
[00:54:41] Speaker C: All right, I got an after show for you guys. It was gonna be a major topic, and then I was like, I don't really know this. Is that applicable to cloud ish? I mean, it is, but it isn't. So for those of you who are familiar with Adam Jacobs, he is one of the co founders of Chef. Where he left Chef, I don't know, four years ago, maybe five years ago. He's been gone for a while. Between getting bought by progress and somewhere in there, he left the organization. He's also really well known in DevOps circles.
He has just announced, basically taken out of stealth for his new startup called System Initiative.
Basically, it's his new take on how to do DevOps in interesting ways.
I asked chat GPT to summarize this a little bit just because it's a very long blog post.
But basically system initiative is introduced as a game changing DevOps automation tool. It offers a fresh approach that addresses longstanding industry issues such as slow feedback loops and complex infrastructure challenges.
Adam Jacobs says this is something he's believed in for a long time, and he's emphasizing the importance of building products you are passionate about. This project is the culmination of five years of work, but feels like the culmination of a career in DevOps tooling, he says.
He says the problem is with infrastructure as code. While functional infrastructure as code is limited, it locks systems and static representations of dynamic environments, causing inefficiencies. And Adam believes that industry is stuck and needs new solutions that are better than ever. And his solution to this is digital twins and simulation, which I think is a really cool use case for digital twins.
I was like, as soon as I got to that part of the article, I was like, okay, you have my interest.
A key innovation in system initiative is using one to one digital twins of cloud infrastructure, decoupling real and hypothetical states. This solves the feedback loop problem by simulating the MT changes without deploying deploying them to the actual infrastructure. System initiative simplifies automation by eliminating the need to understand the underlying domain and the tool itself, and its digital twins offer a one to one translation with no loss of fidelity.
Adam reflects on working with major enterprises and the complexity inherent in all infrastructure, and system initiative embraces this complexity with a platform designed to be powerful, flexible, and expressive system initiatives. Infrastructure is based on a reactive graph of functions, making it easier to create, modify, and automate complex environments dynamically, as well as it needs to enable real time collaboration, allowing multiple users to work on the same infrastructure and see changes instantly. This will drastically improve the communications and productivity in your DevOps team, he says.
The project is 100% open source, inviting contributions and fostering collaborative community to build and extend the platform. And that's a big thing. I was always big on from the chef days. He was a very strong advocate of being open source. He's also commented on some of the crazy licensing changes that happened over the years, typically someone they go to for a quote.
He says this initiative aims to replace infrastructure as code today and transform how teams work together in complex environments in the future. And he represented it as the next steps in the evolution of DevOps.
That's his thoughts. I did sign up for an account. I did watch the first demo.
I will say that I left it unsure.
I don't know what you guys thought, but that was my initial take. The GUI is nice, though. It's nice idea. I like the clickiness of it. It seems like a lot of clickiness though, where I could type a lot faster in id, which is probably my one detriment to it, is I'm like, I like to type because I can do that very fast.
[00:58:01] Speaker B: Yeah. So I haven't watched the first demo I signed up because I'm curious and I think that some of the problems with infrastructure code are valid. There's definitely issues with having a codified state and then having your environment be dynamic. And you can only get so far with, you know, running a plan to see the changes, just actually see all of the mechanizations. So I really like the digital twin component of it and the full simulation of the state. So that part is really neat. I. I just don't know about the.
Yeah, the sort of console esque things about this. Right. I don't know what advantage I get over this versus just go into my cloud workloads console and looking at it there. Like, some of these features have been in AWS cloudformation forever.
[00:58:58] Speaker C: Right.
[00:58:58] Speaker B: So it's like being able to visualize your cloudformation resources, being able to generate a change set on those resources, having cloudformation do that, I'm like, I'm not real sure this is adding a whole lot to that story.
I like that in cloudformation you can define it as a template, and you still get your cake and eat it too. And you've got your codified state or desired template that you want to deploy. And then you also have the cloudformation hosted state centralized, and, you know, because it's a single storage for that state. Cloudformation allows you to do a lot of those chain sets and stuff, which is cool.
So I'm a little. I'm conflicted on this. Like, I'm going to use it. I'm curious enough just based off of the language, but I'm very dubious of previous software that's come out of that sort of chef think, think tank.
I think they understand the problems really well. I don't know if I always agree with their solutions.
[01:00:04] Speaker C: Yeah.
Where I feel like this let me down a little bit in the demo was he starts out and the demos aren't very long, it's like two or three minutes. Can't get you started. And he's like, the first thing you do is you add an AWS credential. And I'm like, why do we start there?
And then you add a region and then you start adding resources and stuff and I'm like, but then he didn't really show where you define the VPC. And so is it making a bunch of assumptions for me as I'm building out the infrastructure? There's a bunch of things that is unclear to me and what I really wished I could do. And hopefully this is something they're working on. I assume they are. I haven't talked to them, but is that they're going to have the ability for you to potentially import your existing cloud formation or terraform code into this visual representation or the state file or maybe even just go scan my AWS account and give me the visualization of it. And then I can kind of see because like his whole pitch in the article is like these are things that you already inherently know because you know Amazon and so being able to do them in the digital twin is so easy because it's just doing Amazon things. And I'm like, okay, but like what you're presenting to me is not what I know in my world. Like, and maybe this is me, maybe a lot of people don't think of this way, but my world does not start at the credential. My world starts at the region and then the VPC and then the subnets and then everything gets attached to that from that perspective. And that's my mental model of it. Maybe that's an incorrect mental model, but that's how I think about it.
And so like I got to watch some more of the demos, I got to play with some more. I didn't, wasn't willing to actually submit code into my production environment. So I was like, well, this is how time I guess to create the cloud pod demo environment so I can go play this somewhere. I feel safer if I mess it up. I don't care. And also be free to your eligible because I really don't want to pay for this, but I want to play with it and see what it does. But there was some other tools that were very similar to this that I recall coming and going at one point or another. And so yeah, I just have doubts. But again, if it had an import, Billy, for me to import my existing infrastructure and I could tweak things easily right here and see how that would happen in the digital twin, I would be sort of more excited about this. It was very ambitious though. I don't want to demean what they're doing. I think that they're doing something cool. They're trying to take a risk and they think they have a different way of paradigm of doing this. And the digital twin concept makes a lot of sense to me at how to do this, especially the feedback loops and all those things.
I think it's being let down a little bit by the implementation.
[01:02:36] Speaker B: Yeah, yeah. I'd like to see import as well as export because I know that there's a lot of solutions on the market right now that are, that will look at your infrastructure and detect drift and just go update your templates, which is cool. I like that.
And so if you think about that capability plus the full simulation capabilities of the digital twin, that's really neat.
[01:03:05] Speaker D: That's the multi person collaboration to be able to in real time. See, because that's one of the biggest pain points of terraform release. You have multiple people working on a specific area of code. You have to do the old school software thing of like, okay, you do this first, you do this next and kind of stage it out at a person operational level. So if you can both be manipulating and be able to see what the other one's doing, so you got don't collide in a bad way. I think that that's a really cool idea that no other real infrastructure's code is taken on and the digital twin concept really helps enable some of that.
[01:03:42] Speaker B: Yeah, I mean, that's only if you like people. So pass.
[01:03:46] Speaker C: I mean, Mike Ryan doesn't like people.
[01:03:48] Speaker B: Yes. And every time, you know, we're editing show notes and you, you enter and move my text down. Like I. If you were closer, I'd stab you.
[01:03:57] Speaker D: I mean, I do it on purpose just to piss you off.
[01:03:59] Speaker B: I know you do, probably, yeah. So cool we're going to do this with infrastructure that would go well.
[01:04:03] Speaker D: Yeah. What could possibly be wrong? I'll be like aging the subnet, adding a Nac gateway on it. Just you're with you while you're trying to like, you know, block all egress traffic from my VPC. So I could picture the old like Amazon game days where they would like go in your account and like all of a sudden just disable network traffic and be like, hey, all of a sudden your thing's not working. Go debug. I'm like picturing us doing that to each other while we're doing show notes. It'll be, it'll be awesome.
[01:04:35] Speaker B: I think the secret might be out, Matt, that when you and I work on projects together, we don't get a lot done.
[01:04:40] Speaker D: It's either that or we get a lot done in exit flows. It's either like we're great, or we're terrible. One of the two.
[01:04:48] Speaker C: It's very true. It's either hot or cold.
I just kind of playing with it here as we're talking and I watched the demo, but I see now why the credential is important because that's how you actually authenticate to make the change to the infrastructure. So I'm like, okay, cool, but why did you make that a box versus me? Just like, please authorize, authorize yourself to the console you want to do. And then why did we in this day and age, we're going to add secret keys instead of like, hey, we're going to give you actually a proper integration through OAuth.
Again, it's an MVP. It's early, they're getting feedback. I don't want to bash. But again, it makes sense now that I'm playing with it just a little bit. Yeah, I've got a great account. Create an AWS account.
[01:05:37] Speaker B: Yeah, I mean, I'm definitely going to play around.
There's little things like that. The credential walkthrough, I immediately, it's like, well, they're not separating out the admin access to workload access, and that's dangerous.
And these are part of the reasons why I don't like UI based driven infrastructure is because it's really easy to just assume that that box is your thing and then you realize your application is running with your super admin grid.
[01:06:03] Speaker D: You know, like you've never seen that happen before, right?
[01:06:06] Speaker B: Never once. Yeah, I know.
[01:06:08] Speaker C: And then like, am I completely restricted to the fact that this is their service and so, like, if their service goes down, I have no way to modify my infrastructure. I mean, like, it'd be nice if it would like, you know, connect to my get still, I guess you can export an import workspace. But again, I still, yeah, there's just things I don't quite get, but I.
[01:06:26] Speaker B: Got your, your infrastructure wouldn't go down, just your ability to.
[01:06:29] Speaker C: Yeah. To modify it, which is.
[01:06:30] Speaker B: Right.
[01:06:30] Speaker C: A big deal.
[01:06:31] Speaker B: So, yeah, I mean, you'd always have the console and direct access to cloud.
[01:06:36] Speaker D: You know, then is this the same thing where, you know, they have to update? I assume they're going to have to update all this because it's not like they have every Amazon service in here. They have 108 assets.
[01:06:48] Speaker C: Yeah, I mean, they have the most common ones right now, but it's also going to be, you know, how do you get this to Azure? How do you get this to GCP? Like, it's going to be a lot of things to maintain. And so I'm hoping, I assume that they're going to go with a similar idea of hashicorp is that other cloud providers, if this takes off, would be able to build these as plugins, which would be what I assume.
So, yeah, it's sort of interesting.
I try to export, but I don't know where it actually exports the workspace too. Yeah. Anyways, I'm intrigued. That's my takeaway at this point.
I hope it's successful because I like the idea of people trying things and trying new things and seeing what they can do with it.
Is this the end solution? I mean, there's a lot of a benefit to having, I think, an SDLC around your infrastructure as code. And yes, the feedback loop is not as ideal, but, like, could you take this twin idea and like, apply that to terraform? Yeah, because, like, like, the idea is sound. I just don't know about the implementation.
[01:07:42] Speaker B: Yeah, I mean, like I said, like, I think that they've got a really good idea where the problems are with infrastructure as code. Like, there are some real challenges there that are real. And so, like, and, but, you know, solve those. I don't know if this is the solution to those. Like, this is more than that. It's more than I want, more than, it's not what I asked for, you know, kind of thing.
But I don't know. Like, again, I'm going to play around with it and see what I like that it's really well documented. I do like the looks of it.
Oh, man.
[01:08:14] Speaker C: Oh, he's back. Can you hear me?
[01:08:15] Speaker B: I can hear you. Okay.
[01:08:18] Speaker C: I was saying, you were talking and then you. We lost you.
[01:08:20] Speaker B: Oh, it was my Internet. I thought it was Matt.
[01:08:22] Speaker C: It was yours.
[01:08:23] Speaker B: Okay. I was just mumbling gibberish anyway.
[01:08:28] Speaker C: All right, well, let's move that. Well, I think we talked about this. I'm gonna keep an eye on it. I'll let you guys know if I see anything that comes out of this that is interesting. But, yeah, maybe we'll talk about it again if you guys have a chance to play with it, because again, I like the idea.
All right, guys, talk to you later.
[01:08:44] Speaker B: All right, bye. Now.
[01:08:48] Speaker A: That'S all for this week in cloud. We'd like to thank our sponsor, Archero. Be sure to click the link in our show notes to learn more about their services.
While you're at it, head over to our
[email protected], where you can subscribe to our newsletter, join our slack community, send us your feedback, and ask any questions you might have. Thanks for listening, and we'll catch you on the next episode.
