217: The Cloud Pod Whispers Its Secrets to Azure Open AI

Welcome to the newest episode of The Cloud Pod podcast - where the forecast is always cloudy! Today your hosts Justin, Jonathan, and Matt discuss all things cloud and AI, as well as some really interesting forays into quantum computing, changes to Google domains, Google accusing Microsoft of cloud monopoly shenanigans, and the fact that Azure wants all your industry secrets. Also, Finops and all the logs you could hope for. Are your secrets safe? Better tune in and find out! 

Titles we almost went with this week:

• The Cloud Pod Adds Domains to the Killed by Google list
• The Cloud Pod Whispers it’s Secrets to Azure OpenAI
• The Cloud Pod Accuses the Cloud of Being a Monopoly
• The Cloud Pod Does Not Pass Go and Does Not collect $200

A big thanks to this week’s sponsor:

Foghorn Consulting, provides top-notch cloud and DevOps engineers to the world’s most innovative companies. Initiatives stalled because you have trouble hiring?  Foghorn can be burning down your DevOps and Cloud backlogs as soon as next week.

News this Week:

01:27 Vault 1.14 brings ACME for PKI, AWS roles, and more improvements 

• HashiCorp recently announced the general availability of ACME for PKI. 
• Vault 1.14 focuses on Vault’s core secrets workflows as well as team workflows, integrations, and visibility.
• This allows you to use Vault to manage your TLS certificates, using the ACME protocol.
• This allows you to use Vault to manage your AWS IAM roles, making it easier to grant access to your applications.
• Vault has also been optimized for better performance, especially for large deployments.
• A number of bugs have been fixed, improving the stability and security of Vault.
• The Vaults Secrets Operator connects Vault secrets directly into native Kubernetes secrets. 

Overall, Vault 1.14 is a significant release with a number of new features and improvements. If you are using Vault, I recommend upgrading to the latest version.

AWS

03:36 Announcing the AWS Amplify UI Builder Figma Plugin 

• Finally! A plugin that makes Amplify work natively with Figma! (Any UI builders out there in our audience? Bueller? Bueller?)
• AWS Amplify now offers you the UI Builder Figma plugin 
• This new plugin makes it easier to empower your design and development teams to seamlessly collaborate within a Figma file. 
• With the Amplify UI kit, easily theme your components, upgrade to new UI Kit versions and generate and preview React code from your designs directly in Figma.  
• Go from design to code in seconds by generating clean React code inside Figma, and see a live preview of the code running before adding it to your application.  

04:15 Justin- “I went in and set this up today because I had never actually used Figma although I heard lots about it. So I signed up for my free account. I signed in for the plugin for Amplify and then I remembered I don't know how to use Amplify. So it didn't go so well for me but I'm gonna keep tackling it because one thing I'm not very good at is front end development and anything that makes me better as a front end developer would be a plus.” 04:40Jonathan - “Figma works really nice; it’s great for prototyping.” 06:44 AWS Transfer Family announces structured JSON log format 

• AWS Transfer Family now delivers logs in a structured JSON format across all resources - including servers, connectors, and workflows and all protocols including SFTP, FTPS, FTP and AS2.  
• The new format allows you to easily parse and query your logs using Cloudwatch Log insights, which automatically discovers JSON formatted fields. 
• You’ll also benefit from improved monitoring with support for CloudWatch Contributor Insights, which requires a structured log format to track top users, total number of unique users, and their ongoing usage. 
• In addition to the new log format, you’re now able to combine log streams from multiple AWS Transfer Family servers into a single Cloudwatch log group of your choosing.  This allows you to create consolidated log metrics and visualizations, which can be added to cloudwatch dashboards for tracking server usage and performance. 

And that very exciting announcement is linked to this one… 07:14 AWS Transfer Family announces Drummond Group Applicability Statement 2 (AS2) Certification

• If you know anything about the Drummond Group or AS2 you probably care about this. 
• AWS Transfer Family has earned the official Drummond Group AS2 cloud certification seal.  Drummond 
• Group is an independent provider of testing and certification services for various industry standards and protocols. 

07:52Matt- “I read this headline and I looked at it and I went, wow, I've set up TransferFamily at least two or three times, set up all the logs and never have actually looked at them to know that it was not in JSON format.” 08:31 AWS launches AWS AppSync abstraction

• If you didn’t know what appSync was before, you can now know even less by abstracting it behind the AWS Serverless Application Model with the new AWS Serverless GraphQL API resource abstraction. Fun! 
• AWS AppSync is a managed service that makes it easier to build scalable APIs that connect applications to data with a GraphQL endpoint.  

08:59 Matt- “So they added it to CloudFormation and to SAM?” 09:02 Justin- “Apparently that’s what they did. And wrote a blog post about it! So thanks, Amazon! We really appreciate that.” 09:20 AWS Announces Generative AI Innovation Center 

• Amazon, in the midst of many other existential threats (unionization, antitrust no biggie.) wants you to know that they REALLY care about Generative AI too. 
• To prove it we are going to commit 100 million to a new Generative AI Innovation program. 
• The new program will help customers successfully build and deploy generative AI solutions. This will help customers successfully envision, design and launch new generative AI products, services and processes.  
• Building on more than 25 years of deep investment in developing AI technologies for customers and is just one part of AWS’s overall generative AI strategy to bring this technology to customers and partners around the world.
  • “Amazon has more than 25 years of AI experience, and more than 100,000 customers have used AWS AI and ML services to address some of their biggest opportunities and challenges. Now, customers around the globe are hungry for guidance about how to get started quickly and securely with generative AI,” said Matt Garman, senior vice president of Sales, Marketing, and Global Services at AWS. “The Generative AI Innovation Center is part of our goal to help every organization leverage AI by providing flexible and cost-effective generative AI services for the enterprise, alongside our team of generative AI experts to take advantage of all this new technology has to offer. Together with our global community of partners, we’re working with business leaders across every industry to help them maximize the impact of generative AI in their organizations, creating value for their customers, employees, and bottom line.”

10:40 Justin - “…they had a lot of AI and ML features, but they didn't have anything as revolutionary as chat GPT. So generative AI is where all the hotness is right now, and they are definitely lagging behind just a little bit.”

GCP

14:26 Google Domains is shutting down; assets sold and being migrated to Squarespace 

• Google domains is winding down and selling the business and assets to Squarespace. 
• Squarespace entered into a definitive asset purchase agreement with Google, whereby Squarespace will acquire the assets associated with the Google Domains Business. 
• This includes approximately 10 million domains hosted on Google Domains spread across millions of customers. 
• Google launched the registrar business in 2014 as a bit proponent of HTTPS and Top-Level domains. The service just exited beta in 2022. 
• It's better than shutting down the service without a guided migration path...but holy crap!
• Can’t wait till they sell the GCP business to a third party….

14:59 Justin- “I mean, if you can’t make money on 10 million domains I don’t know what you’re doing wrong.” 16:10 Jonathan- “Something as fundamental as domain registration for cloud users seems really weird to me that they would sell that and basically resell through a partner.” 19:58 Google Formally Accuses Microsoft of Trapping People in the Cloud 

• In peak American style, Google has employed the “if you can’t beat ‘em, sue ‘em” mentality when it comes to Microsoft's cloud business. 
• Google is accusing Microsoft of anti-competitive practices 
• Google after being beat up by the FTC, and is now complaining that Microsoft uses software licensing restrictions to keep customers locked into its cloud computing services.  
• The letter specifically takes issue with MS using its Windows Server and Office products to keep clients on Azure, and that Microsoft’s control is a national security risk. 
• Google has raised similar concerns to EU regulators.  
• Essentially, MS charges third party cloud providers extra to run its software,  a cost that customers do not bear if they run on the same software on MS Azure’s cloud platform. 
• This has led to an FTP RFC on how the business practices for cloud computing providers affect competition and data security. And it has not been the only submission to raise concerns about Cloud Platform Competition.

20:57 Jonathan- “It’s kind of a weird conversation, because the free market's the free market. The business should be free to set the prices they charge for any product, for any customer. I mean, if you think about enterprise discount agreements, anything like that is a mechanism to provide different pricing to different customers based on usage of either one resource or combinations of resources. So on one hand, I'm like ‘they should be able to charge whatever they like to whoever they like and the market will figure things out’. On the other hand, it is such a monopolistic position to be in.” 23:20 Trace Exemplars now available in Managed Services for Prometheus

• Cross Signal Correlation where metrics, logs and traces work together in concert to provide a full view of your systems health -- is often cited as the “holy grail” of observability.  
• However, given the fundamental differences in their data models, these signals usually live in separate, isolated backends.  Pivoting between signal types can be laborious, with no natural pointers or links between your different observability systems. 
• Trace Exemplars provide cross-signals correlation between your metrics and your traces, allowing you to identify and zoom in on individual users who experience abnormal application performance. 
• Storing trace information with metric data lets you quickly identify the traces associated with a sudden change in metric values; you don’t have to manually cross-reference trace information and metric data by using timestamps to identify what had happened in an application when the metric data was recorded.
• Google is making this easier with the support for Prometheus Exemplars in Managed Service for Prometheus. 

24:36 Matt- “As things become more serverless, everything kind of becomes in its own little areas, tracking everything and tracking requests and all the different pieces that go through your system, has been a problem. And that's why AWS came out with X-ray, and APIMs andall these other things exist. So it's just another way to do a lot of the same things. It's nice that it's integrated into Prometheus. If you're running your own stacks of Prometheus and Grafana, it'll be nice to be able to do it all in one place versus having to use different tools for different aspects of your monitoring solution.”

Azure

26:01 Microsoft Azure OpenAI lets enterprises feed corporate secrets to ChatGPT  Microsoft wants to make it easier for enterprises to feed their proprietary data and queries into Open AI GTP-4 or ChatGPT within Azure and see the results. What could go wrong? 

• Available in preview with Azure OpenAI service, it eliminates the need for training or fine tuning your own generative AI models. 
• A user fires off a query to Azure, MS cloud figures out what internal corporate data is needed, and the data is combined with the public data set and returned to the user. 
• The models are managed by Microsoft in its cloud, preventing Open AI from having direct access to the customer data, queries and output. 
• It is alleged that this new skill is “useful.” Insert side eye gif of your choice here. 
• Your prompts (inputs) and completions (outputs), your embeddings, and your training data:

• are NOT available to other customers.

• are NOT available to OpenAI.

• are NOT used to improve OpenAI models.

• are NOT used to improve any Microsoft or 3rd party products or services.
• are NOT used for automatically improving Azure OpenAI models for use in your resource (The models are stateless, unless you explicitly fine-tune models with your training data).

27:31 Jonathan - “We talked last week about Google telling their own employees not to use generative AI, especially barred for coding. I wonder if Microsoft will do the same thing.” 27:51 Matt - “Microsoft doesn’t use any chat GPT - any anything along those lines. They disable all the code pilot plugins on all their integrations.” 28:34 Justin - “ I can see the advantage of having an AI LLM model in place to help you do things. But data privacy is the biggest issue in all this. And I kind of agree with Matt, if it's secret, don't put it anywhere that you don't trust it or don't control the endpoints. And maybe cloud isn't right for you for that particular use case.” **Side note, any listeners who want to get Jonathan a birthday gift, his list is at 29:18** 30:00 Accelerating Scientific Discovery with Azure Quantum

• Azure is announcing 3 new innovations to Quantum computing
  • Azure Quantum Elements
    • Azure Quantum elements accelerates scientific discovery by integrating the latest breakthroughs in high performance computing (HPC), AI and Quantum computing.
    • Reduce time to impact and costs by accelerating the R&D pipeline
    • Dramatically increase the search space for new materials, with the potential to scale from thousands of candidates to tens of millions
    • Speed up certain chemistry simulations 500kx 
    • Get ready for scaled quantum computing by addressing quantum chemistry problems with AI and HPC, while experimenting with existing quantum hardware
  • Copilot in Azure Quantum
    • Scientists can accomplish complex tasks on top of the fabric of cloud supercomputing, advanced AI and quantum, all integrated with the tools they use today. 
    •  It can generate the underlying calculations and simulations, query and visualize data and help get guided answers to complicated concepts. 
    • Helllllllllo Skynet! 
  • MS Roadmap to Quantum Supercomputer
    • MS has achieved the first milestone towards a quantum supercomputer. They can now create and control majorana quasiparticles.We don't know what these are, but they sound really cool!  With this achievement, they are on the way to engineering a new hardware-protected qubit. 

30:56 Jonathan - “I hadn't really considered that we could use the generative AI tools to actually write code for quantum computers actually. That's very useful to know because it's such a weird and limited language… but getting from a business idea, to code, to actually something that actually runs on quantum computers is a massive step. And actually being able to extract that and have Copilot write that code for you is super interesting. ” 31:39 Matt - “I would have gone with terrifying, but interesting also works.” 31:53 Public Preview: Network Observability add-on on AKS

• And lastly this week from Azure…
• Azure has a new network observability add-on for AKS that will scrape useful metrics from K8 workloads and emit actionable networking observability data into industry standard prometheus format, which can be visualized with Grafana. 
• Key Customer Benefits to YOU (the customer) 
  • Get access to node-level network metrics like packet drops, connection stats and more
  • Support for all Azure CNI’s
  • Support fall AKS node types Linux and Windows
  • Easy deployment using native Azure Tools
  • Seamless integration with the Azure managed Prometheus and Azure-managed grafana offerings. 

And with that, we’re all burned out on logs. Networking and otherwise. 

Oracle

Continuing our Cloud Journey Series Talks

33:40 How to build a FinOps roadmap | Google Cloud Blog

• Justin is going to be at the FinOpsX Conference later this week, but in the meantime…How do you create a roadmap to FinOps? How do you get Cloud costs under control? 
• Define your goals. What are you trying to achieve with FinOps?
• Assess your current state. What are your current costs? Where are you wasting money?
• Develop a plan. What changes do you need to make to reach your goals?
• Implement the plan. This includes setting up budgets, monitoring costs, and making changes as needed.
• Measure your progress. Are you on track to reach your goals? If not, make adjustments to your plan.
• Continuously improve. FinOps is an ongoing process, so be sure to review your plan regularly and make changes as needed. 

30:31 Jonathan - “I like thinking about not just where we're wasting money, but why we're wasting money and how we're wasting money and what is it that got to that place where the finance is knocking your door… So I think one of the things that I'd focus on, would be processes and tooling and figuring out, well, why do we end up with all these objects in an object store that we don't need or want anymore? Or why do we have all these instances stood up and that no one's responsible for?” 38:15 Justin - “In the Google model and their steps, they talk about first defining your stakeholders, which is your CCOE, which we talked about a lot, if you have one. Engineering team is a stakeholder, your platform team is a stakeholder, the business and of course your accountants and finance team, who are the people yelling at you probably that you need a finance practice because the costs are out of control. But that's really kind of step one.”

Closing

And that is the week in the cloud! We would like to thank our sponsors Foghorn Consulting. Check out our website, the home of the cloud pod where you can join our newsletter, slack team, send feedback or ask questions at thecloudpod.net or tweet at us with hashtag #thecloudpod