244: CoPilot For the People!

Welcome to episode 244 of the Cloud Pod Podcast - where the forecast is always cloudy! We’ve got a ton of news for you this week, including a lot of AI updates, including new CoPilot Pro and updates to ChatGPT, including the addition of a GPT store. Plus, we discuss everyone's favorite supernatural axis, MagicQuadrants.It’s a jam packed episode you won’t want to miss.

Titles we almost went with this week:

• Switching from Google is Finally Easier
• Cheaper AI Doesn’t Mean Better AI 
• Is the Cloud Pod Better Than Microsoft at Containers? 
• AWS is the Leader in Containers - Because You Can Run Them in Cloudshell 
• The Cloud Pod is Connecting to the World With Some Undersea Cables 

A big thanks to this week’s sponsor:

We’re sponsorless this week! Interested in sponsoring us and having access to a very specialized and targeted market? We’d love to talk to you. Send us an email or hit us up on our Slack Channel. 

General News

2gather Sunnyvale: Cloud Optimization Summit On February 15, Justin will be onsite in Google’s #Sunnyvale office for the @C2C #2Gather Sunnyvale: #CloudOptimization Summit! Come heckle him, we mean JOIN him, to talk about all things #GenAI and #CloudOps. Consider this your invitation - he’d love to see you there! Sign up →  https://events.c2cglobal.com/e/m9pvbq/?utm_campaign=speaker-Justin-B&utm_source=SOCIAL_MEDIA&utm_medium=LinkedIn

AI is Going Great (or how ML Makes all Its Money)

01:20 Introducing ChatGPT Team

• ChatGPT has added a new self-serve plan called Chat GPT team.  
• Chat GPT team offers access to their advanced models like GPT-4 and DALL-E 3 and tools like advanced data analysis.  
• It additionally includes: 
  • A dedicated collaborative workspace for your team and admin tools for team management. 
  • Access to GPT-4 32K context window
  • Tools like Dall-E 3, GPT-4 with Vision, Browsing, Advanced Data Analysis with higher message caps
  • No training on your business data or conversations
  • Secure workspace for your team
  • Create and share custom GPTs with your workspace
  • Admin console for workspace and team management
  • Early access to new features and improvements. 

03:00 Introducing the GPT Store 

• ChatGPT has also launched their AI Marketplace, which will get you access to over 3 million custom versions of Chat GPT. Yes, 3 MILLION versions. 
• Today, they’re starting to roll out the GPT store to ChatGPT Plus, Team and Enterprise users. 
• There will be new and highlighted Chat GPTs every week at the store, the first week has some interesting options:
  • Personalized trail recommendations from AllTrails
  • Search and Synthesize results from 200M academic papers with Consensus
  • Expand your coding skills with Khan’s Academy Code Tutor
  • Design Presentations or social posts with Canva
  • Find your next read with Books
  • Learn math and science anytime, anywhere with the CK-12 Flexi AI tutor

05:05 Matthew- “You're watching everyone do everyone their own thing. It's going to spread very wide. And I think you'll just see this ebb and flow like, okay, we did it on three. Now let's do it on four, four turbo, et cetera. And then you'll see like all like a whole bunch of die off along the way. And you'll see like the couple key ones that, the top 2%, 1 % of the ones that actually are useful, the 3 million models actually become more and more useful over time.” 09:28 Snyk’s AI Code Security Report Reveals Software Developers’ False Sense of Security 

• Snyk says don’t be fooled by the security of AI generated code. 
• Per their recent report, they said developers have a false sense of security in AI generated code with over 75% of respondents claiming that AI code is more secure than Human code.  
  • However, the same report did say that 56% of survey respondents admitted that AI-generated code sometimes or frequently did introduce security issues. 
• Their survey found that developers recognize the risk of AI, but turn a blind eye due to the increased benefits of accelerated development and delivery, leading to the age-old problem of ignoring security for other goals - such as speed to market and delivery timelines. 
• Make sure you are scanning code generated by AI tools, and that you're aware of what the code is doing and not bypassing security policies/controls. 

10:09 Ryan - “You still have to scrutinize your AI code to make sure that it's actually going to do what you expect. So the idea where it's more secure is a fallacy, right? Because a lot of these things are you're introducing things because of patterns and things that interact with that interact with other libraries. And it's difficult for a human. I don't think it's going to be any easier for a computer.” 13:04 Anthropic’s Gross Margin Flags Long-Term AI Profit Questions 

• Listener note: paywall article
• Per The Information’s reporting, it appears that the margins on AI may not be as attractive as other cloud software. WEIRD. 
• This is based on some analysis that Anthropics Gross margin is between 50-55% which is far lower than the average for cloud at 77%. 
• And worse, it may not improve over time, with at least one major Anthropic shareholder expecting it will be about 60%.
• One of the key things is that the gross margin does not reflect the server costs of training the AI models, which is part of its R&D expense. 
  • These can be up to 100M per model. 
• No major surprise to me here, but good to see my hypothesis becoming real. 

AWS

14:59 Amazon ECS supports a native integration with Amazon EBS volumes for data-intensive workloads

• Amazon ECS now supports an integration with Amazon EBS, making it easier to run a wider range of data processing workloads. You can provision Amazon EBS storage for your ECS task running on EC2 or Fargate. 
• So you now have three storage options for ECS
  • Fargate provides you 20GB of ephemeral storage which can be configured up to 200GB.
  • EFS for workloads that require multiple tasks to access the same data.
  • And now, direct EBS attachment to your task for high-performance, low-cost storage that doesn’t need to be shared across tasks. 

15:58 Justin - “Now you can directly attach an EBS to a task and they're going to handle moving that around for that's, that's actually really handy because before to use EBS volumes, you'd have to set it up at the host level. Then you have to do a kind of task mapping to specific hosts, which could be kind of complicated and some toil. So I, you know, I'm not too upset to see this one.” 18:22 AWS CloudShell now supports Docker in 13 Regions  

• AWS Cloudshell now has built-in support for Docker, making it easier than ever for developers to quickly spin up containers and run commands inside them directly from Cloud Shell.  (Per Justin: God no…please no. WHY!?!)
• Cloudshell users can initialize Docker containers on demand, and connect to them to prototype or deploy docker-based resources via AWS CDK Toolkit.  
• Cloudshell aims to simplify container-based development workflows for AWS Cloudshell users by providing instant access to Docker.  

18:51 Justin - “This is all cute on the surface, until you remember that basically there is no audit logging of Cloud Shell. And inside of a Cloud Shell Docker container, now you have no auditing and you can do anything you want to inside of a container. So basically you just created a huge security hole inside of your Cloud Shell environment that you can't turn off. So thanks, Amazon, I really appreciate this one.”  23:43 Amazon Route 53 Resolver DNS Firewall now supports query type filtering Amazon Route 53 expands geo proximity routing

• Route 53 got two new enhancements
• The Route 53 Resolver DNS Firewall will now support query type (Qtype) filtering.  This can allow you to block any outbound requests to TXT records for example. This is a common attack vector for DNS tunneling. 
• The second Feature is Route 53 geo proximity routing now has an additional routing policy for DNS records in public and private zones.  
  • This will allow you to add the geo proximity routing to your DNS records via the console, API, SDK or CLI. 
• For information on pricing, check it out here. 

27:09 Ryan - “...this opens up Route 53 for being an option to use Route 53 inside your VPC rather than routing all your traffic through sort of an IT -maint DNS device or service so that we're AD, because that's where your security rules for your company are maintained. So this is pretty handy because if you can't use Route 53, you lose a lot of native benefits to AWS.” 27:42 Amazon CloudWatch Logs now supports account level subscription filter

• You can now use account-level subscription filters using the put-account-policy API. 
  • This capability enables you to deliver real-time log events that are ingested into Amazon Cloudwatch Logs to a Kinesis. 
• This allows customers to forward all or a subset of all logs to services like Opensearch.
• Previously you would have to set up a subscription filter for each log group. With the account level customers can egress logs into multiple or all log groups with a single subscription filter policy. 
• This service is available in all regions except for Israel (Tel Aviv) and Canada West (Calgary.) 

32:49 OpenSearch Expands Leadership Beyond AWS   

• In December the Opensearch project launched the inaugural OpenSearch leadership committee. 
  • This is the first step in the goal of making this a true supported OSS project outside of Amazon’s stewardship. 
• The new team has 7 members from AWS/Amazon, 1 from Oracle, 1 from Developmentor, 2 from Aiven, 1 from Aryn.AI and 1 from Logz.io.   

37:58 Amazon EKS extended support for Kubernetes versions pricing 

• It's time to upgrade your K8 clusters on EKS *or* pay some steep costs!!  
• AWS is introducing the public preview of their extended support for K8 versions, which will give you an additional 12 months of support for K8 minor versions.  
  • BUT at a pretty expensive price. 
• K8 versions in standard support will still be $0.10 per cluster per hour, but clusters in non-supported versions will now be $0.60 per cluster per hour. 
• This is available now as preview for no additional cost, but effective April 1st 2024 extended support will be charged. 

38:43 Justin - “I kind of like this from a security perspective of like, Hey, let's, let's put some teeth into it. Like you're not using more modern versions of Kubernetes that are more secure and more capable and more stable. Uh, we're going to charge you more money for it. Kind of like it.” 42:49 Easier EC2 instance maintenance with managed draining for Amazon ECS capacity providers 

• ECS has fixed some additional unnecessary toil in ECS. 
• For a long time, ECS has had the built-in ability to drain tasks that are running on Amazon EC2 instances, and move the tasks to other instances, in order to allow the original instance to be replaced or terminated. However, utilizing this feature required customers to implement a custom solution that relied on auto scaling lifecycle hooks to set container instances to draining, while all tasks were drained. 
• NOW, Amazon ECS provides managed instance draining as a built-in feature of Amazon ECS capacity providers.  
• This new feature enables Amazon ECS to safely and automatically drain tasks from Ec2 instances that are part of an Autoscaling Group associated with an Amazon ECS capacity provider.  
  • This will allow you to eliminate custom lifecycle hooks that were previously required. 

GCP

44:31 Announcing Humboldt, the first cable route between South America and Asia-Pacific

• It's been too long since we talked about undersea cable, which through this podcast the guys now built a semi-unhealthy fascination with. 
• Google, along with their partners is announcing Humboldt, a subsea cable route linking Chile, French Polynesia and Australia; the first ever to directly connect South America to APAC.  
• The cables will be a part of the South Pacific Connect initiative. 
• This has been a goal of the Chilean government since 2016. 
• The cable is named after Alexander von Humboldt, a German polymath, geographer, naturalist and explorer who traveled extensively to the Americas at the turn of the 19th century. 
• It was selected by Chilean residents, who voted via social media in a naming contest. 
• It's a 9200-mile cable, and we’re looking forward to learning more technical details as this project comes to fruition.

47:36 Introducing Bulikula and Halaihai, subsea cables to connect the central Pacific

• Building on the above news, Google is announcing two new intra-Pacific cables, the Bulikula and the Halaihai - part of the Pacific Connect initiative - in collaboration with Amalgamated Telecom Holdings, APTelecom and Telstra. 
• Balikula connects Guam with Fiji, and is the word for “golden cowrie,” a rare shell found in the pacific ocean, and worn by local chieftains as a badge of rank.  
• Halaihai, which will link Guam and French Polynesia, is named after a type of vine that grows on the beaches.  

49:21 Cloud switching just got easier: Removing data transfer fees when moving off Google Cloud

• Google is making us a little bit crank with these price increases. 
• Google Cloud customers who are so cranky they want to get rid of Google Cloud and migrate their data to another cloud provider or on-premise can take advantage of free network data transfer to migrate their data out of Google Cloud (ironically enough as a Google customer I’m also getting hit with an upcoming increase to my egress traffic fees… I guess to pay for the customers who are leaving. Cool. Cool cool cool.)
• Google contends that these high fees restrict the market and limit the ability of customers to choose clouds. While I suspect they're hoping to use this to pressure Azure and AWS, I don’t think it will have much impact. If they truly wanted to be a good partner, they would remove these fees completely for all customers. 

51:02   Ryan - “This almost feels like the result of an antitrust to me, and I don't know of any, but like, because it just logically, it just doesn't make any sense. It's still overhead that you have to pay; even a cost reduction would make more sense than this.” 52:10 Document AI Custom Extractor, powered by gen AI, is now Generally Available

• Google is announcing that the Document AI custom Extractor is now GA and open to all customers, and ready for production use through APIs and Google Cloud Console. The custom extractor, built with Google’s foundation models, helps parse data from structured and unstructured documents quickly and with high accuracy. 
• The results of the document AI extractor are more efficient ways for customers and partners to implement generative AI for their document process workflows, whether to extract fields from documents with free-form text, or complex layouts, customers and partners can now use the power of Generative AI at an enterprise-ready level. 
• For information on Document AI pricing, take a look here. 

52:41   Justin - “…my first real excitement about ML was when Amazon did this with Textract. And so I'm glad to see this also now exists in Google. It's existed in Azure for a little while as well. And so this is a very common use case where I want to take a document and I want to scan it into my computer and I want to do something with it. And it was amazingly hard until these two technologies came out to be. And this is one of the really cool ML use cases that apparently doesn't make money.” 54:27 Standardize your cloud billing data with the new FOCUS BigQuery view 

• Google is rolling out their first FOCUS solution with the BigQuery view for FOCUS v1.0 preview.  
• This BigQuery view transforms the data you would normally get via the standard billing export, detailed billing export and price export with data attributes and metrics defined in FOCUS. 
• The BigQuery view is a virtual table that represents the results of a SQL query. The bigquery view can be formed off of a base query that maps google cloud data into the display names, format and behavior of the Focus Preview dimensions and metrics. 
• Big Query Views are virtual tables, and incur no additional charge for data storage if you are already using Billing export to BigQuery. 

57:00 Monitoring for every runtime: Managed Service for Prometheus now works with Cloud Run

• Managed Service for Prometheus now supports Cloud Run.
• With the release of this feature, you can now export native time-series data and user-defined metrics out of Cloud Run, organizations can continue to use the prometheus ecosystem while exploring the simplicity and ease of use of Cloud Run. 
• You can instrument with Prometheus or OTLP metrics once, deploy it on Compute Engine, GKE or Cloud Run, and then query and alert on the data all together with a single line of PromQL in either Cloud Monitoring or Grafana. 

58:22   Matthew - “...the advantage of this is like is to me, what really the concept of multi cloud is, you know, to move the workloads where you want and leverage different tooling, you know, in that way. And this kind of is to me a step in that direction.” 59:32 Announcing Workflows execution steps history

• Workflow, which is Google's version of Step Function now supports execution steps history.  
• You can now view step level debugging information for each execution from Google Cloud console, or the REST API. 
  • This is especially useful for complicated workflows with lots of steps and parallel branches

1:01:52 Introducing granular cost insights for GKE, using Cloud Monitoring and Billing data in BigQuery   

• Google is announcing Cloud Monitoring Metrics in BigQuery (in preview). 
• With this capability, you can now combine billing data with resource utilization metrics, empowering you to perform detailed analysis in BigQuery.
• While this is super cool for Finops,  it has a lot more use cases than what Google is touting.
• Along with the preview, Google is providing an out of the box Looker Studio template that combines cloud monitoring and detailed billing metrics for GKE, allowing you to pinpoint the exact cost of specific clusters -- no more guessing about what is driving your bill. 
• Benefits for google cloud users, including:
  • Monitoring resource allocation
  • Cost-driven decision making
  • Enhanced cost visibility
  • Improved chargeback visibility

1:02:04   Ryan - “Yeah, I'm looking forward to playing around with this because this is just an example of why I love cloud computing. And yes, I can run Kubernetes in my data center and I have so much more freedom and agency in that process. But I don't get cool visualizations of cost utilization that I didn't have to spend any time on. So I like this enhancement.” 1:03:43 Personalized Service Health is now generally available: Get started today 

• When outages occur in your cloud service, it's critical to understand the cause of the impact, so you can chart a course of action and mount an effective response.  
• In August 2023, they introduced Personalized Service Health, and now it is Generally Available with 50+ google cloud products and services, including compute engine,  cloud storage, all cloud networking offerings, BigQuery and GKE. 
• Several features but three good ways to use Personalized Service Health:
  • Discover Incidents through proactive alerts
  • Control which service disruptions are relevant to you
  • Integrate with your incident management workflow

Our $1 billion investment in a new UK data center 

• Google is announcing their intention to invest 1 billion dollars in a new Uk data center in Waltham Cross, Hertfordshire. The 33 acre site will create jobs for the local community. 
• As a pioneer in computing infrastructure, Google claims their data center centers are some of the most efficient in the world, and they have an ambitious goal to have all their data centers on carbon free energy by 2030. 
• The new data center will allow computing capacity to be available to businesses across the UK, supporting AI innovation and helping to ensure reliable digital services to Google Cloud customers and Google Users in the UK and Ireland. 
• We assume this is for expansion of the current London zone, and not a new region as it's only 28 miles from London. 
• Where the heck is Jonathan when we need him for geography help? Rude. 
• Hopefully this will help with some of Google’s capacity issues. 

Azure

1:09:31  General Availability: Premium SSD v2 and Ultra disks support with Trusted launch 

• Premium and Ultra SSD are now supported with trusted launch configurations.
• We’re ultra bored. 

1:11:36 Bringing the full power of Copilot to more people and businesses 

• Microsoft has launched CoPilot pro, a new subscription that delivers the most advanced features and capabilities of MS Copilot to individuals looking to supercharge their copilot experience. Whether you need advanced help with writing, coding, designing, researching or learning. 
• Copilot provides a single AI experience that runs across your devices, understanding your context on the web, on your pc, across your apps and soon on your phones to bring the right skills to you when you need them. 
• Copilot is available in Word, Excel, Powerpoint, Outlook, and One Note on PC, Mac and IPAD for Personal and Family subscribers.
• Leveraging GPT4-Turbo model. Access to Enhanced AI Image creator from designer and the ability to build your own CoPilot GPT a customized copilot tailored for your specific topic, in their new building coming soon. 
• For commercial Microsoft 365 they’ve removed the 300 seat purchase minimum, and you can buy from 1-300 users for $30 per person per month. 

1:12:20   Justin - “So basically this means that if you want to Copilot for yourself, but your business didn't want to pay for it because they didn't want to buy 300 seats, you can now pay for it on your own for $20 a month, or you can get the enterprise version which is $30 a month. I picked this up. I said I wanted to try it out and so now I have my own personal laptop which has my own Office 365 personal subscription. I paid the 20 bucks and I've been playing with it and it's not too bad. I actually kind of like how it plugs into Excel and Word and Excel. Definitely something to now go back to the enterprise and say yeah, yeah, we should buy this because it's not as bad as I feared which is good.” 1:17:22 Microsoft named a Leader in the 2023 Gartner® Magic Quadrant™ for Container Management 

• Break out your crystal balls, its Magic Quadrant time! 
• Microsoft must be the only one paying big bucks to Gartner to post the container management magic quadrant. 
• Microsoft, Google, AWS and Red Hat are all within close company in the upper right of the quadrant, with VMware and Alibaba taking a lower spot in the leader quadrant. 

Microsoft  Strengths

• DevOps Platform: GitHub and Microsoft Azure DevOps have large mind share and market share as DevOps platforms. Microsoft’s container services, which are tightly integrated with those offerings, provide an advantage over other vendors.
• Hybrid Cloud: Microsoft offers container service options for hybrid cloud environments. These include AKS, which can be used on Azure Stack HCI, and Azure Arc, which allows users to manage Kubernetes clusters running outside of Azure using Azure Resource Manager.
• Integration With Microsoft Azure: Microsoft container management services integrate well with Microsoft Azure and other services, such as Azure Monitor and Azure Security Center. This brings the significant strengths in the core public cloud over to its container management offerings.

Cautions

• Resiliency: Gartner clients have reported frustrations with outages of Azure Kubernetes Service (AKS), particularly in association with updates and maintenance events.
• Heterogeneous Environments Support: Although Microsoft has an advanced vision of container management in multi cloud heterogeneous environments, it still lacks some capabilities, such as full cluster management on other public clouds or VMware vSphere.
• Certified Container Images: Microsoft has fewer certified container images for common programming languages and frameworks than other leaders in this Magic Quadrant.

GCP Strengths

• Differentiated Portfolio: GCP has the highest number of differentiating features of all the vendors’ products in this Magic Quadrant. Examples of advanced features include managed service mesh (Anthos Service Mesh), cross-cluster networking capabilities, and tools for managing policy and security across fleets of Kubernetes clusters (Anthos Config Management).
• Influence Kubernetes Community: As the creator of Kubernetes and the top vendor by contribution toward Kubernetes, Google has an influential voice in the community and at the forefront in meeting enterprise Kubernetes requirements. Google recently strengthened this position in terms of application platforms by donating Istio and Knative to the Cloud Native Computing Foundation (CNCF).
• Simplicity: Google tends to address requirements with fewer services, rather than adding new services, which helps keep its portfolio simple. This principle is also the basis for the simple user experience of Google’s serverless container services, such as Google Cloud Run.

Cautions

• Edge and On-Premises: GCP has container management products to cover edge and on-premises environments, but it has less adoption and business volume than other leaders in this Magic Quadrant.
• Traditional Enterprise Systems: Gartner believes that GCP has a lower presence in traditional enterprise systems use cases such as data center migrations centered on lift and shift or the modernization of legacy applications, compared to cloud-native and modern application development use cases. These traditional systems are where there are many container-based modernization initiatives.
• Managed Service Provider (MSP) Ecosystem: GCP lags behind in the MSP ecosystem capable of meeting complex enterprise requirements, when compared to other Leaders in this Magic Quadrant. This includes partners that are driving enterprise application modernization

AWS Strengths

• Integration With AWS and Its Ecosystems: AWS’s container services integrate well with other AWS infrastructure and management services and a wide range of AWS partner offerings on AWS Marketplace. This brings the significant strengths in the core public cloud over to its container management offerings.
• Customer Base: AWS’s extensive customer list demonstrates that its two main container services, Amazon ECS and Amazon EKS, are highly trusted in production. From startups that value serverless simplicity to complex enterprises that value standards-based governance, a wide variety of customers are running their critical container applications on AWS.
• Serverless Portfolio: AWS pioneered serverless container services, releasing AWS Fargate in November 2017. It has continued to expand serverless container options so users, from complex large enterprise organizations to single app development teams, can choose one that suits their requirements.

Cautions

• Multi Cloud Support: AWS does not proactively add features or solutions for managing containers in multi cloud environments. Its main multicloud option is to deploy Amazon EKS Distro on other public clouds. This alone is not enough compared to other vendors’ multi cloud capabilities, and EKS Distro is supported only by AWS partners.
• Solution Navigation: AWS has a wide breadth of sometimes overlapping container management offerings. For example, enterprises struggle to differentiate between Amazon EKS and Amazon ECS.
• Kubernetes Cluster Fleet Management: AWS publishes best practices for Kubernetes multi cluster environments, but lacks native fleet-management tools for managing the life cycle and state of distributed clusters

At the end of the day, we’d really like to meet the Gartner people. We have questions. 1:22:04 Microsoft Forms Team to Make Cheaper Generative AI

• Listener note: paywall article
• Microsoft is doubling down on AI that is smaller and cheaper to run than OpenAI’s. 
• The new team is developing the conversation AI to use less computing power.  This is a sign that Microsoft is forging a path not solely reliant on Open AI. 

1:22:54  Ryan - “I think people should be using standard models and lower capacity and not doing as much model training and building as they're doing today. And typically for use cases, there are use cases where that makes sense. And so I think that maybe this would counter that, right? Making it more economical and more sensible for training custom models - if it was smaller cost and hopefully a smaller environmental impact as well.” 1:26:24 Automatic Image Creation using Azure VM Image Builder is now generally available  

• Automatic Image Builder is now GA, using the Azure Image Builder.  This improves your speed and efficiency by allowing you the ability to start image builds for new base images automatically. 
• Automatic image creation is critical for keeping your images up-to-date and secure. It also minimizes the manual steps required for managing individual security and image update requirements. 

1:26:48   Ryan - “Like, this is, it's such a kind of ho-hum feature, but like this is solving a very large problem in businesses. Like it's, it's very easy to let your images go out of date, right? And you can rely on patch Tuesdays to keep them up or, you know, but you know the reality is that it's a little cumbersome to constantly have a fresh image that's available and managed.”

Closing

And that is the week in the cloud! Just a reminder - if you’re interested in joining us as a sponsor, let us know! Check out our website, the home of the Cloud Pod where you can join our newsletter, slack team, send feedback or ask questions at theCloud Pod.net or tweet at us with hashtag #theCloud Pod