Welcome to episode 252 of The Cloud Pod podcast, where the forecast is always cloudy! This week Justin, Jonathan, Ryan, and Matthew are talking about InfluxDB, collabs between AWS and NVIDIA, some personnel changes over at Microsoft, Amazon Timestream, and so much more! Sit back and enjoy – and make sure to hang around for the aftershow, where Linux and DBOS are on the docket. You won’t want to miss it.
Titles we almost went with this week:
- Light a fire under your Big Queries with Spark procedures
- All your NVIDIA GPU belong to AWS
- Thanks, EU for Free Data Transfer for all*
- Microsoft, Inflection, Mufasta, Scar… this is not the Lion King Sequel I expected
- The Cloud Pod sees Inflections in the Timestream
- The Cloud Pod is a palindrome
- The Cloudpod loves SQL so much we made a OS out of it
- Lets run SQL on Kubernetes on Top of DBOS. What could go wrong?
- The Cloud Pod is 5 7 5 long
A big thanks to this week’s sponsor:
We’re sponsorless this week! Interested in sponsoring us and having access to a specialized and targeted market? We’d love to talk to you. Send us an email or hit us up on our Slack Channel. Please. We’re not above begging. Ok. Maybe Ryan is. But the rest of us? Absolutely not.
AI Is Going Great (Or, How ML Makes All Its Money)
1:00 PSYCH! We’re giving this segment a break this week. YOU’RE WELCOME.
AWS
01:08 Anthropic’s Claude 3 Haiku model is now available on Amazon Bedrock
- Last week Claude 3 Sonnet was available on Bedrock, this week Claude 3 Haiku is available on Bedrock.
- The Haiku model is the fastest and most compact mode of the Claude 3 family, designed for near-instant responsiveness and seamless generative AI experiences that mimic human interaction.
- We assume, thanks to how much Amazon is stretching this out, that next week we’ll get Opus.
- Want to check it out for yourself? Head over to the Bedrock console.
02:02 Jonathan – “I haven’t tried Haiku, but I’ve played with Sonnet a lot for pre over the past week. It’s very good. It’s much better conversationally. I mean, I’m not talking about technical things. It’s like I ask all kinds of random philosophical questions or whatever, just to kind of explore what it can do, what it knows…If I was going to spend money on OpenAI or Anthropic, it would be on Anthropic right now.”
04:03 AWS Pi Day 2024: Use your data to power generative AI
04:49 Ryan – “So what’s awesome about that CSI driver is that we can run a SQL server in Kubernetes with the files being stored in S3 for all that. It’ll be awesome!”
07:41 Run and manage open source InfluxDB databases with Amazon Timestream
-
- You can now use InfluxDB as the database engine behind Amazon Timestream.
- This support makes it easy for you to run near real-time time-series applications using Influxdb and open source APIs, including open source telegraf agents that collect time-series observations.
- This is the second engine supported in Timestream, with the first now being known as Timestream for LiveAnalytics
- Amazon says you should use InfluxDB engine if your use case requires near real-time time-series queries or specific features in InfluxDB, such as flux queries.
- If you are needing to ingest more than tens of gigabytes of time-series data per minute and run SQL queries on petabytes of time-series data in seconds they recommend Timestream for LiveAnalytics.
- “The future of open source is powered by the public cloud—reaching the broadest community through simple entry points and practical user experience. Amazon Timestream for InfluxDB delivers on that vision. Our partnership with AWS turns InfluxDB open source into a force multiplier for real-time insights on time-series data, making it easier than ever for developers to build and scale their time-series workloads on AWS.” — Paul Dix, Founder and CTO at InfluxData
- Pricing between the two models, LiveAnalytics, you pay 0.50 per 1 Million writes of 1KB size, 0.01 per GB scanned in queries, 0.036 per GB stored per hour in memory and 0.03 for magnetic storage.
- You also get a free tier which will allow you to receive 50gb of ingestion, 100GB of magnetic tier storage, 750GB-HR of memory storage and 750GB for query usage.
- Vs Influx model. Single AZ or Multi-AZ deployment option. Two dimensions database instances and database storage. Influx Multi AZ goes from 1×8 to 64×512. 0.239 per hour vs 15.30 per hour. Data storage for 3000 iops minimum of 20gB at 0.20/gb/month, 12k with a minimum of 400gb at 0.70/gb/month and 16iops at 400gb minimum 1.00/gb/month
- Want more information on Timestream pricing? Find it here.
10:24 Matthew – “The question is, do they even have the InfluxDB in the Amazon calculator? Because in the past, it’s always been very delayed.”
10:35 Justin – “I’m sort of surprised they went with live analytics versus serverless. Because what they’re describing is basically a time stream serverless server. Because you don’t have to worry about servers, you just worry about compute and storage and things in memory. But apparently they decided not to use the serverless moniker for live analytics.”
15:08 AWS and NVIDIA Extend Collaboration to Advance Generative AI Innovation
-
- AWS and NVIDIA are announcing that the new NVIDIA Blackwell GPU platform, unveiled by NVIDIA at GTC 20204, is coming to AWS. Get ready to spend ALL your money!
- AWS will offer the NVIDIA GB200 Grace Blackwell Superchip and B100 Tensor Core GPUs, extending the company’s long standing strategic collaboration to deliver the most secure and advanced infrastructure, software and services to help customers unlock new generative AI capabilities.
- “The deep collaboration between our two organizations goes back more than 13 years, when together we launched the world’s first GPU cloud instance on AWS, and today we offer the widest range of NVIDIA GPU solutions for customers,” said Adam Selipsky, CEO at AWS. “NVIDIA’s next-generation Grace Blackwell processor marks a significant step forward in generative AI and GPU computing. When combined with AWS’s powerful Elastic Fabric Adapter Networking, Amazon EC2 UltraClusters’ hyper-scale clustering, and our unique Nitro system’s advanced virtualization and security capabilities, we make it possible for customers to build and run multi-trillion parameter large language models faster, at massive scale, and more securely than anywhere else. Together, we continue to innovate to make AWS the best place to run NVIDIA GPUs in the cloud.”
- AWS will offer these new GPUs later this year vai their EC2 Ultra Clusters and Amazon EC2 Capacity Blocks for ML offerings.
- GB200 will also be available on NVIDIA DGX cloud, an AI platform co-engineering on AWS, that gives enterprise developers dedicated access to the infrastructure and software needed to build and deploy advanced generative AI models.
17:30 Justin – “I am intrigued by the fact that this thing is going to run a multi-trillion parameter large language models. And all I can think about is the cash register is going brrrr. Because I don’t think we even have a trillion parameter large language model that’s publicly available that I’m aware of… but a multi-trillion one is even more fascinating to me… actually, I just did a Google search for it live, real-time fill-up. So January 8th, 2024, there was an article in DataCenter Dynamics – which I don’t know this website. “Frontier supercomputer trains one trillion parameter LM and just over 3000 GPUs’ and it says ‘researchers at Oak Ridge National Laboratory’ and I’m like, oh yes, okay, thank you FBI, CIA for letting us know you have this, appreciate it.”
GCP
19:07 Jonathan Does a Thing – Google Support
Listener alert: Major vent session re Google customer service.
25:54 Announcing SQL Server Reporting Services (SSRS) in Google Cloud SQL
- You can now build and run SSRS reports with databases hosted on Cloud SQL for SQL Server, including the report server database required to setup and run SSRS.
26:25 Matthew – “Fun fact, you can’t run SSRS in Microsoft SQL Managed Service.”
27:27 Google named a Leader in The Forrester Wave: AI Infrastructure Solutions, Q1 2024
- Forrester Research has published their latest Forrester Wave: AI Infrastructure Solutions, Q1 2024.
- Google is so far in the lead due to their “vision and strong track record of delivering continuous innovation and leading AI infrastructure products.”
- “Google has strengths across the board with the highest scores of all the vendors in this evaluation.” – The Forrester Wave: AI Infrastructure Solutions, Q1 2024
- Vendor Positions
- Leaders: Google, AWS, Microsoft, NVIDIA, Dell
- Strong Performers: IBM and HPE
- Contenders: Alibaba Cloud, Lenovo and Oracle
- Challengers: Graphcore and Cerebras Systems
- Google offers the whole package for AI workloads.
- AI continues to be a core capability of Google’s many consumer and business services, such as internet search and advertising. So to say Google has a head-start is an understatement. Doing AI efficiently at Google-scale is a feat that few other companies in the world are capable of. Google brings that experience and infrastructure to Google Cloud AI infrastructure. Google’s early and ongoing investments in AI for its other businesses drives its vision of “where the puck is going to be” for enterprise AI. Google’s superior roadmap and innovation is to make Google-scale accessible to all customers, whether a bright tiny startup or a large global enterprise, while at the same time abstracting the complexity with easy-to-use tools.
- Microsoft makes supercomputer AI infrastructure easy to use at cloud scale. Microsoft offers numerous sizes of GPU-optimized virtual machines for direct use. The Azure AI portfolio offers several AI-centric services, such as Azure OpenAI Service and Azure AI Studio, to help customers develop custom AI applications that use Microsoft’s underlying AI infrastructure. Microsoft’s strategy is to bring AI to every application, every business process, and every employee. Microsoft plans to achieve this through a combination of business and productivity applications and by making Microsoft Azure AI infrastructure attractive for AI developers. Its $13 billion investment in OpenAI adds proof to the pudding. The company’s superior innovation and roadmap is driven by infusion of AI into all of the company’s business applications, developer tools, and cloud services.
- Amazon Web Services (AWS) is your one-stop AI shop with a wide range of options. AWS’s AI infrastructure portfolio is extensive. AWS’s vision is to offer customers a wide range of options to run AI workloads from preconfigured instances to training services abstracted behind its AI development tool — Amazon SageMaker. Amazon’s AI strategic infrastructure portfolio includes expected compute instances/virtual machines based on NVIDIA GPUs, but also instances based on Intel’s Gaudi chips. AWS also offers AI infrastructure based on its own chips: AWS Inferentia for inferencing and AWS Trainium for training. Additional services include AWS Neuron SDK to make it easy to use AWS’s custom chips, AWS Elastic Inference to optimize cost/performance, and AWS IoT Greengrass for edge inferencing.
- Oracle offers cost effective AI infrastructure but needs more tooling. Oracle has emerged as an attractive cloud AI infrastructure provider because it has a mature public cloud, a breadth of complementary AI services, and the hardware horsepower to back it up. In addition, because of its huge enterprise application business, enterprises already have plenty of training data in the Oracle Cloud. Oracle’s strategy is to be a cost-effective alternative to the major cloud service providers. Oracle can improve its strategy by greatly expanding its AI infrastructure vision beyond raw GPU instances to why enterprises should consider a long-term, strategic relationship with Oracle Cloud. Oracle can improve its roadmap with enhancements to its AI-specific development tools.
- Want to download the report for yourself? Of course you do! You can do it here.
32:20 Unify analytics with Spark procedures in BigQuery, now generally available
- BigQuery is a highly scalable and capable SQL engine, and Google will tell you it’s the best. However, you may want to leverage open-source Apache Spark expertise or existing spark based business logic to expand BigQuey data processing beyond SQL.
- For example like a package to handle complex JSON processing or graph data processing, or use legacy code that was written in spark before you migrated to BigQuery. Historically, this would require you to leave BigQuery, enable a separate API, use an alternative UI, manage disparate permissions and pay for a Non_BQ SKU.
- To fix all of this, they are extending BigQuery data processing to Apache Spark and announcing the GA of Apache Spark Stored Procedures in BigQuery.
- BigQuery users looking to extend their queries with Spark-based data processing can now use BigQuery APIs to create and execute Spark stored procedures. It brings Spark together with BigQuery under a single experience, including management, security, and billing.
33:47 Jonathan – “What’s cool though is it’s not just like SQL sort procedures. You can actually write code in a sensible language. So you can write sort procedures in Python if you want.”
Azure
36:03 Now available: Free data transfer out to internet when leaving Azure
- Microsoft supports customer choice, including the choice to migrate your data away from Azure
- They now offer a credit for over 100 GB (free tier) if you move to another cloud or on-premise.
- Thanks, European Data Act!
- Just like AWS, you have to open a Support case, you indicate when you will start and have 60 days, and then you need to cancel all subscriptions associated with the account and then request the invoice credit.
- There are some fine print details. You must provide advance notice and cancel all Azure Subscriptions associated with your account after your data is transferred out before you can request your invoice-level credit.
- Standard charges for Azure services and data transfer from specialized services, including Express Route, VPN, Azure Front Door and Azure CDN, aren’t included.
- If you buy Azure through a partner, the partner is responsible for giving you the credit.
37:26 Matthew – “ It’s confusing. Also, does your 0365, if you still left your 0365 there, where’s that live? Right, so do you have to cancel your entire 0365 data and then your SharePoint and Teams? Like, and then I’m sitting here going like, okay, through Azure CDN or front door, so I’m gonna post all my private data in a front door bucket…And then what? Download it through Azure front door, but that defeats the purpose of CDN if I’m willing to download it once. So I have many questions about why they threw that one in there.”
39:35 AKS Updates
41:10 Microsoft promises Copilot will be a ‘moneymaker’ in the long term
- Microsoft is telling investors to cook their jets on quick financial returns from Copilot.
- Testers told a wall street journal that they had mixed feelings about their usage of co-pilot and if it justified the price tag yet.
- Juniper Networks CIO Sharon Mandell told the paper they aren’t ready to spend 30 per user yet.
- Jared Spataro, CVP of Modern Work and Business Applications at MS, said that various applications are in different stages of development, and that it is most effective when sophisticated information retrieval and sophisticated task completion is available in three areas: Office, Teams and Outlook. And When more of these are finished with development the price point will be well justified.
42:00 Ryan – “I think the problem I have with this is the user model. I just don’t know if that’s the right model for this, because it does sort of just burn you up. And you want to make this a tool that’s available. It is not something that you can clearly demonstrate a return on any kind of value yet.”
45:30 Mustafa Suleyman, DeepMind and Inflection Co-founder, joins Microsoft to lead Copilot
- Microsoft has hired Mustafa Suleyman and Karen Simonyan to join a new organization called Microsoft AI, focused on advancing Copilot and their other consumer AI products and research,
- Mustafa will be EVP and CEO Microsoft AI and report directly to Satya Nadella. Karen is joining the group as chief scientist, reporting to Mustafa.
- Mustafa is the founder of both DeepMind and Inflection and is a visionary product maker and builder of pioneering teams.
- Karen was co-founder at Inflection and was the lead in developing some of the largest AI breakthroughs over the past decade including AlphaZero.
- Several other members of the inflection team have joined as well.
- The Copilot, Bing Edge Team, and Gen AI team will all now report to Mustafa.
- Kevin Scott, the CTO and EVP of AI, is responsible for all AI strategy, including all system architecture decisions, partnerships and cross-company orchestration will continue in his current role
- Inflection AI is apparently abandoning its ChatGPT challenger as it announces its new CEO, Mustafa, joining Microsoft. They had previously announced and were selling Pi; their conversational LLM. It’s unclear what their pivot will be.
48:42 Microsoft open sources Retina: A cloud-native container networking observability platform
- Microsoft is open sourcing a container network observability tool called Retina. The tool provides actionable network insights for cloud-native applications and helps you troubleshoot latency, packet drops and many more, its non-intrusive and easy to use and supports diverse environments. (As long as they’re in containers.)
49:00 Ryan – “Cool, so like just go a whole different way with your observability platform than everyone else and well, because you had to, because that’s the only thing that’ll support Windows containers. All right!”
Aftershow
49:18 What if the operating system is the problem’: Linux was never created for the cloud — so engineers developed DBOS, a new operating system that is part OS, part database
Meet DBOS: A Database Alternative to Kubernetes
- Turing Award Laureate Dr. Mike Stonebraker loves to invent databases being one of the investors of the first relational system ingress 40 years ago, followed by Postgres SQL 30 years ago, and more recently he co-created an in-memory transactional database called VoltDB. And now his latest startup is looking to replace the entire cloud native computing stack with DBOS (Database Operating System)
- The claim is Linux is too old, and K8 is too complicated, and that a database can replace all of them.
- DBOS, Inc has raised 8.5 M to fund Dr Stonebraker and Apache Spark Creator (and Databricks co-founder and CTO) Matei Zaharia and a joint team from MIT and Stanford to create DBOS
- DBOS runs the operating system services on top of a high-performance distributed database. All state, logs, and other system data are stored in SQL-accessible table.s
- The result is a scalable, fault tolerant, and cyber-resilient serverless compute cloud for cloud native apps
- With an OS on top of a distributed database you get fault tolerance, multi-node scaling and state management. Observability and Security gets easier.
- Today distributed systems are largely built on an OS designed to run on a single server.
- In the DBOS design, a high-performance distributed OLTP would implement a suite of OS services .
- It would run a minimal OS kernel, with support for memory management, device drivers, interrupt handlers and basic tasks of byte management.
- Initially they built the database on VoltDB, but the backers wanted an to go with an open source key-value system instead, so they went with FoundationDB as the base.
- The first commercial service built around DBOS cloud, a FaaS platform, available for developers, built on top of AWS Firecracker, which is available for developers to experience via DBOS cloud, which they launched.
Closing
And that is the week in the cloud! Just a reminder – if you’re interested in joining us as a sponsor, let us know! Check out our website, the home of the Cloud Pod where you can join our newsletter, slack team, send feedback or ask questions at theCloud Pod.net or tweet at us with hashtag #theCloud Pod
