220: The Cloud Pod Read Llama Llama Red Pajama

Welcome episode 220 of The Cloud Pod podcast - where the forecast is always cloudy! This week your hosts, Justin, Jonathan, Ryan, and Matthew discuss all things cloud, including virtual machines, an AI partnership between Microsoft and Meta for Llama 2, Lambda functions, Fargate, and lots of security updates including the Outlook breach and WORM protections. This and much more in our newest episode. 

Titles we almost went with this week:

• Too Many Bees died for Honeycode
• Microsoft announces that AI will only cost you 3 arms and a leg.  
• The Cloud Pod also detects Recursive Loops in cloud news
• The cloud pod disables health checks bc who needs them

A big thanks to this week’s sponsor:

Foghorn Consulting, provides top-notch cloud and DevOps engineers to the world’s most innovative companies. Initiatives stalled because you have trouble hiring?  Foghorn can be burning down your DevOps and Cloud backlogs as soon as next week.

News this Week:

AWS

02:02 Detecting and stopping recursive loops in AWS Lambda functions

• Do you utilize AWS Lambda? Here’s an update for you. 
• AWS Lambda is introducing a recursion control to detect and stop lambda functions running in a recursive or infinite loop.  
• This supports Lambda Integrations with SQS, SNS or directly via the Invoke API.  
• Lambda defects functions that appear to be running in a recursive loop and drops the request after exceeding 16 invocations
• This can help reduce costs from an unexpected lambda invocation because of recursion. 
• You’ll receive notification that this action was taken through the AWS Health Dashbboard, email or by configuring Amazon Cloudwatch Alarms. 
• You can turn this off by reaching out to AWS support, if you have a valid use-case where recursion is intentional, or if you need to loop something through more than 16 times.
• This is also the trap - if you say turn it off and then cry about a ridiculous bill due to your runaway recursion - they will now force you to pay it. So, listeners beware.

03:50 Matt- “I can definitely say I’ve caused an ‘in the hundreds of dollars’ very rapidly by this in the past in a dev account. So it's definitely something that's easy to do if you are doing recursion and you make an ‘if’ statement the wrong way.” 04:28 AWS Fargate Enables Faster Container Startup using Seekable OCI

• Are you a Fargate user who has been jealous of all those folks using ECS who have been able to utilize the seekable OCI or Sochi capability of lazy loading of containers? Well pine away no more! This feature is now available to you! 
• As you most likely know, AWS last year started supporting lazy loading of containers via the Seekable OCI (SOCI) technology. 
  • This was due to research that said image downloads accounted for 76% of container startup time, but on average only 6.4% of data is needed for the container to start and do useful work.  
• Now this feature is coming to AWS Fargate, which will help your application deploy and scale out faster by enabling containers to start without waiting to download the entire Container Image. 
• As of launch, you can now use it for both Fargate ECS as well as Fargate Naturally and ECS Compute. 
• Note that supporting this capability does require you to build a SOCI index for the container image.  
  • Amazon has made this part easier, however, with a SOCI index builder. 
  • This is a serverless solution for indexing container images in AWS.  
  • If you like you can also create the SOCI indexes manually via the SOCI CLI provided by the soci-snapshotter project. 

06:27 Justin- “I suspect this is a big issue if you're doing data learning sets and containers, right? So you need to load up a large amount of data set into the container, to basically then be able to train the model, but you know, you can start training the model on a subset of the data; you don't need the full thing to be loaded. And so I suspect that's really where the use case of this comes into play - in big data training and AI training.” 07:56  Amazon FSx for NetApp ONTAP Now Supports WORM Protection for Regulatory Compliance and Ransomware Protection   

• FSX for Netapp OnTap now supports Snaplock, an ONTAP feature that gives you the power to create volumes that provide Write Once Read Many (WORM) functionality.  
  • (Or as we refer to it… how to turn your SAN into a paperweight and use it with care).  
• Snaplock volumes prevent the modification or deletion of files within a specified retention period, and can be used to meet regulatory requirements and to protect business-critical data from ransomware attacks and other malicious attempts at alteration or deletion.
• FSx for OnTAP is the only cloud-based file system that supports Snaplock, and the ability to move Snaplock data to lower costs cloud storage. *For now.* 

08:38Jonathan - “This is kind of a can of worms really. I see the advantage of protecting against ransomware, but also customers or consumers have a right to have the data deleted. So what happens if your data is on a worm drive with a policy that says it can't be deleted, but the regulatory requirements say that you have to delete customer data.” 20:29 Announcing AWS Fault Injection Simulator new features for Amazon ECS workloads AWS Fault Injection Simulator supports chaos engineering experiments on Amazon EKS Pods

• AWS Fault Injection Simulator now supports Chaos experiments for EKS and ECS workloads. 
• We’re not sure just how much ADDITIONAL chaos you want to add to your containers, but now you have options! 
• ECS Actions Supported
  • Task-cpu-stress
  • Task-io-stress
  • Task-kill-process
  • Task-network-blackhole-port
  • Task-network-latency
  • Task-network-packet-loss
• EKS supports all of the same actions. 

11:02 Jonathan - “We don't need to inject defects. We have plenty of our own.” 12:42 Ryan - “Yeah, other than the basics of fault injection when it first came out, I don't think I've really used it since because like you said - I *wish* I could get to a level where I maintain application to a level where I'm like, yeah, I'm gonna make it really hardened and resilient.”  13:39 The future of Amazon's Honeycode cloud service is not looking so sweet 

• Filed under news we’re not at all shocked about, Honeycode may be on its last legs. 
• Originally launched in 2020, Honeycode was supposed to be the answer to AWS “low-code” development, which uses a simple drag and drop interface to help users easily build apps without advanced software engineering skills. 
• Amazon is currently providing bare minimum support for Honeycode, with no active promotions or sales activities for the app, according to people familiar with the matter. 
• It's basically a KTLO product, joining other services like Workdocs, Workmail and SimpleDB. 
• Amazon has great infrastructure as code, but has struggled with SaaS apps (outside of Connect IMO).  With competing products for things like Dropbox, Slack, Tableau with only marginal success. 
• Interestingly enough, Honeycode was a high profile project when it launched.  
• Honeycode is still listed in the Amazon Directory but appears to have been absorbed by the new Next Generation Developer Experience team, which is focused on Generative AI. 

GCP

15:48 Document AI introduces powerful new Custom Document Splitter to automate document processing

• Google is focusing a lot on documents this year, with the launch of Custom Document extractor in February, and Custom Document Classifier in March. 
• Now they are announcing the latest feature in Document AI Workbench: the Custom Document Splitter. This will help users automatically split and classify multiple documents in a single file. 
• CDS allows customers to sort and classify their documents. 
  • For example, businesses can validate if they have all the needed documents for an applicant.  
  • Furthermore, individually classified documents can help automate other downstream processes. 
  • The goal is to help businesses lower their documenting time and costs. 

17:52 Ryan - “In the pre-show I was talking about my expense report, and having to basically give the top page that has the account summary, but I don't really want all my individual cell phone transactions. And so being able to do stuff like that - automatically pre-processing, where you're splitting that up and not storing ages and ages of ‘this page intentionally left blank’ in your cloud storage is probably a pretty good idea.”

Azure

17:49 Hotpatch is now generally available on Windows Server VMs on Azure with the Desktop Experience installation mode 

• Hotpatch is now available for Windows Server Azure Edition VMs with Desktop Experience installation mode using the newly released image. 
• Hotpatch is a feature that allows you to patch and install OS security updates on Windows Server Azure Edition Virtual machines on Azure without requiring a reboot.
  • Justin has a problem with this assertion, however…
• Apparently, previously available only for Server Core Installations (with no GUI), now, they can do it with a full GUI every month. 
• Benefits
  • Lower workload impact with fewer reboots (allegedly)
  • Faster deployments of updates as the packages are smaller, install faster, and have easier patch orchestration with Azure Update Manager (allegedly)
  • Better protection, as the Hotpatch update packages are scoped to Windows security updates that install faster without rebooting (allegedly)

19:09 Matthew- “I prefer not to ever log into my servers, ever deal with them in any way, shape or form. If there is a patch, the windows auto OS update feature, I don't know what the official name is on Azure for it, but it literally just takes care of it for you in the scale sets. You don't have to deal with it. Works great. Why do I need to actually patch local servers? I prefer not to do this… That is why I pay Microsoft to write it for me.” 19:41 Ryan - “Well, with improvements like this, like Azure is going to be the only place to host Windows workloads, right? Because it's all the gripes with Windows. You're like, well, why would I run this another cloud provider? I have to reboot it every five minutes.” 20:07 Always Serve for Azure Traffic Manager

• So you hotpatched your server and now need a reboot. 
• Now you can now use Always Serve for Azure Traffic Manager! 
• You can disable endpoint health checks from an ATM profile and always serve traffic to that given endpoint. You can also now choose to use 3rd party health check tools to determine endpoint health, and ATM native health checks can be disabled, allowing flexible health check setups. 

20:55 Jonathan - “It’s a pretty decent feature, actually. It seems weird to remove health checks, but what they're providing is a way to plug in your own health check infrastructure. So if you need something more complex than just a REST call or a web call that gets 200 or 500 back, then you can build something a lot more complex that runs much better tests, and then plug that into the load balancer.” 21:24 Justin - “It's a lot of heavy lifting for me to now pull this all into APIs where… why don't you just give me the ability to run a custom health check as the health check through serverless, and then based on the output of what I give you, you can then do different scale set operations. Why completely divorce yourself from the responsibility and say, now you have a third party that's responsible. We're off the hook, when you could have given me a system that allows me to run my own code to do health checks.” 23:56 Microsoft and Meta expand their AI partnership with Llama 2 on Azure and Windows

• Microsoft is doing all the AI things, and now announced support for the Llama 2 family of LLMs on Azure and Windows. 
• LLama2 is designed to enable developers and organizations to build generative AI powered tools and experiences. 
• Meta and Microsot share a commitment to democratizing AI and its benefits, and they are excited that Meta is taking an open approach with Llama 2. 

23:56 Furthering our AI ambitions – Announcing Bing Chat Enterprise and Microsoft 365 Copilot pricing

• Microsoft Inspire has announced Bing Chat Enterprise and Microsoft 365 Copilot pricing.
• Bing chat enterprise delivers an AI-powered chat for the workspace, rolling out in preview to over 160 million people.
• Also, for budgeting, you should know that Copilot is going to cost you $30 per user per month on top of your MS365 E3, E5, Business Standard and Business Premium customers.  
• Timing will be shared soon. We’re on pins and needles over here. 

24:46 UPDATE: Analysis of Storm-0558 techniques for unauthorized email access

• Storm-0558 acquired an inactive MSA consumer signing key and used it to forge authentication tokens for Azure AD enterprise and MSA consumer to access OWA and Outlook.com. All MSA keys active prior to the incident – including the actor-acquired MSA signing key – have been invalidated. Azure AD keys were not impacted. 
• The method by which the actor acquired the key is a matter of ongoing investigation. 
• Though the key was intended only for MSA accounts, a validation issue allowed this key to be trusted for signing Azure AD tokens. 
• This issue has been corrected. (Allegedly)
• The big questions remain: what did they steal and how did they steal it? 

26:09 Justin - “So they fixed the root, which is good, but they still don't actually know how they got the acquired the key or at least they've not publicly announced how the packer got the key that was used and the whole thing. So this is not great, but I appreciate the thoroughness of this writeup versus the original document. And I do hope they answer the final piece of the puzzle. So we all. feel maybe a little better or a little worse. I'm not sure how I feel.”

Closing

And that is the week in the cloud! We would like to thank our sponsors Foghorn Consulting. Check out our website, the home of the cloud pod where you can join our newsletter, slack team, send feedback or ask questions at thecloudpod.net or tweet at us with hashtag #thecloudpod