Onica’s Announcements At A Glance series analyzes the latest AWS news and announcements, simplifying and explaining the significance for AWS consumers.
May was a packed month in the AWS universe, with many new announcements and feature updates that were rolled out across the platform. Each month, Onica produces a list of announcements that is curated specifically for enterprise users. This month, we will cover a new feature for Amazon S3 called Batch Operations, a path deprecation announcement for Amazon S3, AWS Transit Gateway announcement for Direct Connect, Amazon CloudWatch Container Insights preview, and a public IP address announcement for Amazon EKS. Let’s dive in!
Amazon S3 Batch Operations
The name gives it away! Ever wished there was an easy way to replace all of the tags on all objects in a bucket in a batch? How about a simple way to run a Lambda function against all objects in a bucket to process them in some custom way? What about restoring from Amazon Glacier? The newly announced Amazon S3 Batch Operations tool empowers you to do it!
You can either use an Amazon S3 inventory report or a custom manifest file in CSV format to specify which objects to target by the batch operation. You first select the path to the manifest file as well as the path to its location in Amazon S3. You can even specify a version to target if you have versioning enabled in Amazon S3. Once you’ve selected the above to target the specific objects you want to operate on, you can choose your operation. Below is a screen shot of the operations currently available in the console:
The availability of the “Invoke AWS Lambda function” operation extends the type of operation you can perform in batch to any type of function you can code, which offers endless possibilities to this utility. You can even give the job you are creating a priority ranking. The highest number priority job in your account runs first. The second highest runs second, and so on. With no intrinsic meaning to Amazon S3 itself, this allows you to order job priority across your account for all batch operations. After submission the job runs some brief verification of the manifest file and enters a state of “Awaiting Your Confirmation.” You then can review the confirmation to understand what actions the batch operation will perform and click “Run Job.” The job enters a ready state and starts to run. When it is finished, it enters a “Complete” state.
You can also configure the operation to generate a completion report to get information on error codes, outputs, and descriptions. These reports are encrypted using SSE-S3. You can choose to report on only failed tasks or all tasks run. The operation like all other AWS services is governed by IAM. So your operation will need the appropriate IAM policy to execute.
For more information, click here.
AWS Transit Gateway Direct Connect Support
A brief history:
First came the Amazon VPC. Very quickly, the need arose for Amazon VPC assets to communicate directly with one another. You could spin your own VPN servers to connect them and many did. But this resulted in the inheritance of the technical overhead for those servers. Enter the VPC peering connection in 2014. Finally, a simple AWS managed way for internal VPC assets to communicate with each other. As enterprises grew their environments, the need arose to have transitive communication for VPC’s. One example would be a development VPC having to authenticate against a Domain Controller in a shared services VPC before accessing production resources, such as a staging environment. This gave rise to AWS Transit VPC in 2016. This was a great way to create a hub and spoke design for VPC’s, allowing for transitive routing in AWS. But it resulted in more overhead since the Transit VPC solution had Amazon EC2 routers at its core. Enter AWS Transit Gateway. Finally, an AWS managed service that would allow transitive routing in a hub and spoke design across multiple accounts to connect VPC’s. But what about AWS Direct Connect?
May brought the announcement of AWS Transit Gateway support for AWS Direct Connect. This allows applications that you run in AWS to communicate with each other, and with your on-premises applications, at speeds of up to 10 Gbps per Direct Connect connection, leveraging the aforementioned hub and spoke architecture. This can simplify your network architecture and management overhead, as well as potentially reduce the number of VPN connections in your environment. For those currently using AWS Direct Connect, you may be able to consolidate the number of dedicated or hosted connections as well. For more information, check out the announcement here.
Amazon S3 Path Deprecation
An important announcement was made concerning Amazon S3 paths in May that won’t officially roll out until September 2020, but needs to be on your radar now. Take note of the two Amazon S3 addressing models below:
- Path based URI – https://s3.amazonaws.com/mybucket
- Virtual Hosted Style URI – https://mybucket.s3.amazonaws.com
One is a path style that uses the bucket name in the path. The second is the virtual hosted style, where the bucket name is used at the start of the URI. As of September 2020, the path based style will be deprecated and only the virtual hosted style will be allowed. Because the subdomain model is more distributed and more specific to the name of your bucket, it will be easier to protect as well as more scalable. What does it mean for you? Applications that currently use the path based style will need to be updated to use the virtual hosted style bucket name to continue to have access to your bucket. There is still some time to get the change implemented, but it will be here before you know it! You can get more details here.
Amazon CloudWatch Container Insights Preview
When Amazon CloudWatch Container Insights preview was announced in May, it enabled DevOps engineers and system administrators insights into their container based applications and microservices using Amazon CloudWatch. This allws for the creation of Amazon CloudWatch automated dashboards to summarize the health and performance of Amazon EKS and Kubernetes clusters by pod, node, namespace, and services. You can monitor metrics such as CPU, memory, disk usage, network I/O, as well as diagnostics such as container restart failures. This utility is in preview in various regions around the world. You can find out which regions, as well as learn more at the documentation link found here.
Amazon EKS Supports Public IP’s
Another notable announcement for enterprises is the support of public IP’s in a VPC for Amazon EKS. This is good news for those who need to utilize an AWS owned public IP, or who need to bring one that they own to the cloud for their Amazon EKS clusters. You can find out more here.
Ready to revolutionize your business with AWS? Get in touch!
