AWS Storage Services

heartin's picture

S3 Endpoints and APIs

Will list down imporant notes on S3 APIs here.


“dual-stack” endpoint and IPv6

  1. The “dual-stack” endpoint supports access over both IPv4 and IPv6. In most cases, no further configuration is required for access over IPv6, because most network clients prefer IPv6 addresses by default.

heartin's picture

Lifecycle Management Policies

S3 Lifecycle management provides the ability to define the lifecycle of your object, mainly to reduce your cost of storage. For example, you can set infrequently accessed objects to move into lower cost storage tier (like S3-IA) after a period of time. After another period, it can be moved into Amazon Glacier for archive and compliance, and eventually deleted.

You can set lifecycle transition policy to:

heartin's picture

S3 URL Signing and Pre Signed URLs

A signed URL includes additional information such as an expiration date and time that gives you more control over access to your content. This additional information appears in a policy statement, which is based on either a canned policy or a custom policy.

Custom policy provides more flexibility over canned as it allows reuse the policy, and to specify date and time that users can begin to access and IP address or range of IP addresses of the users who can access, and includes a base64-encoded version of the policy.

heartin's picture

S3 Popular Features - Static Website, CORS, CRR, Multipart Upload

We will discuss some of the features of S3. Other features may have a separate page of its own.


Static Website Hosting

You can host your entire static website on Amazon S3 for an inexpensive, highly available hosting solution that scales automatically to meet traffic demands. Static content ideal include html files, images, videos, and client-side scripts such as JavaScript. 

heartin's picture

S3 Pricing, Policies and Limits

We will discuss some of the policies of S3. Other policies may have a separate page of its own.


S3 Pricing

S3 charges you for following:

  • Data storage (monthly). Is different for different region.

  • Requests to the buckets and objects

  • All data transferred out (e.g. downloads), but no charges for data coming in (e.g. uploads).

heartin's picture

AWS Glacier Overview

Glacier is a secure, durable, and extremely low-cost cloud storage service for data archiving and long-term backup. Customers can reliably store large or small amounts of data for as little as $0.004 per gigabyte per month.


Glacier Retrieval Process

  1. To retrieve Amazon S3 data stored in Amazon Glacier, initiate a retrieval request using the Amazon S3 APIs or the Amazon S3 Management Console.

heartin's picture

Amazon S3 Versioning and MFA Delete

Once you enable Versioning for a bucket, Amazon S3 preserves existing objects anytime you perform a PUT, POST, COPY, or DELETE operation on them. You can enable versioning while creating an object or from Properties tab of the bucket.

Important notes on versioning:

  1. The bucket owner, the AWS account that created the bucket (root account), and all authorized IAM users can enable versioning.

heartin's picture

Additional S3 Features - Tagging, Analytics, Inventory, Metrics, Events

S3 Analytics, S3 Inventory, and S3 CloudWatch metrics can be found under “Management” tab in the S3 Console. Tags and Cross Region Replication (CRR) can be found under advanced settings under Properties tab. As CRR is a very common use case, we will also have a lab for it. Lifecycle rules are discussed in detail here.


heartin's picture

Amazon S3 Tired Storage - Standard S3, S3-IA, RRS, Glacier

S3 support tiered storage and has lifecycle management to decide what data sits on which tier. 


S3 (regular S3)

  • Amazon guarantees an SLA of 99.99% availability and 99.999999999% (11 9s) durability for Standard S3.

    • Note: Durability means the chance of not losing your data.

  • Is the default.


S3 – IA (Infrequently Access)

  • For data that is used less frequently but needs faster access when needed.

heartin's picture

Using Amazon EFS with Amazon EC2

Amazon Elastic File System (Amazon EFS) is a simple, scalable file storage for use with Amazon EC2 instances in the AWS Cloud.  With Amazon EFS, storage capacity is elastic, growing and shrinking automatically as you add and remove files.