AWS CLI has go a life saver when you want to manage your AWS infrastructure efficiently.

This also provides lot of possibilities for Automation and reduce the number of times that you accept to login to AWS Management panel.

If you are new to AWS CLI. Please consider reading this article on how to setup AWS CLI

With no further ado, Lets become into the objective of this post.

aws s3 cp

In this post we are going to talk about a very specific command of AWS CLI which is AWS S3

While AWS S3 provides complete tool set to manage your S3 bucket. we are going to see one specific characteristic of S3 CLI today. which is re-create

Before nosotros get there. we need to understand few things virtually AWS S3 CLI

There are ii AWS S3 CLI commands bachelor

  • AWS S3
  • AWS S3API

Allow the states run across what they both have to offer.

What is AWS S3 and S3API

The AWS CLI provides two tiers of commands for accessing Amazon S3

The s3 tier consists of high-level commands that simplify performing common tasks, such as creating, manipulating, and deleting objects and buckets.

The s3api tier behaves identically to the same S3 tier simply information technology enables you to carry out advanced operations that might non be possible with s3 tier.

In this article we are going to talk near but the s3 tier and very specifically s3 cp command which helps united states of america copying files from and to S3 buckets.

Before going any further I want you lot to know few handy commands which help to listing the buckets.

  1. aws s3 help – To get a list of all of the commands available in high-level commands.
  2. aws s3 ls – To become the list of all buckets.
  3. aws s3 ls s3://bucket-name – Will list all the objects and folders I that saucepan.
  4. aws s3 ls s3://saucepan-proper noun/path/ – This command will filter the output to a specific prefix.

Quick Caveats on AWS S3 CP control

  • Copying a file from S3 bucket to local is considered or chosen equally download
  • Copying a file from Local system to S3 saucepan is considered or chosen as upload
  • Please be warned that failed uploads can't exist resumed
  • If the multipart upload fails due to a timeout or is manually cancelled by pressing CTRL + C, the AWS CLI cleans up whatsoever files created and aborts the upload. This process can take several minutes.
  • If the process is interrupted past a kill command or organization failure, the in-progress multipart upload remains in Amazon S3 and must be cleaned up manually in the AWS Management Panel or with the s3api abort-multipart-upload command.

AWS S3 CP Command examples

Hither we have listed few examples on how to apply AWS S3 CP command to copy files.

Copying a local file to S3

Uploading a file to S3, in other words copying a file from your local file system to S3, is done with aws s3 cp command

Let'southward suppose that your file name is file.txt  and this is how you can upload your file to S3

aws s3 cp file.txt s3://bucket-name

while executed the output of that command would like something like this.

aws s3 cp

Copying a local file to S3 with Storage Class

S3 Provides various types of Storage classes to optimize the cost and to manage the deejay efficiency and IO performance during file read and write operations.

  • S3 Standard
  • S3 Intelligent-Tiering
  • S3 Standard-IA
  • S3 I Zone-IA
  • S3 Glacier
  • S3 Glacier Deep Archive

You can read more information nearly all of them here

aws s3 cp file.txt s3://bucket-name --storage-class class-proper name

aws s3 cp

Console Output:

In the prceding snapshot you can run into that the test2.txt file which we take uploaded merely at present is showing the Standard-IA as the storage class

Copying an S3 object from ane bucket to some other

At times we would want to re-create the content of 1 S3 bucket to another S3 bucket and this is how it tin be washed with AWS S3 CLI.

aws s3 cp s3://source-bucket-name/file.txt s3://destination-saucepan-proper name/

aws s3 cp

How to Recursively upload or download (copy) files with AWS S3 CP control

When passed with the parameter --recursive  the aws s3 cp command recursively copies all objects from source to destination.

It tin be used to download and upload large gear up of files from and to S3.

Here is the AWS CLI S3 command to Download list of files recursively from S3.  here the dot . at the destination stop represents the current directory

aws s3 cp s3://bucket-name . --recursive

aws s3 cp

the same command can exist used to upload a large set of files to S3. past just changing the source and destination

aws s3 cp . s3://bucket-proper name  --recursive

Here nosotros have just changed source to electric current directory and destination to the bucket and now all the files on the current directory(local) would be uploaded to the bucket.

It includes all the subdirectories and hidden files

Setting the Access Control List (ACL) while copying an S3 object

For security and compliance reasons, we might want to restrict the files that we are copying to S3 buckets with fix of Access control. Like Read only or Read Write etc.

Hither is the command that copies a file from i bucket to some other bucket with specific ACL applied on the destination saucepan ( after copied )

aws s3 cp s3://source-bucket-proper name/file.txt s3://dest-bucket-name/ --acl public-read-write

At that place are 5 types of ACL permissions available with S3 which are listed here on the following snapshot.

aws s3 cp
Image credits to AWS

Hope this helps.

How to re-create files between EC2 and S3

We have and then far discussed how to re-create files from local to EC2 and vice versa.

When information technology comes to EC2 instances information technology is a trivial dissimilar equally the authentication tin exist taken care of by IAM roles assigned to the server.

S3 buckets tin also have certain bucket policies to allow servers coming from a certain AWS account of VPC endpoint etc.

We have created an commodity that talks well-nigh how to hands copy files between EC2 and S3 with a simple 4 step configuration.

Copy files from EC2 to S3 Bucket in 4 steps

Automate S3 tasks with Ansible

We have an exclusive article covering diverse examples of Ansible S3 module usage and examples.

If you lot are looking for some automation with S3. I would recommend you to requite information technology a attempt.

How to use ansible with S3 – Ansible aws_s3 examples | Devops Junction

If you have any feedback or all-time practices. please feel free to comment and permit us know.

Cheers
Hanumanth

gritfy