How to copy files from S3 Bucket to Local with AWS CLI

While it is not necessary needed to copy files from S3 to current directory in production environment with CLI, it is certainly useful method to help test certain logic while development.

In this illustration, we will be using AWS CLI as the main command line method to perform the operation. You may refer to the AWS documentation to setup AWS CLI if not installed.

We will be using s3 commands here and you may refer to here to illustrate the differences between s3 and s3api commands.

Method to Copy files from S3

• Copy single file from S3
• Copy files from whole bucket

Do note that there are some fundamental differences between cp and sync, please refer here to view the comparison.

Copy single file from S3

In this example, we will copy only 1 file from S3 to local directory.

The command consist of a few segments. The parts that we will need to change are endpoint, bucket name and original file name.

# Copy 'old-name.txt' located at host 'http//s3-url' from my-bucket to your current local directory
aws --endpoint-url=http//s3-url s3 cp s3://my-bucket/old-name.txt . 

If there is a need to change the name of the file after copy, we can change the command to the below version in 1 line.

# Copy 'old-name.txt' to 'new-name.txt'
aws --endpoint-url=http//s3-url s3 cp s3://my-bucket/old-name.txt new-name.txt

Before copying, please be careful that the current directory does not that the file with the same name. This is because the command will overwrite the existing file without warning.

Copy files from whole bucket

If there is a need to copy all the files from a bucket, add recursive so that it will copy everything in subfolders to your current directory as well.

# Copy everything from 'my-bucket' to current directory
aws s3 cp s3://my-bucket . --recursive

Conclusion

With AWS CLI, it is easy and straight forward to copy files from S3 to current directory. More information on how to use cp is in AWS documentation.