What is the difference between S3 cp and sync command

Let’s explore the differences between S3 cp and sync command. We will look into various areas and discuss the difference.

Comparison Area

• Operation
• Efficiency
• Use Case

Operation

S3 cp – Copy all the files to the new location. It will not check if there is an existing file and will just overwrite with the file from source location. The files at the new location will stay even if it does not exist in the source location.

S3 sync – Will check the new location if the file already exist. This operation will only copy the file and update it if the file is not found or updated compared to source location. There is also an option to delete files not found in the source location.

Efficiency

S3 cp – Will read all the files from the source location and write into the new location.

S3 sync – Will scan the new location and only overwrite the file from the source location if the file is newly created or updated(via file size and modified timestamp comparison).

From the looks base on the operation, sync is much more efficiency as it will save time by not copying the files that is the same between source and new location.

Use Case

S3 cp – We can use cp if we want to just copy a few files from source location to new location and no need them to be at the same state.

S3 sync – We can use sync is we need the folders and files to be in the same state between source location and new location

S3 cp and sync Examples

Here, we will show how the commands look like between cp and sync. We also have a more detail illustration on cp operation here and sync operation here.

Please ensure that you have the latest version of AWS CLI installed by referring to AWS documentation.

# Copy all the files from 1 folder to the other recursively
aws s3 cp s3://my-bucket-old/folder s3://my-bucket-new/folder  --recursive

# Sync the 2 locations while removing files that does not exist in source location
aws s3 sync s3://my-bucket-old/folder s3://my-bucket-new/folder --delete

Conclusion

Although it may seems that sync is much more efficient compare to cp command, deciding on which to use depends on the use case. If only copy a few files, we can use cp. If we need to have the 2 location having the same state, then we will use sync.