0

Goal: Get files

  • from Bucket 1 in ca-central-1 in Account A
  • to Bucket 2 in us-east-1 in Account B
  • using the AWS CLI from a third machine using an the IAM role with correct S3 read and write permissions (assume unless that's unlikely)

I got the error:

ClientError: An error occurred (AccessDenied) when calling the CopyObject operation: VPC endpoints do not support cross-region requests

and this broke my mental model about how S3 works. I thought that S3 was not behind any VPC and that VPC endpoints were just about an alternate routing pathway (other than the internet) for a machine within a private subnet.

But if you're using the CLI and asking to transfer files from one S3 bucket to another, why would a VPC come into play at all?

Improve this question

1 Answer 1

Reset to default
1

I assume since you say "using the AWS CLI from a third machine using an the IAM role" that your "third machine" (why third?) is an EC2 instance inside a VPC.

EC2 instances are inside VPCs. If that VPC has an S3 VPC endpoint then the EC2 instance will use it, as use of endpoints is based on DNS. You've found that VPC endpoints don't support cross-region copying.

Options:

  • Use built in S3 replication to copy the bucket / folder, but this is probably not flexible enough to do individual files
  • Remove your VPC endpoints
  • Use the --endpoint-url of the S3 CLI to manually specify the URL of a public S3 endpoint. I'm not 100% sure this will work but it's worth a try
  • Write a lambda function to do the copy, make sure it runs outside the VPC - this is the default unless you configure it to run inside the VPC
Improve this answer
3
  • I was wondering if you were going to bring up the "third machine" (third bullet point, just a machine). I've sent the S3 CLI operations from both an EC2 instance and just a Mac, both with the same ~/.aws files. I guess that's why I was confused: the S3 file transfer wouldn't be going through the machine sending the CLI commands, would it?
    – Ben Ogorek
    Oct 25, 2022 at 19:26
  • 1
    The copy operation could download / upload, or it could be an internal transfer, I don't know.
    – Tim
    Oct 25, 2022 at 19:59
  • 1
    Thank you for the insight, @Tim. I was thinking that the S3 buckets were somehow in VPCs but now I understand that the VPC endpoints themselves were getting in the way.
    – Ben Ogorek
    Oct 27, 2022 at 17:02

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .