Questions tagged [amazon-s3]
This is for questions on Amazon's cloud storage service.
1,124
questions
2
votes
2
answers
533
views
How to point route53 records to bucket for *any*subdomain?
I have an S3 bucket called www.mybucket.com
I have an A record: name = www.mybucket.com. It answers well with the index.html contained in the bucket.
Now I want to send any subdomain to that bucket.
I ...
- 101
0
votes
0
answers
48
views
Delete object everyday in S3 bucket
i'm trying to delete folders(objects) in the S3 bucket everyday (with lifecycle rule). I have this scenario :
TestBucket -> connect/ -> test/ -> 2023/ -> 12/ -> 01/, 02/, 03/ etc.
I ...
1
vote
1
answer
4k
views
Domain to S3 Bucket AccessDenied
Hello i have a problem i recibe AccessDenied in my domain
I upload my site in to a AWS S3 Bucket, i make public, create a static page, but when i access via my domain.com/mypage i get a AccessDenied.
...
CommunityBot
- 1
11
votes
2
answers
11k
views
Determine which AWS account owns an S3 bucket
If someone gives me an S3 url with a bucket and an object, but I don't know which account owns the bucket, how can I determine which credentials to present in order to gain access to the object, ...
- 1,142
0
votes
1
answer
85
views
S3 site in Cloudfront Preflight 302 Error
I have a static s3 website hosted through CloudFront. In it, I make a request that will result in a redirect if the user is not logged in. Preflight then fails because of a 302:
Access to fetch at '...
- 119
0
votes
1
answer
46
views
My Cloudfront distribution is showing access denied despite enabling all public access to the bucket objects
I am using the following CF policy in my S3 bucket:
{
"Version": "2008-10-17",
"Id": "PolicyForCloudFrontPrivateContent",
"...
- 176
45
votes
11
answers
77k
views
Most efficient way to batch delete S3 Files
I'd like to be able to batch delete thousands or tens of thousands of files at a time on S3. Each file would be anywhere from 1MB to 50MB. Naturally, I don't want the user (or my server) to be waiting ...
- 133
0
votes
1
answer
82
views
Best method to copy image files from AWS EC2 instance to S3 bucket based on query
What is the most efficient method to copy selected image files from a EC2 instance (Ubuntu 20.04) in production to a S3 bucket while checking if the file exists in EC2?
This is a one-time operation. ...
- 5,553
2
votes
2
answers
7k
views
How to find owner of a aws account with account number
I inherited a couple of AWS environments. I have been recently doing security audits of s3 and found several policies with principals containing aws account numbers I don't know and nobody at my ...
- 101
0
votes
0
answers
37
views
Oracle MySQL operator for kubernetes backup to S3 using service account
I've installed the operator, and trying to create the innodb cluster with backup profile which uses S3. There is no mention of S3 in the documentation, but they mentioned in a conference about S3 ...
- 101
0
votes
0
answers
38
views
Is Mastodon remove-orphans safe when multiple instances sharing an S3 bucket?
I have two Mastodon v4.2.0 instances, a staging server and a live/production instance. They share an AWS S3 bucket for file storage. Both are hosted on DigitalOcean.
Is the command RAILS_ENV=...
0
votes
0
answers
52
views
Maximize duplicity efficiency
I'm creating incremental backups for a database. The size is around 8 GB per dump.
I have a backup creation policy, that looks like this: (don't worry, I already avoided collisions)
An increment ...
26
votes
7
answers
39k
views
How to verify a AWS VPC (S3) endpoint works?
I added a VPC endpoint to my VPC using CloudFormation, and allowed s3 usage. The routes are visible in the AWS console, but not in the local routing tables of the
EC2 instances:
$ route -n
Kernel IP ...
1
vote
1
answer
16k
views
AWS S3 bucket - Allow download files to every IAM and Users from specific AWS Account
Look for a policy for S3 bucket that will allow all IAM roles and users from different account, to be able to download files from the bucket that is located in my AWS account.
Thanks for help
CommunityBot
- 1
0
votes
0
answers
37
views
Why do I need to map separate s3 buckets for www subdomain requests in AWS?
I have a question about how DNS works in conjunction with AWS s3 buckets that I am trying to understand. I have a modest knowledge about DNS and AWS, but I don't understand why my routing solution ...
- 101
1
vote
1
answer
10k
views
How do I extract a list of all folders of my AWS S3 storage bucket or directories?
Does Amazon provide an easy way extract a list of all folders that have files greater than 500 MB from a s3 bucket? want to limit the scope to the '/files/ftp_upload/' directories also This is so I ...
CommunityBot
- 1
1
vote
2
answers
2k
views
CloudFront distribution can't access S3 bucket for website static assets
The problem
I'm trying to deploy an S3 bucket hosting my static website asset and a cloudfront distribution to access it, but the distribution still return a bare xml file for an 'access denied' error:...
0
votes
0
answers
190
views
Dropzone: How to upload video to S3 using presigned URL and play it in browser? Doesn't quite work
I have Dropzone set up to allow users to upload videos directly to an S3 presigned URL using the PUT method. Here is my Dropzone script:
var presign = "#";
Dropzone.autoDiscover = false;
var ...
- 73
4
votes
4
answers
10k
views
How can I get a breakdown of s3 cost at per bucket level in Cost Explorer?
It seems like I can get total cost of S3 using the service level in the filter.
However I want to get cost per bucket level.
Can it be done in Cost Explorer?
If not, can I get the breakdown with ...
- 1
0
votes
0
answers
204
views
minio A header you provided implies functionality that is not implemented
I'm trying to move object on MinIO yet running into weird issue (even though I'm able to move other objects fine without any issues)
summary:
mc: Failed to copy https://FQDN/path1/file.txt. A header ...
- 13.2k
1
vote
2
answers
1k
views
ansible - couldn't resolve module/action 'amazon.aws.s3_object' / The module amazon.aws.s3_object was not found in configured module paths
Per requirements, I did installed all the required packages, however amazon.aws.s3_object is one of those things:
couldn't be resolved/found
was not found in configured module paths
step(s), I used ...
- 13.2k
1
vote
1
answer
127
views
Scheduled task to upload a file to S3 with PowerShell ignores user profile settings
I am using Windows task scheduler with a user that has an AWS profile configuration under C:\Users<username>.aws\configuration (content below)
[default]
s3 =
max_concurrent_requests = 300
...
- 36.3k
0
votes
1
answer
86
views
How to know if an S3 bucket with Cross-Region Replication is fully replicated?
I want to setup my S3 bucket to replicate in another region. Once it's fully replicated, I will make that bucket the main one (the idea is to migrate all data from one region to another). I know how ...
- 32.2k
3
votes
2
answers
809
views
Why AWS encryption for EBS and S3 is disabled by default? [closed]
For EBS and S3 encryption (even via the default key from KMS) is disabled by default, as I understood from the documentation, there is no difference to the user if volume/objects is/are encrypted ...
- 32.2k
0
votes
1
answer
75
views
Proper Setup for a simple AWS S3 website using CloudFront & Route, with base domain and www subdomain
Which is the better design, and why? (see 2 tables)
1. Here is the configuration of my working website (example.com is the prevent domain)
as setup via a Youtube Tutorial...
Route53
Cloudfront Dist
...
- 32.2k
0
votes
0
answers
67
views
Multi-region internal, domain resolved static website hosted in S3
Statement
I do have a website that has a single page application hosted in S3, and a web service hosted in AWS Lambda. To avoid cross-site requests, both share the same domain, and I route the ...
- 101
0
votes
1
answer
434
views
Restoring Terraform state from backend
My Terraform Git repository ended-up having some unreconciliable data error and I couldn't push a recent change. I exported my commit to a patch, recloned, applied the patch, and then realized I just ...
- 560
4
votes
3
answers
12k
views
s3fs "could not determine how to establish security credentials"
I'm trying to use s3fs to mount an S3 bucket on to a standard AWS Amazon Linux AMI (with all the necessary dependencies installed). However when following this tutorial when I run: s3fs mybucketname -...
- 11
0
votes
1
answer
304
views
How to set correct Content-Type HTTP header for all files uploaded on S3 bucket?
What is please the correct way to set Content-Type HTTP header for files uploaded on S3 bucket? We use S3 bucket as an origin server for hosting static websites. Bucket is accessed through the ...
- 1
4
votes
4
answers
9k
views
s3fs Input/output error
I'm trying to set up a backup system with s3fs and the Amazon S3 service.
I followed this this guide and this guide.
Tailing /var/log/messages I get:
Aug 28 13:37:46 server s3fs:###response=403
I ...
- 1
52
votes
8
answers
47k
views
Chrome S3 Cloudfront: No 'Access-Control-Allow-Origin' header on initial XHR request
I have a webpage (https://smartystreets.com/contact) that uses jQuery to load some SVG files from S3 through the CloudFront CDN.
In Chrome I will open an Incognito window as well as the console. Then ...
- 31
9
votes
3
answers
22k
views
s3cmd: Deleting files or folder does not work
s3cmd s3://MYBUCKET/FOLDER does not work. (folder is empty)
s3cmd s3://MYBUCKET/FILE does not work.
Output of both commands is "File FILE sucessfully deleted", yet the file remains on the bucket.
...
- 101
0
votes
0
answers
65
views
Enable MFA S3 delete
I have THALES TOTP hardware tokens (MFA) which I would like to use as and additional protection against the accidental S3 object deletion (https://docs.aws.amazon.com/AmazonS3/latest/userguide/...
- 1
0
votes
1
answer
91
views
Best practice for storing personal information on AWS that we shouldn't be able to see
We store customer information that we shouldn't have access to, and don't want to inconvenience users by making them lose information if they forget their passwords, is there a good way to solve this ...
- 21.9k
0
votes
0
answers
65
views
AWS AppStream is unable to push session script logs to s3
I have configured session scripts to log stdout and stderr to s3 bucket as described in https://docs.aws.amazon.com/appstream2/latest/developerguide/use-session-scripts.html#enable-S3-bucket-storage-...
- 1
0
votes
0
answers
131
views
AWS Lambda's indirect S3 requests
I noticed AWS Lambda causes S3 tier 1 and 2 requests. Do these cost anything after 12 months free tier expires?
Specifically:
Create new AWS Lambda function in AWS console causes 1 tier 1 S3 request
...
- 121
8
votes
5
answers
30k
views
How can I set up an SFTP server backed by S3 (or similar)
I need to set up an SFTP server that, essentially, has very large capacity. I need to give one of our partners SFTP login details to a server where they will upload millions of files, totalling a few ...
- 1
0
votes
0
answers
287
views
botocore.exceptions.ClientError: An error occurred (InvalidToken) when calling the ListObjectsV2 operation
I have an mlflow server that is hosted on aws, that uses s3 in the background for storage. I am trying to run a lambda-function that runs a python script which accesses the mlflow server, loads a ...
- 3
0
votes
2
answers
208
views
How I can Log what and who can access and where my S3 bucket?
Recently I received an email in order to tell me that some IE connections has access to my bucket:
eu-central-1|media.myapp | REST.GET.OBJECT|TLSv1|9|[Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 6.1;...
- 22.8k
2
votes
2
answers
864
views
Redirecting HTTPS -> HTTPS on AWS?
I have a static website w/ S3 + Route53 + Cloudfront set up. I am successfully able to do the following redirects:
http://example.com -> https://example.com
http://www.example.com -> https://example....
158
votes
8
answers
159k
views
Amazon Cloudfront with S3. Access Denied
We're trying to distribute out S3 buckets via Cloudfront but for some reason the only response is an AccessDenied XML document like the following:
<Error>
<Code>AccessDenied</Code&...
- 107
0
votes
0
answers
78
views
How do I setup a Google Cloud Storage bucket to be accessible via the Simple Storage Service (S3) standard?
I understand from Google documentation that it's possible for google cloud storage to be accessed via the Simple Storage Service, which is just what I need for a program that supports this. The ...
google-cloud-platform
0
votes
1
answer
128
views
Storing 100 million files in the same "directory" under S3-compatible storage?
I have > 100 million image files (book covers) as a flat list of files under a single "directory":
/images/000000093e7d1825b346e9fc01387c7e449e1ed7
/images/...
- 134k
0
votes
1
answer
213
views
S3 / Cloudfront / Load balancer Architecture explanation needed
My architecture is like so:
I have a static website xyz.com hosted on s3 and served with cloudfront. The website is accessing an api that seats behind a load balancer. I want to make sure that only ...
- 32.2k
0
votes
0
answers
776
views
Whitelist an S3 Bucket in server firewall
I am looking for a way to whitelist an S3 Bucket on my client's server. In this S3 bucket there is a public website, thus, my idea is to find the ip address used for this S3 bucket. However, I have ...
- 103
2
votes
1
answer
1k
views
SSHFS mount to AWS Transfer for SFTP: cannot create regular file
I'm trying to setup an SSHFS mount point via Amazon's new service Transfer for SFTP. I can sftp into the endpoint just fine using sftp and can get/put files.
I can mount it using sshfs. I'm having ...
- 1
-2
votes
1
answer
108
views
Does Amazon have a web server named 'AmazonS3'?
Note that I understand what "Amazon S3" is; I'm asking only about the software that handles the HTTP interface.
If I check the HTTP headers for github.com, for example, one of the response ...
- 32.6k
0
votes
0
answers
276
views
How to serve both static and dynamic content via AWS cloudfront?
I have a site hosted on a digital ocean droplet which serves both static and dynamic content. The static content (e.g. /images, /js, /css, /fonts) is served from an s3 bucket with folders /images, /js,...
- 119
3
votes
2
answers
6k
views
`Access Denied` for some files, when syncing buckets
I'm using awscli to sync 2 buckets (same account):
aws s3 sync --only-show-errors s3://bucket-1 s3://bucket-2
but for some files I get permission errors:
copy failed: s3://bucket-1/dirname/file....
0
votes
1
answer
721
views
Getting 403 (Forbidden) when try to load an S3 file from AWS CloudFront behaviour
I have created an S3 bucket. All objects are publicly accessible.
I can access the image file via S3 url: https://S3_BUCKET_NAME.s3.eu-west-1.amazonaws.com/xyz.jpg
I can access the image file via ...
- 1