Questions tagged [amazon-s3]
This is for questions on Amazon's cloud storage service.
1,124
questions
0
votes
1
answer
294
views
Transition of a large bucket to S3 Glacier
I want to move a 72 TB size bucket from S3 Standard to S3 Glacier.
I'm going to create a Lifecycle rule and set "Transition current versions of objects between storage classes" to 1 day.
I ...
- 21
0
votes
2
answers
248
views
Best Practice for AWS IAM access keys for use with AWS SDK
I want to know the best practice used by big company's for programmatic access for multiple AWS services as there are multiple programs needing access to different-2 services so how it is managed? Did ...
- 49
0
votes
0
answers
764
views
400 Bad Request errors (infrequent) on public Amazon S3 assets
We are hosting S3 public assets (images) under a local path using a reverse proxy from NGINX to S3.
We have noticed periodic errors in our logs (400 errors) which are very infrequent, but are causing ...
- 715
0
votes
0
answers
251
views
AWS S3 static website: unable to access https://mydomain.tld but http://mydomain.tld redirects to https correctly
I've set up a static website on AWS S3, using a custom domain hosted at Namecheap, and Cloudfront serving over https. I am able to access my site at http://mydomain.tld, which redirects (using a DNS ...
- 101
1
vote
0
answers
323
views
S3 through Nginx Proxy doesn't allow more than 10 files to be downloaded simultaneously
I setup a Nginx Proxy in front of my S3 object storage and I'm able to download files without any issues so far.
However, if I try to download the same file from, let's say, 30 different servers, at ...
- 111
0
votes
1
answer
1k
views
Why can't my DB back up to my S3 bucket?
I'm trying to test a differential backup and restore on an RDS SQL Server 2019 instance in preparation for migrating some web resources into AWS. Both the EC2 instance (Windows Server 2019) on which I'...
- 143
0
votes
1
answer
841
views
cannot Delete objects on s3 ceph bucket
I have a ceph cluster and running a few s3 bucket on it, 'gitlab-s3-api' user has full permission on everything on ( users=;buckets=;metadata=;usage=;zone=* ) but very oddly it cannot delete any file ...
1
vote
1
answer
4k
views
Domain to S3 Bucket AccessDenied
Hello i have a problem i recibe AccessDenied in my domain
I upload my site in to a AWS S3 Bucket, i make public, create a static page, but when i access via my domain.com/mypage i get a AccessDenied.
...
- 11
0
votes
1
answer
164
views
Cloudfront distribution with react app and wordpress app hosted on same domain - how to get browsers with react app cached to render wordpress app?
I have a domain that hosts a react website in an s3 bucket via cloudfront. There is also a wordpress site hosted on a subdomain of that domain, and in the cloudfront distribution for the web app, I ...
- 103
6
votes
1
answer
4k
views
AWS: How to redirect many domains to a page on another domain?
My objective
I have a number of domains (e.g. 10 or 20) and I would like to redirect any visitors to anywhere on those pages to one page on another domain (for example my stackoverflow.com profile ...
- 1,169
1
vote
1
answer
2k
views
AWS Cloudfront CORS trouble with font files
I have a Javascript application that is delivered to the browser from a CloudFront endpoint. All static files of the application are hosted in an S3 bucket. That involves a whole bunch of javascript ...
- 247
3
votes
2
answers
9k
views
How to *actually* exclude a directory in AWS S3 sync?
The aws s3 sync command has an --exclude flag which lets you exclude a folder from the sync. However, even though the files are not uploaded from that directory, the command still looks at and ...
- 171
0
votes
0
answers
216
views
AWS SES Managing and Forwarding Incoming Mail
I'm probably in over my head with this one. Please bare with me while i fumble through my explanation of what it is that i am trying to accomplish here.
We own a domain name. Have access to the back ...
- 1
1
vote
0
answers
3k
views
Converting specific folder of S3 into browsable directory list without making it public
I have a bucket that I'd like to access using a browser similar to http://data.openspending.org/
and I'd like only a subfolder to be visible.
So if Bucket1 has multiple folders, I only wanna show and ...
- 113
3
votes
1
answer
2k
views
Why is AWS S3 Object count is measured iin units used to measure file size?
In AWS, the files/objects are stored in S3 bucket and the files/object count should list the total no of files/objects stored in S3 bucket as an INTEGER value.
But I see the S3 object count as ...
- 155
3
votes
1
answer
622
views
AWS S3 File Browser for Windows
Is there any official AWS GUI like S3 file explorer for Windows?
I know there's plenty non-official like S3 Browser but, I am avoiding using third-party software.
- 1,351
0
votes
1
answer
2k
views
Why is my database restore failing with ConnectFailure exception?
I am trying to restore a SQL Server database to an AWS RDS instance from a full backup file created on a different non-RDS server. I am running the operation from an EC2 Windows Server instance in the ...
- 143
1
vote
1
answer
1k
views
AWS S3 Policy: One non-public bucket, separate sub-folders for each user, restricted access
at the moment I'm struggling how to create a secure policy for my Amazon S3 bucket.
My plan is to have one bucket with several sub-folders for separate (IAM) users.
Access should only be ...
1
vote
0
answers
80
views
AWS Elastic Beanstalk SNI SSL in S3
In our application we need to support customer ssl certificates and domains.
Application stack: AWS Elastic Beanstalk, NodeJs and nginx server.
I've found that we can use nginx variable "$...
- 11
0
votes
1
answer
799
views
aws s3 mv command is not deleting the folder structure
aws s3 mv s3://test/paltest/ . --recursive
All files within the paltest is been moved but paltest is not deleted. But i am able to remove paltest using aws s3 rm s3://test/paltest/
0
votes
1
answer
296
views
AWS Glacier and Ransomware
I'm trying to understand the structure of how AWS Glacier works because I have a problem.
Problem:
I have a NAS that backs up to Glacier about once a week. About two weeks ago the NAS got infected ...
1
vote
0
answers
237
views
Use increment instead of random string in AWS S3
When I upload files with similar filenames Amazon S3 adds random chars to the filename. How I can change it to use increment instead?
1
vote
0
answers
349
views
Should I archive file copies from S3 into Glacier one-by-one Or should I zip them first?
I have an s3 bucket named media and in it I have a folder named contracts that stores pdf files ~400kb each. For the files in this folder I keep a record in a database as well with the following ...
0
votes
1
answer
83
views
Using IAM Users for customer access to a bucket
For each of my customers, they have data stored on individual dropbox accounts and im in the process of consolidating all of these into AWS in the shape of a single s3 bucket where each customer has ...
- 101
0
votes
1
answer
275
views
Cannot write to AWS S3 bucket using CLI
Im running a Github CD actions that takes some files from my repo and tries to upload them to a S3 bucket I have.
The action runs this command:
/usr/bin/docker run --name ...
- 365
-1
votes
1
answer
74
views
How to get complete bucket access of aws s3 as public?
When we set a file in s3 bucket as public, we can directly download it from its URL.
But, if I set a bucket or a sub folder in bucket to public, I am getting below error.
<Error>
<Code>...
- 372
0
votes
0
answers
192
views
Copy old data from local Centos to AWS S3 bucket
I have the following task to complete. I hope someone can provide the feedback to get me started.
We have a directory with thousands of images in many folders and sub-folders
The goal is to have a ...
-1
votes
2
answers
346
views
How do I share all resources between two AWS Root accounts?
We have one person that has been in charge of all AWS resources for our company, including EC2, elastic IPs, EBS storage, etc. Now we want a second person to also have access to ALL the resources in ...
- 109
1
vote
1
answer
16k
views
AWS S3 bucket - Allow download files to every IAM and Users from specific AWS Account
Look for a policy for S3 bucket that will allow all IAM roles and users from different account, to be able to download files from the bucket that is located in my AWS account.
Thanks for help
- 141
3
votes
1
answer
704
views
Delete AWS RDS S3 Exported Snapshots
So I have created S3 Exports from existing snapshots in RDS. I would like to delete duplicated ones. But I don't see any button to delete them.
I have already deleted the files from the bucket. But ...
- 131
1
vote
0
answers
188
views
Combination of S3 bucket, fuse, rclone mount and incron - not detecting changes
and thanks for reading.....
I'm trying to setup a method whereby I have an on-premise Centos 7 server which uses rclone (mount) to mount an AWS S3 bucket as a file system. I've installed fuse to make ...
- 11
19
votes
2
answers
25k
views
Why don't I have permissions to edit an S3 bucket policy when logged on as the person who created the AWS account
I am new to AWS. I am setting up an S3 bucket that I want to use to store media files for a Django App I am developing.
I am logged in as the person who created the AWS account, but when I click on ...
- 605
1
vote
0
answers
128
views
Fastest way to import files in Spark?
I’m playing around with Spark 3.0.1 and I’m really impressed by the performance with Spark SQL on GB of data.
I’m trying to understand what’s the best way to import multiple JSON files in the Spark ...
- 193
1
vote
1
answer
1k
views
Unable to access S3 static website using https
I have followed all the steps from https://aws.amazon.com/premiumsupport/knowledge-center/cloudfront-https-requests-s3/ but my website still cannot be accessed using https.
My certificate was issued ...
- 83
-1
votes
1
answer
125
views
Suggestions for AWS setup wanted
For my next project, I'm giving AWS and server-less a go, but I'm not sure how to reason about the different AWS services. So here's hoping that someone here has a good feel for it.
The project ...
0
votes
1
answer
272
views
Split Traffic Between S3 Buckets
I have a static S3 site in bucket_old and will rebuild it route by route in S3 bucket_new using the strangler pattern.
URLs /home, /contact map to bucket_new
All other URLs map to bucket_old
What ...
- 101
0
votes
1
answer
457
views
Cloudfront SSL with Subdomain pointing to S3 bucket works, but still redirects to S3 URL
I've set up an S3 Bucket that I want to use as a CDN but with my own subdomain.
For example, myfiles.mydomain.com.
So, I set up a Cloudfront web distribution that points to the bucket named "...
- 247
0
votes
1
answer
3k
views
How to access an aws s3 bucket using Access Key ID and secret access Key?
I have a free account on AWS and somebody shared me an Access key ID and Secret access key to access their s3 bucket.
I am new in AWS, how can I access their s3 bucket using AWS web console?
- 101
1
vote
0
answers
60
views
Best Practice: Ansys FEA Output Data Storage
I'm trying to make this as least subjective as possible by providing finite inputs and parameters. I truly want to know what is the most resource affective, given the constraints. Given my available ...
- 11
1
vote
0
answers
82
views
Data Ingress to S3 from an FTP Server
I have about 1.5TB of data on an FTP server from my web hosting provider that I want to migrate into S3.
From what I have read there are a number of ways to do this, including FileZilla Pro which ...
- 11
9
votes
3
answers
4k
views
CNAME with S3 buckets
I have a Web and I wanted to move its images to AWS S3. Say it's called mypage.com and I can access to an image like this:
https://mypage.com/pics/one.jpg
I created a bucket called static.mypage.com ...
- 235
1
vote
1
answer
716
views
Universally apply restriction to bucket policy, for all S3 buckets
I want to apply a specific restriction to all S3 buckets. Generally AWS published resources describe a per bucket solution. However this does not solve the issue of other users creating bucket ...
- 11
0
votes
1
answer
226
views
Need some help granting access to AWS S3 bucket, without exposing to Public
I'm trying to set up an AWS S3 bucket for storing files that a website will dish up to users of the site. The site is an intranet (only logged in users can access any pages which provide links to the ...
- 173
4
votes
1
answer
204
views
AWS EC2 Windows server with S3 storage for Veeam backup
I want to create a remote repository for the backup of my local infrastructure.
I'm using Veeam v10.
I think I need a remote machine with a Veeam component installed, then I was wondering if I can ...
- 1,312
0
votes
0
answers
28
views
Static WP website and web app are in different host with one domain
I don't know if this is the right place to post this but I am so much in deep confusion right now.
I have a static WP website hosted in Fastcomet with a domain name www.hpdc.com (namecheap) and also
a ...
- 101
1
vote
1
answer
4k
views
Is there a way to see the list of pending S3 Glacier retrievals in the AWS console?
I've initiated the retrieval of an S3 file using the Glacier Deep Archive storage class, using the AWS console. I just got a success message, and no link to a page where I can monitor the status of my ...
- 4,557
0
votes
1
answer
402
views
How to analyze AWS S3 costs?
Into "Cost Explorer Monthly costs by service" report I saw $300 for S3 for certain month. But when I tried to drill-down into the bill for this month, I saw just $4 for the "Simple ...
3
votes
1
answer
600
views
AWS S3 bucket cross account policy mixed with internal account
we use s3 bucket to share files between two companies. We created private S3 bucket and we have dedicated user for application which uses this bucket (using it's aws_key and aws_secret). We figured ...
- 33
2
votes
2
answers
1k
views
s3 bucket/IAM user policy "Deny takes priority above all other access"?
Two policies, got one "Deny", I should not be able to do any operations to bucket,
but I can still list and view bucket objects. Why? Thanks
S3 bucket policy
{
"Sid": "...
- 21
0
votes
1
answer
92
views
VM ram requirement for aws s3 sync job
I have a VM setup to sync 15TB of data split in 25GB volumes to S3 using the latest awscli v.2. I have a 100Mbps upload capacity, and the job will probably take 16 days. I haven't had any experience ...
- 215