All Questions
20
questions
0
votes
1
answer
43
views
Cloud File Storage: Does zipping files reduce data transmission speed / costs?
If I need to store, say, a 5gb file on a cloud file storage location (e.g., aws s3 bucket), will zipping the file reduce the cost / latency when people download it?
Obviously zipping a file reduces ...
- 105
1
vote
0
answers
128
views
Fastest way to import files in Spark?
I’m playing around with Spark 3.0.1 and I’m really impressed by the performance with Spark SQL on GB of data.
I’m trying to understand what’s the best way to import multiple JSON files in the Spark ...
- 193
1
vote
0
answers
60
views
Best Practice: Ansys FEA Output Data Storage
I'm trying to make this as least subjective as possible by providing finite inputs and parameters. I truly want to know what is the most resource affective, given the constraints. Given my available ...
- 11
1
vote
1
answer
179
views
Export 20TB from Amazon S3
I have recently inherited 20TB of data after a company takeover. The files are currently stored on an Amazon S3 account which is due to close.
What would be the most cost effective method of ...
- 113
1
vote
1
answer
1k
views
Files Not Appearing in S3 File Storage Gateway Bucket
I created an AWS S3 File Storage Gateway and mounted it to a Linux server on-prem. On that same server where the file gateway is mounted I have a separate NFS share mounted. My goal is to copy files ...
- 688
2
votes
2
answers
130
views
Backup strategy for user uploaded files
Note: Backing up data stored on Amazon S3 is similar but quite old, and doesn't address common practice.
Our service involves each user upload a couple of files. There are hundreds of thousands of ...
- 23
-1
votes
1
answer
85
views
Best and cheapest way of expanding storage on a server
I provide photo editing services, so customers send their photos to me through website, and I send them back through website once edited. I need about 1TB storage when I keep files for 1 month. I'm ...
4
votes
0
answers
196
views
S3 + Cloudflare vs S3 + 2 Web Server + Cloudflate
Currently, I run two websites and each site own it own big server. Currently, all of those two sites has roughly 300GB of images. Its grow rate is 15GB of images per month. I know that eventually, one ...
- 405
2
votes
1
answer
134
views
Performance issue in using Amazon S3 for run-time updates
I have a web application hosted in Amazon EC2 as well as in Rackspace. Users can upload files through this application and also modify them. Those files are stored in local storage in each of these ...
-4
votes
1
answer
322
views
Where to store social network photos and videos? [closed]
I have a social network with 100,000 users. I allow users to upload photos and videos now I feel like it is becoming too much for my dedicated server. Should I get another dedicated server? A content ...
- 369
-1
votes
2
answers
341
views
Cross platform, blob storage system
I have a couple of different servers around the house, and a lot of storage on each of these machines. I also have a couple of dedicated servers, and some VMs on these boxes... Most of the boxes and ...
- 744
53
votes
6
answers
67k
views
How to move files between two S3 buckets with minimum cost?
I have millions of files in a Amazon S3 bucket and I'd like to move these files to other buckets and folders with minimum cost or no cost if possible. All buckets are in the same zone.
How could I ...
- 823
3
votes
1
answer
3k
views
Change Amazon S3 storage class on A LOT of files
We are using Amazon S3 and a great deal of our files have the STANDARD storage class. We want to change this to REDUCED_REDUNDANCY class to save the extra money.
However, is has been hard to find a ...
- 194
14
votes
10
answers
12k
views
S3-compatible Object Storage systems [closed]
Has anyone come across a near 100% Amazon S3 RESTful API compatible object storage system?
What I am after is a layer that sit on top of any (preferably a POSIX) filesystem that provides Amazon S3 ...
- 304
4
votes
2
answers
2k
views
S3 upload delayed
I have 2 servers using the same data located in AWS S3 storage. One is processing HTTP traffic, the other does longer lived jobs. How it works is that HTTP one saves data to S3, dispatches event to ...
- 505
1
vote
1
answer
2k
views
Amazon S3: allow users to upload on a restricted basis (per bucket maybe)?
I'm thinking about signing up to the Amazon S3 storage service.
What I want to do is create a service where other people can register their own bucket with a certain amount of storage.
These users ...
- 631
1
vote
2
answers
360
views
Amazon cloud: What should I backup exactly?
ec2 Micro type instance comes with only EBS storage - Should I backup everything critical from there?
other ec2 instances come with local storage - Should I backup everything critical from there?
s3 - ...
- 336
1
vote
1
answer
1k
views
Preventing Amazon S3 Bandwidth Stealing?
I have a Image Hosting Site that is using Amazon S3 as a main source for bandwith and storage.
I have a few option here to prevent other sources from stealing my bandwith, but I am hoping there are ...
Anraiki
2
votes
3
answers
1k
views
Does cloud storage replicate the data over many datacenters if so it means i benefit content delivery network?
Let's assume that i want to use cloud storage service from one of the cloud storage provider, i got X gb structured and unstructured data and i will use this data as my contents of my interactive web ...
- 431
3
votes
3
answers
1k
views
How is Amazon S3 storage billed? [closed]
Various Amazon S3 questions
Hey everyone,
I just got an Amazon S3 account and I am developing my production backup script / strategy and just had a few questions.
Monthly Storage - how exactly is ...
John