All Questions
87
questions
0
votes
1
answer
190
views
Backup from S3 to S3
I'd like to backup multiple S3 buckets into another S3 buckets. It seems like most backup tools only support S3 as a destination, but not as a source e.g. restic. The backup should support pruning, ...
- 129
4
votes
1
answer
204
views
AWS EC2 Windows server with S3 storage for Veeam backup
I want to create a remote repository for the backup of my local infrastructure.
I'm using Veeam v10.
I think I need a remote machine with a Veeam component installed, then I was wondering if I can ...
- 1,312
-1
votes
2
answers
73
views
Backup of multiple sites on Ubuntu
I have multiple sites in my /var/www/html/ folder like following:-
/var/www/html/site-1
/var/www/html/site-2
/var/www/html/site-3
/var/www/html/site-4
i want to zip each site on my server and to ...
- 2,009
2
votes
1
answer
285
views
Best strategy for daily back up of terabytes of data with millions of files
Currently I create an NFS and mount it on a special server that does the actual backups and uploads it to s3. My main concern is I feel like overcomplicated this simple task. I schedule cron job to ...
- 174
3
votes
1
answer
842
views
Guidance on Offline/Online Backup for AWS EC2, RDS & S3 [closed]
I have some EC2 instances (Linux and Windows) with attached EBS Volumes, some MySQL Database and S3 buckets in an AWS account. I am in a situation where
I won't have time to work on this project ...
- 33
-1
votes
1
answer
182
views
Data Backup to AWS S3 using Backup Agent
I am having cloudberry backup client, using which I need to backup some data. Consider the data to be backed up is 100 GB on a daily basis. I have 2 different use cases, to understand the number of ...
- 764
1
vote
1
answer
1k
views
Clever way to reduce rsync/rclone load with backups to Amazon S3?
I started running rclone to keep a set of files on my laptop that are synced up to the cloud and encrypted. I wanted to 'control' the encryption because of trust issues. Rclone does this pretty well, ...
- 13
5
votes
2
answers
3k
views
How to create an OFFLINE Incremental backup of an AWS S3 bucket
I'm looking for a way to take daily backups of an AWS bucket as incremental backups. These are to be stored offline and away from AWS.
For other storage systems (such as NAS drives) I use a daily ...
- 1,761
0
votes
0
answers
207
views
Backup AWS VOLUME to S3 (not EBS Snapshot on s3)
Backup AWS #Volume (to #S3 or similar).
Problem:
I have some old instances, and even worse, old data volumes, that needs to be stored for some time, then removed. Very likely noone would look at it ...
- 121
1
vote
1
answer
283
views
Multiple server backups using kubernetes and rsnapshot
I am looking to setup multiple backups, from multiple servers in different geographical locations at different times using something like a docker/rsnapshot and kubernetes (master + a couple nodes at ...
- 11
2
votes
2
answers
130
views
Backup strategy for user uploaded files
Note: Backing up data stored on Amazon S3 is similar but quite old, and doesn't address common practice.
Our service involves each user upload a couple of files. There are hundreds of thousands of ...
- 23
5
votes
1
answer
894
views
Syncing / backup from S3 buckets to openstack swift containers
I need to backup/sync few hundred GB of files in S3 buckets to openstack swift containers.
I have thousands of files in several S3 buckets I want to copy/sync over openstack swift containers but I ...
- 51
4
votes
5
answers
780
views
Transfer backups or directly create backups on Amazon S3
I'm planning to use Amazon S3 to store our regular SQL Backups. I would like to know if it is recommended to upload the backup after locally created or create it directly on S3. I think it could be ...
- 299
2
votes
4
answers
14k
views
Unable to run aws s3 sync as a cron job
I'm having trouble running aws s3 sync as a cron job (on Ubuntu 16.04). Here's what I am able to do:
Manually run the script in the /etc/cron.hourly directory using sudo
(so it doesn't seem to be a ...
- 247
3
votes
1
answer
11k
views
How to keep MinIO backed up?
We're running MinIO internally for object storage. Ideally, we'd like to be able to do backups the same way that would with other important data. Ideally, we'd like to be able to take snapshots hourly,...
- 143
0
votes
1
answer
285
views
Goodsync S3 Empty Secret Access Key (AKA Hacking you Linux Goodsync configurations file)
I'm trying to use Goodsync from Linux (Wheezy). I have set up a backup to S3 (see below) but I keep getting the error Empty Secret Access Key.
=== Job === test:<
'file:///testgsynclocal/' <-&...
- 535
2
votes
3
answers
535
views
How correctly and cost effective to store EC2 instance backups
How correctly and cost effective to store EC2 instance backups?
I have a scenario like:
Create EC2-instance snapshot.
Move it to S3 bucket.
Move it from S3 bucket to Glacier
Done
It is correctly ...
- 145
2
votes
1
answer
1k
views
How do I order a retrieval drive from AWS Glacier?
I have 3.5TB of data (file backups) in AWS Glacier. I would like AWS to ship me a retrieval drive/appliance with this data because I don't think the download would successfully complete. I attempted ...
- 125
0
votes
2
answers
1k
views
Large mysql dump pipe to s3
Is there any problem in doing this with lots of data?
mysqldump ... | gzip | s3cmd put - s3://bucket/file.sql.gz
The MySQL dump is about 100GB in size. What happens if gzip or s3cmd can't process ...
8
votes
1
answer
5k
views
Copy s3 contents of bucket, including all versions
I have a few S3 buckets that I want to hand access over to another organisation, as am handing over an existing hosting client to them.
After looking through the docs, if I no longer want to be ...
- 709
2
votes
2
answers
2k
views
Is Amazon S3 versioning really a reasonable backup options?
I am in the process of moving my web apps project storage to S3 and I am wondering if S3 versioning is really a good way to handle backups of the data or is there some better way? If the files were ...
- 201
53
votes
3
answers
78k
views
Rsync to AWS S3 bucket
For a server I am hosting a website on I want to backup the data and settings to an S3 bucket. I found out that you can't directly use rsync to backup to an S3 bucket. Is there another way to achieve ...
- 653
2
votes
1
answer
113
views
How to secure the db dump taken from daily run script
In one of my mysql slave servers I have written a daily run script, which
1) stops slave, 2) takes a db dump, 3) starts slave again, 4) encrypts it, 5) copies it to my s3-bucket.
I am using aws-cli ...
- 123
0
votes
1
answer
37
views
AWS backup on another attached drive
In AWS-EC2, I was thinking of backup my code data and database in RDS via bash script on separate attached drive on same instance. I want to do it like that way so it becomes easier for me during ...
- 379
0
votes
1
answer
934
views
How can I easily backup multiple folders on EC2 to S3
I move dockers around a lot on EC2 instances. Each docker has a specific folder on the host where it stores it's data.
I want to create a single backup script that will look for any of a list of ...
- 121
0
votes
2
answers
2k
views
S3 backup with versioning plus lifecycle management to prevent malicious deletion of backups?
I just want to run this idea by some smarter people to make sure I'm not overlooking something obvious:
I want to backup my Linux server to S3 using one of the many backup scripts that allow ...
- 355
2
votes
4
answers
3k
views
Duplicity backup to S3: BackendException
I'm trying to configure my Duplicity installation to backup to an Amazon S3 bucket. I keep getting a BackendException error with no further details.
EDIT: I've removed my Duply configuration from ...
- 163
0
votes
1
answer
2k
views
Duplicity for remote backup to AWS S3
I am using Bacula with S3 for backups. I keep bacula volumes for 60 days or so. Using duplicity as I would like to use GPG encryption of the volumes.
Is there any way to do an rsync style sync using ...
- 58
3
votes
0
answers
523
views
Solution for file data backup, versioning and replication
We have a growing set of data files (.wav files, image files etc) which are data, i.e not part of the application code - uploaded and modified by users. The number of files is in the 1000s and the ...
- 227
1
vote
1
answer
758
views
Redirect warning when running s3cmd sync
When running s3cmd I get a redirect warning as follows:
WARNING: Redirected to: xyz.s3-eu-west-1.amazonaws.com
The script that produces the warning is this:
s3cmd sync /var/backups/db/ s3://xyz/...
- 189
0
votes
1
answer
512
views
Back up S3 bucket to another?
I am looking at duply and duplicity currently to mirror one bucket to another but it looks like I may be running in to a brick wall?
So far I have been able to get things working backing up local ...
- 790
2
votes
1
answer
7k
views
How to best backup AWS-hosted systems? [closed]
We are deploying a number of servers at AWS and need to work-out a general-purpose backup strategy. The requirements are:
Retain the ability to recover full systems or individual files or ...
- 2,347
10
votes
5
answers
12k
views
how to pipe a mysql dump to s3cmd
I want to transfer a mysql dump, compressed, to s3.
I tried:
mysqldump -u root -ppassword -all-databases | gzip -9 | s3cmd put s3://bucket/sql/databases.sql.gz
but then I get:
ERROR: Not enough ...
- 329
1
vote
0
answers
453
views
Duplicity -> S3 backup failing with "[Errno 105] No buffer space available"
I have a nightly script on my Ubuntu 10.04 VPS which uses duplicity (0.6.24) to run incremental, encrypted backups to Amazon S3. This script had been working until a month or so ago when it started ...
- 239
3
votes
2
answers
778
views
How to discourage a hacker with root access from deleting remote backups?
I'm currently researching the best backup solution for my CentOS webserver, and I'm thinking of going with either Tarsnap or straight-up Amazon S3.
I am trying to figure out how to discourage a ...
- 355
16
votes
3
answers
10k
views
Backing up data stored on Amazon S3
I have an EC2 instance running a web server that stores users' uploaded files to S3. The files are written once and never change, but are retrieved occasionally by the users. We will likely accumulate ...
- 213
1
vote
2
answers
2k
views
Can I re-attach my EBS while a snapshot is pending?
I have my database backed up to an EBS drive. My current backup method is
1) queue all writes (not important how)
2) kill database process
3) detach + umount ebs volume
4) take snapshot
5) ...
- 125
1
vote
0
answers
574
views
Amazon S3 sync theory
I'm setting up a backup of my vital data (database and images, mainly) to S3 from my Ubuntu box. I've set up Amazon S3, and installed S3FS to mount the bucket on my machine. This is working well, but ...
- 193
2
votes
1
answer
2k
views
What is your general strategy to backup S3 buckets?
We are starting a project which involves storing large data quantities in S3. S3 is good at scaling and we've expect to have up to 5TB and million of files in bucket. While i may trust Amazon in ...
- 3,047
0
votes
1
answer
870
views
How continue sync with S3CMD after broken connection?
I'm using s3cmd to backup some logs into Amazon S3 buckets. Which is a great tool.
But I notice that, very often, my sync's are broken, they just cancel and try to upload again, until they can upload ...
- 155
4
votes
2
answers
1k
views
Taking Cassandra Backups
We currently have 12 nodes running in our Cassandra cluster. Ultimately even if a couple of the nodes go down, we're still up and running. The paranoia in me would like to do at least one backup a day ...
- 1,981
0
votes
2
answers
2k
views
Upload database backup from mysql to Amazon S3 or Glacier without creating local file
Is there a tool that makes possible to backup a Mysql database to Amazon S3 or Amazon Glacier without having o create a local file with the database contents?
Something like that:
mysqldump -u ...
- 695
10
votes
6
answers
5k
views
Does a CDN still work even when my server is down?
I am a website owner planning to use Amazon's cloudfront S3. I have been reading all the stuffs about what a CDN can do but i still have an unanswered question.
Does a CDN still work even when my ...
- 101
2
votes
1
answer
2k
views
Best failover strategy for e-mail servers on AWS to ensure high availability
We have our e-mail server hosted on AWS. Last week Amazon had a failure in their East Coast region which brought down our server along with many others.
We now want to implement a failover strategy ...
- 365
3
votes
3
answers
4k
views
What's the best way to backup cPanel WHM server to Amazon S3? [closed]
I would like to archive cPanel backups to Amazon S3 instead of FTP.
I've done some googling, but I'm not finding anything authoritative.
I found the following options:
whms3backup.com - $250 for ...
- 311
1
vote
3
answers
536
views
Backing up data (including mysqldumps) to S3
We have a web app on a number of servers and we want to add an additional layer of redundancy by backing up the key data to S3. The key data is the MySQL database and a folder containing dynamically ...
- 990
0
votes
4
answers
2k
views
Easiest to use backup software for live linux server [closed]
I need a backup software that is capable of:
doing a full HDD image (from live server, if that's possible at all?)or just copy everything except /dev, /tmp, etc
updates the backup incremental just ...
- 49
8
votes
3
answers
6k
views
Duplicity not writing to a pre-existing S3 bucket
I'm trying to backup a directory to a pre-existing Amazon S3 bucket using the following command:
duplicity --no-encryption system/ s3+http://MY_BUCKET_NAME/backup
However, I'm getting the following ...
- 489
2
votes
1
answer
1k
views
Amazon S3 Website & MySQL Backup
I have a website (a digital asset management system/gallery - http://www.resourcespace.org) that has a huge amount of images. The total size of the website, including the images is approximately 6gb.
...
user991830
18
votes
5
answers
12k
views
Any need to make backup of data on Amazon S3?
I'm hosting 200 GB of product images at S3 (this is my primary file host).
Do I need to back that data up somewhere else, or is S3 safe as it is?
I have been experimenting with mounting the S3 ...
- 463