Questions tagged [rsync]
Rsync is a fast and extraordinarily versatile file copying tool. It can copy locally, to/from another host over any remote shell, or to/from a remote rsync daemon
1,602
questions
0
votes
0
answers
114
views
File transfers very slow in one direction
I am having trouble debugging a connection issue :
Server A (OMV) has a fast internet connection : 900mbps down / 600 mbps up
Client B (Windows 10 using wsl) has a decent internet connection : ...
- 1
0
votes
1
answer
31
views
Rsync creating infinite nested directories
To be truthful, I don't really know if this is caused by symlinks or rsync but I've stopped using symlinks and migrated over to using rsync months ago. So I guess this is just to be sure %100.
This is ...
0
votes
1
answer
56
views
rsync cache/deduplication for similar files
I am using rsync to transfer/update 100s of TB from one fileserver to another, millions of files.
Many of those files are quite similar (but not identical).
I was wondering if there's a way to benefit ...
- 315
2
votes
0
answers
228
views
Why on earth would rsync over SMB run nearly five times faster than rsync-to-rsync (over SSH)?
I just modified many files on a 36TB Unraid array (not quite half full yet). Now I want to sync all of those changes up to a backup array using rsync.
My usual rsync backup was going a lot more slowly ...
- 129
0
votes
0
answers
128
views
Rsync stopped syncing files
I recently moved to a new server and my rsync script that ran perfect for months stopped working. I'm at a loss for why. I have a 'dev' server and production server. When I run the following ...
- 101
0
votes
1
answer
87
views
rsync cannot write files to folders it created
As the title says, Rsync is not able to copy files to the folders that Rsync itself created seconds before. I thought it could be a problem with curlftpfs, which is what mounts the ftp folder.
I ...
- 162
0
votes
1
answer
38
views
copy many files with big sizes over rsync
I have database server on AWS and the backup of databases are over 1000 files with sizes can reach 5GB,
I am using a script to copy the backup to my data center using rsync, but the copy is slow, I ...
0
votes
1
answer
93
views
Suddenly, outbound scp, sftp, rsync, & ssh /bin/true fail; ssh still works
Problem
On my MacBook Pro running Mac OS 10.15.7, I cannot do outbound scp, sftp, or rsync to seemingly any server (I've tried ~10 I own). I also cannot run remote commands by ssh, e.g. ssh ...
1
vote
1
answer
103
views
Why is there a discrepancy between Finder and rsync when handling dot files?
Starting from two files
~/foo/bar1.txt
~/foo/bar2.txt
on my Mac's root volume, I copy the folder ~/foo (it's large; this is just an example) to /Volumes/Kiwi. I know from past experience that Finder ...
- 111
2
votes
2
answers
95
views
rsync uncompressed to single compressed archive
Question
How can I rsync uncompressed (but highly compressible) data to a single compressed archive (.tar.gz, .zip, etc) without transferring all the data each time?
Current
This is what I am using ...
- 121
0
votes
1
answer
75
views
How to make ddrescue rescue unreadable files along with rsync run when you backup files, when they exist only at the source?
How to make ddrescue rescue unreadable files along with rsync run when you backup files, when they do exist only at the source?
I think that major programs that work on backup disks must "merge&...
0
votes
1
answer
347
views
Rsync daemon, if it isn't using ssh, how does it authenticate the user trying to transfer data?
I understand that you can avoid ssh authentication with rsync daemon, but what method of authentication is it using then? How is it transferring the data?
0
votes
0
answers
28
views
rsync replicate existing rsync server data
I have a master backup system that rsync's with windows servers and does incremental backups with the --link-dest flag. Works great. Each day it looks at the previous day and only backs up what ...
- 1
0
votes
0
answers
78
views
Rsync Webmin from master VPS1 to child VPS2
I am attempting to replicate(rsync) the etc/webmin directory from the Master VPS (VPS-1) to the Child VPS (VPS-2). However, after performing the copy, the server summary on the Child VPS still ...
0
votes
1
answer
90
views
rsync directory is smaller at destination
I am having a difficult time using rsync from my computer to a remote server. I have a folder with a few sub directories and a lot of small files. On my computer the total size of my directory (using ...
- 1
0
votes
1
answer
71
views
Rsync when system resource utilization is low
I have a machine that suffers periods of resource contention (CPU or Memory or Disk or some combination) and I don't want Rsync to add to the problem when that occurs. Is it possible to setup Rsync to ...
3
votes
2
answers
907
views
Test network transfer speeds with rsync from a server with limited storage
I want to test transfer speeds from server A to server B, but server A has limited disk space (<50GB) and because the network speeds are fast within the same data center, transferring 50 GB may be ...
- 133
1
vote
1
answer
374
views
rsync: No space left on device (28)... with a lot of free space
I need to make a copy of 200GB in a 500GB external SSD disk to a 2TB NVMe M2 SSD, which has more than 1TB free. All my other disks and partitions have lots of GBs of free space, too (well, except for ...
- 123
0
votes
0
answers
45
views
Migrating form Hostinger VPS to DigitalOcean Droplet Ubuntu 22.04
I am planning to migrate a Linux server to a new machine using rsync for a complete clone. What are the best practices to ensure a successful migration, including preserving configurations, libraries, ...
- 1
2
votes
1
answer
96
views
Filenames change when RSYNC to NAS with CIFS or NFS
I have noticed that after a RSYNC to a Truenas core 13 NAS with either a nfs or cifs mounted folder, file names will change. Source device Kubuntu 22.04
example: Pirates of the Caribbean: At World'...
- 61
0
votes
1
answer
121
views
rsync skip transferring files that take longer than n minutes
I run rsync as a cron job within a script. Sometimes transfer speed falls down and my large files (say over 1G) take so long to transfer and this stops my script functionality. Is there a way I can ...
- 35
2
votes
1
answer
212
views
Discrepancy in file usage after rsync
I copied files from an exFAT filesystem to an ext4 filesystem:
sudo rsync -avz /media/user/1TB-SD-EX/ -e ssh example.com:/data/pc/sd1tb
After successful run I check source and destination size with ...
- 463
0
votes
2
answers
943
views
How to install rsync 3.2.7 on Ubuntu 20.04.2
I want to isntall rsync 3.2.7 on Ubuntu 20.04 - however I'm not sure if that's possible; currently I have 3.1.3 installed on the system (which is installed by default).
I found this question but it ...
- 1,033
1
vote
1
answer
51
views
extract full path FIND with special character handling
Executing a comand like:
find /some_path -mtime -3 -type f -name "*.txt"
the out is something like:
/some_path/file 1.txt
/some_path/file2.txt
If I include this output in rsync with ...
- 11
1
vote
0
answers
68
views
Backup drive on Linux system keeps getting corrupt. How to troubleshoot?
My webserver (Debian BullsEye) has an internal 2TB harddrive which sole purpose is to hold a backup of the server's main HD. Every day a cronjob is started that creates a backup using rsync. But every ...
- 259
0
votes
1
answer
172
views
rsync Connection getting failed to remote server
I've configured a rsync server on a Windows Server 2019 (Server Name - ServerX).
rsync is working fine within the ServerX.
But none of my local machines are connected to ServerX via rsync.
rsync is ...
0
votes
0
answers
123
views
How to rsnapshot keep mirror backup and 'retain' directories have only deleted and changed files
I'm triyng/testing rsnapshot in my Ubuntu desktop to make incremental backups from a remote server to my desktop.
I use a custom config file where I set retain alpha 7
I executed rsnapshot -V -c /path-...
1
vote
1
answer
241
views
rsync is not preserving file timestamps with the 'a' option when writing to cifs mount
My Ubuntu 22.04 VM (on my synology nas) preserves file timestamps when writing to it's local filesystem, but not when writing to a cifs mount (on the nas). This causes it to re-grab the non-preserved ...
- 11
0
votes
1
answer
384
views
"rsync: [generator] mknod … failed: File name too long (36)" though file & path name lengths are way shorter than 255 & 4096
On a QNAP NAS I'm trying to copy a huge backup directory from internal to an USB disk. The backup set was created using back-in-time, therefore it uses hard-links for deduplication and encfs generated ...
- 141
0
votes
0
answers
91
views
backup /home over sftp - root
My current task is to set up a (remote) backup (using rsync/rsnapshot) over sftp of /home (linux). Obviously some files are read only for the owner (ie -r-------; and should stay that way) so i guess ...
- 1
0
votes
1
answer
82
views
rsync failure not detected by duply/duplicity; corrupted backup "successful"
First of all, I recognise these are not the most recent versions of duply and duplicity. The client runs servers on various OS versions, and we currently install duply and duplicity from the system ...
0
votes
1
answer
171
views
Does daily rsync OS backup push SSD drives to its limits?
Since past year I'm using Rsnapshot (which uses rsync) to do my daily laptop Linux Ubuntu OS backup to a local NAS.
Some months ago I started having serious problems with the OS stability. It seems ...
- 256
0
votes
1
answer
126
views
rsync: updated files to new folder
rsync allows to copy/synchronize folders without transferring files that already have been synchronized before and that did not change since last run. For that, you typically have to specify the ...
- 117
0
votes
0
answers
84
views
Does the rsync command will end once it is completed
I'm running a command to sync the folder.
I would like to know whether the rsync command will keep on running even if all files are synced.
Since I'm having script
#!/bin/bash
function simSync {
ssh ...
- 1
0
votes
0
answers
225
views
rsync --inplace does not update file
I try to use rsync to backup a virtual box vm. This does not succeed: the disk image is not updated. I tried it on the vdi file alone as a test. The virtual machine was not running.
The file is very ...
- 101
0
votes
0
answers
810
views
Is there a tool for copying files directly from any running Kubernetes pod to another?
I would like to move a large file set from a running Kubernetes pod to another (possibly in another cluster) in order to migrate quickly from one Storage Class to another with little or no downtime.
I ...
1
vote
1
answer
113
views
rsync Delete only Matching files
How can I use rsync (or another program) to delete identical files between directories? To be identical, location and metadata should match (permissions, ownership, timestamp)
For example, I backup ...
- 156
0
votes
0
answers
386
views
Ubuntu 20.04 / 22.04 - Remote ssh or rsync killing all process from user
I got a weird strange odd behavior (there's not enough adjectives for that) on Ubuntu 20.04 LTS and Ubuntu 22.04 LTS.
This is a IBM Db2 test server. The server runs a Db2 service under db2inst1 user ...
- 113
0
votes
1
answer
189
views
rsync: remote command not found (code 127) at io.c(254) [sender=3.1.2]
To establish a backup between a local file server and a remote webspace (with ssh and rsync access) I wanted to use rsync.
With the command
rsync -aPv -e "ssh -vv IonosSB" --delete /volume1/...
- 1
1
vote
0
answers
268
views
Samba AD sysvol sync unison permissions issue
I'm having an issue with syncing the unix permissions and ownership on the sysvol between two Ubuntu Samba AD DCs using unison. I have copied the idmap.ldb file from DC1 to DC2 and the Windows ...
0
votes
0
answers
171
views
Can I prevent rsync from setting attributes on the topmost directory?
I'm currently building a solution to rsync Gerrit data into a Kubernetes container which has storage (a persistent volume) mounted at /var/gerrit/logs/. I am trying to use the rsync command below, but ...
- 1
1
vote
1
answer
537
views
Rsync makes the destination folder unwritable before copying files to it (and then of course it fails with Permission Denied)
I want to use rsync to deploy a project from my dev machine (Windows 11, CygWin) to a remote server (Ubuntu) through SSH. I've made sure that my SSH user has full write access to the destination ...
- 133
0
votes
0
answers
3k
views
rsync error: /etc/ssh/ssh_config: Bad configuration option: clientaliveinterval / clientalivecountmax
I am trying to copy directories from one server to another using rsync using:
rsync -avr /home/<username> <username>@111.111.111.11:/home/<username>/
I get this error (on the ...
- 73
4
votes
0
answers
1k
views
why `rsync --sparse` create much larger file than the source?
I try to rsync between 2 local disks, thus there is no compression. the sparse file has apparent size of 50GB, actual size of 16GB. I tried rsync -a -P --sparse, it did generate sparse file on ...
- 318
0
votes
1
answer
94
views
Optimize rsync for bandwidth usage
I need to use the minimum amount of bandwidth when patching files from a server. How can I configure rsync to achieve this?
I've tried the following for 10Mb file filled with random text, then made ...
- 101
2
votes
2
answers
323
views
rsync exclude file: correct syntax to exclude specific folders whose names include spaces
I'm having a hard time trying to exclude some folders whose names include spaces from a rsync command using the --exclude-from=excludefile.txt option.
So far I've tried:
"Spaces in Folder Name&...
- 43
1
vote
1
answer
883
views
rsync "Operation not supported" errors for ACL on ZFS
While migrating our file server to a ZFS file system with rsync -avxHAXS ... I get errors like:
rsync: [generator] set_acl: sys_acl_set_file(path/to/file, ACL_TYPE_ACCESS): Operation not supported (95)...
- 141
1
vote
1
answer
286
views
how to force rscync to unmunge symlinks
I make backups with rsync (with -a) on my own rsync server, and all links are munged (added /rsync-munged/ before each symlink) - which is good
But today I need to restore my backup from rsync server ...
- 793
0
votes
0
answers
51
views
Tentative of shh->rsync to wrong IP; potential security risk?
I have set up a little RPI backup 'server' (with rpi OS) on my LAN. I use it to create a backup of my main personal machine data using rsync, and I reach it from outside my network through SSH (key-...
0
votes
0
answers
232
views
How suitable is a S3 backend for rsync operations?
I run a large server providing open source software (https://ftp.halifax.rwth-aachen.de), currently serving more than 30 TByte of data with Multi-Gigabit throughput. Data is synchronized and kept up-...
- 334