Questions tagged [filesystems]
A file system (or filesystem) is a means to organize data expected to be retained after a program terminates by providing procedures to store, retrieve and update data, as well as manage the available space on the device(s) which contain it.
1,989
questions
239
votes
17
answers
468k
views
df in linux not showing correct free space after file removal
I have file servers which are used to store files. Files might reside there for a week, or for a year. Unfortunately, when I remove files from the server, df command doesn't reflect the freed up space....
user11350
195
votes
6
answers
85k
views
LVM dangers and caveats
I have recently started using LVM on some servers for hard drives larger than 1 TB. They're useful, expandable and quite easy to install. However, I could not find any data about the dangers and ...
- 13.4k
189
votes
6
answers
478k
views
SSL Certificate Location on UNIX/Linux
Is there any standard or convention for where SSL certificates and associated private keys should go on the UNIX/Linux filesystem?
- 2,195
129
votes
11
answers
18k
views
FreeBSD: Directory called ^C (really!) - how to remove?
I made a typo:
$ history
169 9:34 la /usr/local/etc/
170 9:35 sudo mkdir ^C
171 9:36 sudo mkdir /usr/local/etc/dnsmasq.d
Now I have a file that is called ^C (ctrl+C)!!
When I use ls I just ...
- 1,347
121
votes
10
answers
473k
views
How to extend an ext4 partition and filesystem?
I have a 400GB disk with a 320GB ext4 partition.
I would like to grow the ext4 partition to use the left space (80GB of free space).
+--------------------------------+--------+
| ext4 ...
- 1,997
74
votes
13
answers
71k
views
ZFS vs XFS
We're considering building a ~16TB storage server. At the moment, we're considering both ZFS and XFS as filesystem. What are the advantages, disadvantages? What do we have to look for? Is there a ...
- 843
68
votes
10
answers
106k
views
Question marks showing in ls of directory. IO errors too
Has anyone seen this before? I've got a raid 5 mounted on my server and for whatever reason it started showing this:
jason@box2:/mnt/raid1/cra$ ls -alh
ls: cannot access ...
jaymoo
67
votes
9
answers
126k
views
How do I determine the block size of an ext3 partition on Linux?
How do I determine the block size of an ext3 partition on Linux?
- 3,983
67
votes
4
answers
36k
views
Drawbacks of mounting a filesystem with noatime?
Having every file be updated just when accessing them sounds like a waste.
What's the catch with mounting a file system with the noatime option. What kind of applications/servers relies on the access ...
- 2,438
65
votes
1
answer
24k
views
Why are my XFS filesystems suddenly consuming more space and full of sparse files?
I've run XFS filesystems as data/growth partitions for nearly 10 years across various Linux servers.
I've noticed a strange phenomenon with recent CentOS/RHEL servers running version 6.2+.
Stable ...
- 198k
60
votes
3
answers
9k
views
What is the difference between a soft (symbolic) link and a hard link?
I hear that you can now create soft links in Vista too. So, what is the difference between a soft (symbolic) link and a hard link on UNIX/Linux/Vista?
Are there advantages of using one over the other?...
- 1,525
53
votes
7
answers
38k
views
filesystem for millions of small files
Which Linux filesystem would you choose for best speed in the following scenario:
a hundred million files
~2k file size on average
>95% read access
pretty random access
high concurrency (>100 ...
- 2,304
52
votes
4
answers
106k
views
Linux/Windows/Unix/... file names: Which characters are allowed? Which are unescaped?
Which characters are allowed and which of them must be escaped on the command line in different operating systems?
50
votes
14
answers
62k
views
Linux - What directories should I exclude when backing up a server?
I'm backing up a Linux server and storing it on another server.
I began with a simple
rsync -aPh --del server.example.com:/ /mnt/backup
Then someone pointed out that I shouldn't back up /proc, ...
- 31.9k
49
votes
7
answers
55k
views
What is the best Linux filesystem for MySQL (InnoDB)?
I tried to look for benchmark on the performances of various filesystems with MySQL InnoDB but couldn't find any.
My database workload is the typical web-based OLTP, about 90% read, 10% write. Random ...
- 3,110
49
votes
3
answers
90k
views
.nfsXXXX files appearing, what are those?
I have an application running (on RHEL5) that streams data onto an NFS share. Recently, I saw a lot of .nfsXXXX... (xxx being a hexadecimal number) appearing in its working directory, where the ...
- 2,438
46
votes
3
answers
92k
views
What is the maximum number of files a file system can contain?
Given the current structure of a directory entry on a ext4 file system on Ubuntu, what is the maximum number of files a file system can contain?
What is the general method of calculating the maximum ...
user32678
45
votes
7
answers
9k
views
Removing an (apparently) infinitely recursive folder
Somehow, one of our old Server 2008 (not R2) boxes has developed a seemingly infinitely-recursing folder. This is playing havock with our backups, as the backup agent tries to recurse down into the ...
- 1,147
44
votes
7
answers
51k
views
Why can't you fsck a mounted partition?
It's well-known that you should never fsck a mounted partition. I can understand how this could easily lead to corruption if the filesystem is written to by fsck (e.g., the -a option is used), but why ...
- 3,983
42
votes
7
answers
57k
views
du vs. df difference [duplicate]
I have a fileserver where df reports 94% of / full. But according to du, much less is used:
# df -h /
Filesystem Size Used Avail Use% Mounted on
/dev/sda3 270G 240G 17G 94%...
- 791
40
votes
7
answers
47k
views
What happens if I delete lost+found
When you create some Linux filesystems like ext3 a 'lost+found' directory is created. According to this files will be placed there if files were damaged from some kind of system crash.
What happens ...
- 131k
38
votes
9
answers
68k
views
Need to fix file permissions in a user's home directory
Does anyone have a tool or script that will recursively correct the file permissions on a directory?
On an Ubuntu Linux machine, a bunch of files were copied to a USB disk with full 777 permissions (...
- 1,221
38
votes
3
answers
13k
views
When is fsck dangerous?
Recently I've seen the root filesystem of a machine in a remote datacenter get remounted read-only, as a result of consistency issues.
On reboot, this error was shown:
UNEXPECTED INCONSISTENCY: RUN ...
- 483
38
votes
1
answer
7k
views
Why does deleting a big file take longer?
My understanding is that running rm on a file simply unlinks it, marking the space as free in the filesystem. It should then follow that deleting one file always takes roughly the same amount of time (...
- 2,136
36
votes
2
answers
6k
views
Unable to write to disk but disk isn't full
I am using Ubuntu 12.04 and can't write to any file, even as root, or do any other operation that requires writing. Neither can any process that needs to write, so they're all failing. df says I've ...
- 866
36
votes
5
answers
15k
views
Does btrfs have an efficient way to compare snapshots?
While diffing mounted snapshots would work, it sounds like it could be horribly slow in many cases.
Is there btrfs specific functionality for diffing snapshots? (I was unable to find any in the docs)
- 1,939
36
votes
2
answers
26k
views
See what files are being written to (like iotop but for files, not processes)
I have a Linux server, and it's doing a lot of disk io. There is one process that is doing most of the disk writes. I found this out with iotop, which shows the processes that are doing disk I/O. Is ...
- 31.9k
36
votes
6
answers
37k
views
Mount Docker container contents in host file system
I want to be able to inspect the contents of a Docker container (read-only). An elegant way of doing this would be to mount the container's contents in a directory. I'm talking about mounting the ...
- 573
34
votes
7
answers
12k
views
How beneficial are self-healing filesystems for general usage?
I have recently looked into advanced filesystems (Btrfs, ZFS) for data redundancy and availability and got interested in the additional functionality they provide, especially their "self-healing" ...
- 443
34
votes
4
answers
74k
views
Is there such a thing as a 'quick' format for ext4?
In Windows, it's very fast to format with NTFS. I have a low powered Linux machine, with little RAM. Formatting a 2TB volume to ext4 takes a long long time.
Is there anything I can do to speed up the ...
- 697
33
votes
9
answers
49k
views
Best way to compare (diff) a full directory structure?
What's the best way to compare directory structures?
I have a backup utility which uses rsync. I want to tell the exact differences (in terms of file sizes and last-changed dates) between the source ...
- 13.4k
33
votes
2
answers
21k
views
Copy directory tree without empty directories?
I have the following tree
# upper letters = directory
# lower letters = files
A
|-- B
|-- C
|-- D
|-- e <= file
|-- F
|-- G
I need to copy this tree to another destination, ...
- 1,289
33
votes
8
answers
48k
views
How to workaround the NTFS Move/Copy design flaw?
As anyone that has dealt with file server permissions is aware, NTFS has an interesting design feature/flaw known as the Move/Copy problem.
As explained in this MS KB article, the permissions for a ...
- 599
32
votes
6
answers
5k
views
What are .DS_Store files and why does OSX leave them on Windows shares?
Whenever I access windows shares from OSX 10.5 it leaves .DS_Store files on the remote filesystem. What are they used for, and are they necessary, and can they be prevented from being created?
- 700
32
votes
4
answers
34k
views
Performance difference between ramfs and tmpfs
I need to setup an in memory storage system for around 10 GB of data, consisting of many 100 kb single files(images). There will be lots of reads and fairly periodic writes(adding new files, deleting ...
- 1,841
32
votes
5
answers
12k
views
What are the performance implications for millions of files in a modern file system?
Let's say we're using ext4 (with dir_index enabled) to host around 3M files (with an average of 750KB size) and we need to decide what folder scheme we're going to use.
In the first solution, we ...
- 549
31
votes
5
answers
7k
views
Turning off atime on a filesystem
I am setting up a mongoDB replica set and one of the first things I am suppose to do is turn off atime on the file system. After researching this a bit, I am not opposed to doing this, but I have to ...
- 927
31
votes
11
answers
21k
views
Distributed Storage Filesystem - Which one/Is there a ready to use product?
With Hadoop and CouchDB all over in Blogs and related news what's a distributed-fault-tolerant storage (engine) that actually works.
CouchDB doesn't actually have any distribution features built-in, ...
30
votes
2
answers
58k
views
Transparent compression filesystem in conjunction with ext4
I am trying to test a project that needs compressed storage with use of the ext4 file system since the application I use relies on ext4 features.
Are there any production/stable solutions out there ...
- 319
30
votes
3
answers
18k
views
The XFS filesystem is broken in RHEL/CentOS 6.x - What can I do about it?
Recent versions of RHEL/CentOS (EL6) brought some interesting changes to the XFS filesystem I've depended on heavily for over a decade. I spent part of last summer chasing down an XFS sparse file ...
- 198k
28
votes
6
answers
18k
views
Preferred format of file names which include a timestamp
As we all know "unix" can have anything in a file except '/' and '\0', sysadmins however tend to have a much smaller preference, mainly due to nothing liking spaces as input ... and a bunch of things ...
- 751
26
votes
5
answers
59k
views
How do I identify the file system of a drive in Mac OS?
I have an external drive hooked up to my Mac, and I'm trying to determine things like, e.g., is this HFS or FAT, is it 32-bit or 64-bit, etc. It seems like there should be some trivial command that ...
- 444
26
votes
12
answers
8k
views
Storing and backing up 10 million files on Linux
I run a website where about 10 million files (book covers) are stored in 3 levels of subdirectories, ranging [0-f]:
0/0/0/
0/0/1/
...
f/f/f/
This leads to around 2400 files per directory, which is ...
- 4,557
25
votes
2
answers
7k
views
In Linux, why isn't superuser root's folder under /home?
You can create a user that has privileges like root, and it's home directory will fall under /home/username. Why does root get its own folder at the top level of the file system? Is this just ...
- 3,250
25
votes
6
answers
35k
views
What's the maxium number of files a Unix folder can hold?
What's the maxium number of files a Unix folder can hold?
I think it will be the same as the number of files.
PERR0_HUNTER
25
votes
7
answers
5k
views
Do I need to defrag unix filesystems?
I know Windows file systems are very sensitive to fragmentation and Unix such as UFS, ZFS, ext3, or ext4 are not as sensitive.
But, is there any reason to defragment these filesystems at all?
If so, ...
- 4,164
25
votes
9
answers
23k
views
Maximum number of files in one ext3 directory while still getting acceptable performance?
I have an application writing to an ext3 directory which over time has grown to roughly three million files. Needless to say, reading the file listing of this directory is unbearably slow.
I don't ...
- 1,809
25
votes
2
answers
19k
views
How to track all files a process opens in its lifetime
I am aware of using lsof for checking the files currently accessed by a process. Does there exist a way to see all files that an application opens in its lifetime?
- 1,679
25
votes
2
answers
8k
views
ZFS Data Loss Scenarios
I'm looking toward building a largish ZFS Pool (150TB+), and I'd like to hear people experiences about data loss scenarios due to failed hardware, in particular, distinguishing between instances where ...
- 557
24
votes
5
answers
20k
views
last time a filesystem was checked?
What command can tell me the last time that a filesystem was fsck'd? A date would be nice, but I'd settle for the mount count since the last fsck.
I've looked all around for this in fsck*, lsattr, ...
- 8,841