My employer acquired a company with one particular piece of software that stores a lot of PDF and PNG file files in one directory. When I first replicated it from AWS, there were about 11.5 million files. Now the number is approaching 13 millon and performance is -- to be charitable -- pathetic.
The directory has to be shared between four servers so just attaching a LUN to each server is out. When I did the original copy I tried an ext4 filesystem but I started having serious problems at about 10 million. I considered trying XFS but the short lead time demanded I just get them compied. I finally put them on a Dell Isilon which has a UFS file system and runs BSD. The directory is exported using NFS.
If the decision is to build a new NFS server just for this, which file systems will be able to handle such a ridiculous number of files and still give decent performance when retrieving them? I know the best solution would be to break things up so there are not so many files in one directory but in the contest between fast, cheap, and good, the good always gets last place.