Questions tagged [disk-cache]
Disk-cache: reserved area in RAM to speed-up disk access
55
questions
26
votes
3
answers
40k
views
Linux not freeing large disk cache when memory demand goes up
Running Ubuntu on a 2.6.31-302 x86-64 kernel. The overall problem is that I have memory in the 'cached' category that keeps on going up and will not be freed or used even when our application needs it....
- 491
14
votes
2
answers
3k
views
Safety of write cache on SATA drives with barriers
I've been reading lately about write caching, NCQ, firmware bugs, barriers, etc regarding SATA drives, and I'm not sure what's the best setting that would make my data safe in case of a power failure.
...
- 270
14
votes
2
answers
3k
views
How to setup an NFS server that caches a network share?
User data is stored on two fairly large (>1 PB) OpenStack Swift storage clusters. Let them be Cluster A and Cluster B.
In addition, there are several PoPs that need to interact with that data. ...
- 240
9
votes
2
answers
4k
views
Are bcache and/or dm-cache considered stable for production in 2016?
I would like to use linux SSD caching (dm-cache or bcache) with Debian Jessie production servers. (kernel 3.16)
My question: Are dm-cache and bcache modules reliables in linux 3.16 ? Do I need to ...
- 93
8
votes
3
answers
25k
views
Use Varnish Cache only on-disk
The situation is:
I' m building a PHP application and need http caching.
Varnish is great and lots of people tell me that Varnish stores the cached data in RAM. I want it to cache on hard disk.
Is ...
Great Kuma
7
votes
2
answers
5k
views
Windows Server virtual disk cache settings
In Windows under Azure or Hyper-V, the Disk Policy write cache settings for a virtual disk will always appear to be enabled from the server configuration, regardless of the actual status of write ...
- 171
7
votes
4
answers
11k
views
Disable all disk caching for apache2 on linux
For benchmarking purposes I want to force Apache 2 to load each requested file from disk instead of loading it from a cache in memory. From what I have read doing a sync followed by
echo 3 > /proc/...
- 375
6
votes
9
answers
10k
views
Rapid SSD vs. RAM in servers
Per GB prices of rapid SSD (like Intel X25-E) come close to prices of high-end RAM.
So what extra advantages does SSD give you? What are particular reasons, why you buy SSD, instead of just putting ...
6
votes
3
answers
10k
views
Why does mdadm write unusably slow when mounted synchronously?
I have a 6 disk raid6 mdadm array I'd like to benchmark writes to:
root@ubuntu:~# cat /proc/mdstat
Personalities : [raid6] [raid5] [raid4]
md0 : active raid6 sda[0] sdf[5] sde[4] sdd[3] sdc[2] sdb[...
- 974
5
votes
4
answers
18k
views
Should I disable write caching on my Windows 2008 VM?
I have a Windows Server 2008 x64 Standard virtual machine that runs on a machine with a hardware RAID controller, a Perc 6/i, which has a battery on-board.
Doing everything I can for additional ...
- 4,212
5
votes
1
answer
3k
views
Linux read disk cache and NFS
I am interested in the impact of the read disk cache on accessing a file through NFS. Lets say I have a file on NFS. I access it (do "cat myfile") from a linux host "A". If there's enough RAM a file ...
- 153
5
votes
2
answers
427
views
Data corruption when disk write caching is enabled
I've recently learned that enabling disk write caching can significantly improve system performance. However, I'm concerned about the potential risks of data corruption or loss in case of a sudden ...
5
votes
1
answer
1k
views
Solaris 10: Drop disk cache? (to benchmark app)
Is there a command to drop the ZFS and UFS disk caches? (Preferably just one folder hierarchy.)
I'm trying to tuning the performance settings of an application that read/writes data from a large ...
- 1,099
4
votes
1
answer
3k
views
How much does HDD cache matter with Linux softraid?
I'm in a process of renewing/expanding my disk sets, but not quite sure what kind of disks to get, cache-wise.
What difference does disk cache amount of 16/32/64MB do, in capacities of, say, 1/1.5/...
- 305
4
votes
2
answers
3k
views
Write caching in linux
I have read a lot on serverfault and searched on google about write caching, but still cant find the answer.
I have an HP ProLiant DL380 G5 with HW raid with 512 MB Battery backed cache. I use debian ...
- 43
4
votes
1
answer
5k
views
Disable file cache for mount point
I ran into a bug similar to this one:
RHEL NFS client returns NULL bytes when reading a growing file
So my solution was to check for \0 Bytes and reread the file. Problem here is, that the wrong ...
- 141
4
votes
1
answer
2k
views
Linux - Disk / Filesystem as a LRU Cache (with transparent expiry)
I have a case where I'd want to use local disk as a LRU cache for (hot) files from a separate web service (something like S3). If the file doesn't exist on disk, the file is read over the internet, ...
- 151
3
votes
4
answers
5k
views
Would a PHP application benefit from being served from a RAM drive?
I am in charge of hosting a PHP application that is large and slow, but easy to scale. The application is entirely static, with writable disk storage needed. We've profiled the application, and the ...
- 2,136
3
votes
1
answer
4k
views
How to ensure HDD Write Caching is disabled on MS Storage Spaces
I have a Storage Spaces pool on a server that is not currently power protected. It consists only of mechanical hard drives.
I want to compare performance of write caching enabled vs disabled so I can ...
- 123
3
votes
1
answer
89
views
varnish delete least requested files
I see that varnish can be configured to set -smalloc or -sfile with a certain size.
I want to set a file cache of 1G such that least requested files are deleted first when cache is full. Is this ...
- 378
3
votes
1
answer
2k
views
On a system with 64GB mem the Linux Buffer run full while copying with dd to dev null and io stops till manual drop_caches
I am running a server with linux software raid 10. It is a dual CPU system with 64GB Ram. 2x16GB dimms related to each of the CPUs. I want to use dd to backup kvm virtual machines and run into a ...
- 73
2
votes
3
answers
1k
views
Would the data on the RAID Card Cache and Drive Cache be lost when a kernel panic happens?
Is a BBU necessary when you have A+B power?
Would the data on the RAID Card Cache and Drive Cache be lost when a kernel panic happens and you are forced to do a cold reset?
- 529
2
votes
1
answer
5k
views
Why is mongod not using all available RAM?
We have an mongod instance runnning on a VM, and it doesn't seem to be using all available memory. It's page-faulting significantly more than usual, and the system's performance has been significantly ...
- 356
2
votes
1
answer
940
views
ram disk mirroring part of physical disk
What I would like to do/have is:
Some kind of application/driver that creates a RAM drive/partition that is actually assigned a drive letter (eg. D:). This drive letter should be based on/mirroring a ...
- 147
2
votes
1
answer
385
views
Linux filesystem page cache - hot or not
How can you tell how much of the filesystem cache is hot (frequently accessed) or not?
The use case would be determining when you are approaching the limits of your memory but before you actually hit ...
- 21
2
votes
1
answer
123
views
CentOS VM only using a third of available memory
I have a VM slice running CentOS, on Parallels.
Here is the output of free -m
[root@server ~]# free -m
total used free shared buffers cached
Mem: 960 ...
- 141
2
votes
1
answer
1k
views
How Does The Linux Disk Cache Work With iSCSI and GFS2?
I'm planning to share a GFS2 filesystem via iSCSI to my web servers.
Are the files read by my web servers cached locally using the linux disk cache stored in memory or are they only cached on the ...
- 51
2
votes
2
answers
2k
views
Why would diskspd perform better without cache?
We are currently investigating high disk latency on a Windows server 2012 r2 that run as an SQL server. It is a virtual machine under VMware and the datastore of the faulty disk is linked with a very ...
- 21
2
votes
2
answers
649
views
Active process swapping despite free memory
For some reason our system starts swapping (actively used memory) at around 60GB used memory. (see edit below, it seems I/O and disk cache usage, even of previously run processes has an effect)
...
- 121
2
votes
2
answers
2k
views
How to increase the disk write buffer for rsync on Mac OS X?
I have to copy some TB of data from one external drive to another (mostly ~400 MB files).
The source is NTFS-formatted USB2 and the best read speed I can get from it is 7MB/s.
The target is HFS+ USB3 ...
- 185
2
votes
1
answer
3k
views
Apache when to use mod_cache? disk_cache vs memory_cache?
I'm making use of Apache 2.4.10 on Ubuntu 15.04 and my entire server runs on a SSD and is virtualised in Hyper-V.
When should I enable mod cache in Apache 2.4?
Currently I have pretty good TTFB for ...
- 117
2
votes
3
answers
5k
views
High disk I/O activity in CentOS server
I have about 16 websites in a CentOS dedicated, and I am having some problems on high traffic hours, it seems to be a high disk I/O activity causing a general slowdown.
I've installed atop and this ...
- 21
1
vote
1
answer
4k
views
How do I see if my disk cache is disabled on Linux?
I have an ubuntu linux virtual machine here and I think there is no disk cache.
For example this is the output from free -m:
total used free shared buffers ...
- 31.9k
1
vote
1
answer
1k
views
Linux /proc/sys/vm/drop_caches in guest VMs
Question:
Would it ever be a good idea to disable page cache inside guest VM's and instead rely on the ZFS ARC (and SSD based L2ARC) of the host?
Context:
I'm asking since I'm running a Proxmox ...
- 883
1
vote
3
answers
327
views
Confused by write cache flags
I'm looking to maximise performance of my hard drive by enabling write caching. I understand the risks in doing this, but believe them to be insignificant for how I will use it.
On the HD Properties ...
- 1,583
1
vote
1
answer
2k
views
Windows Server 2012 write caching
From MSDN:
By default, Windows caches file data that is read from disks and written to disks. [..] Caching is managed per file object. The policy of delaying the writing of the data to the file and ...
- 375
1
vote
1
answer
2k
views
What is the "right way" to combine sync and drop_caches?
When I need to do echo 3 > /proc/sys/vm/drop_caches with a sync beforehand, is it better to use
sync && echo 3 > /proc/sys/vm/drop_caches or
sync ; echo 3 > /proc/sys/vm/drop_caches....
- 38
1
vote
1
answer
16k
views
What is read cache in "[sda] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA"?
I'm trying to figure out what the read cache part of the following message refers to:
[sda] Write cache: enabled, read cache: enabled, doesn't support DPO
or FUA
Is it "IDE drive's read-...
- 9,488
1
vote
0
answers
246
views
How to make two squid servers share the same disk cache?
I'm trying to setup squid proxy in a kubernetes environment as a caching server with 3 replicas. I wanted to know whether all three replicas can share the same disk storage, or I'll have to assign ...
- 21
1
vote
0
answers
159
views
Debian max_ratio setting for device not working
My debian 8 machine has 2 drives. Sometimes I move/copy files from drive A to drive B. Drive B is slow, so the file system cache is full causing the system to be slow/unresponsive when copying large ...
- 83
1
vote
0
answers
890
views
Disable nginx fastcgi_cache when disk is full
When there is no disk space, nginx is still able to work correctly.
It can serve static files, and it can pass the request to a fastcgi server (such as PHP), and it can serve the dynamic response to ...
- 299
1
vote
1
answer
619
views
Apache mod cache mem/disk saves the cache for 10 seconds and than re-cache
I have this set up in my 000-default:
<VirtualHost 192.168.1.10:80>
ServerName live.website.com
DocumentRoot /var/www/live.website.com
SetOutputFilter INFLATE;proxy-html;DEFLATE
...
- 11
1
vote
0
answers
278
views
Disk cache (write back) disables automatically
some days ago I discovered (using Munin) that my server had high iowait times.
The reason was that write cache was disabled. I enabled it using hdparm -W1 /dev/sda but after some time (ramdonly) it ...
- 11
0
votes
1
answer
690
views
Raid Caching configuration suggestions
I'm wondering if there's the optimal Cache setting for the following Raid controllers and raid setups. (OS: windows 2008 R2, App: II7, DB: SQL Server 2008 R2 Enterprise)
PERC 6/i:
(The default ...
- 169
0
votes
1
answer
3k
views
Google Cloud Function warning "OpenBLAS WARNING - could not determine the L2 cache size on this system"
I get the warning
OpenBLAS WARNING - could not determine the L2 cache size on this system, assuming 256k
in the logs of the "LOGS" tab of the Cloud Function.
There is already a Q/A on this ...
google-cloud-platform
0
votes
1
answer
129
views
Run similar disk intensive tasks parallel or serial on regular Linux?
Scenario
I'm on a regular Ubuntu 18.04 LTS with ext4 Filesystems.
I'm using restic to backup my machines. Restic is a backup tool that support different backends (local, sftp, aws, gcs, ..) but only ...
- 285
0
votes
1
answer
276
views
Questions about Varnish file storage engine
I have Varnish cache in front of some websites with the following storage engine configuration:
Storage: file
Size: 130GB
The server where it is installed has the following configuration:
RAM: 7.5 ...
0
votes
1
answer
923
views
nginx proxy_cache or Memcached
I currently use nginx's proxy_cache for file-based caching in front of three Django applications (Apache/mod_wsgi). For some locations/URLs, I disabled proxy_cache and used Memcached (per view) within ...
- 257
0
votes
1
answer
1k
views
Which Linux filesystem is best for a huge Apache mod_disk_cache partition?
Talking about a huge (50GB, 500.000 entries) Apache disk_cache partition: Which Linux filesystem performs best for this task?
In my example, the partition has a lot (500.000) of very small files (<...
- 394
0
votes
1
answer
1k
views
Linux server with JBoss , the system cache increased each day
We run an Apache Mod_proxy cluster with 3 JBoss in three dedicated server. Because we need share many files (small object files which our apps need to read/write ) between these JBoss servers ,we ...
- 47