Linux : disable disk caching

c_hockland used Ask the Experts™
I need to diable disk caching on RHEL

# free
             total       used       free     shared    buffers     cached
Mem:     528493876   86554832  441939044          0     930332   78327364
-/+ buffers/cache:    7297136  521196740
Swap:     16777208          0   16777208

in the output above, the system is using 86554832  bytes of memory, but almost all of it is disk cache, 78327364 bytes

I want to disable it entirely, and reboot.

how can i do this ?
Watch Question

Do more with

Expert Office
EXPERT OFFICE® is a registered trademark of EXPERTS EXCHANGE®
Chris SandriniSenior System Engineer
Top Expert 2012

There is no way you can directly disable this feature. But there is a few kernel parameters that help you shape its behaviour
Why do you want to disable it?  What problem are you trying to solve?


My DBA has asked me to disable it because he thinks the performance issues he is having on his Oracle DB might be related to the cache value that i posted.
Ensure you’re charging the right price for your IT

Do you wonder if your IT business is truly profitable or if you should raise your prices? Learn how to calculate your overhead burden using our free interactive tool and use it to determine the right price for your IT services. Start calculating Now!

What on the basis that it is taking up RAM that Oracle could use?
given the fact that 441G out of 528G are free, that would be weird to say the least

if his point is that the ram is effectively divided by 2 because the os and the rdbm both cache the same data, this is true if oracle's tablespaces are on the filesystem, but since most of the ram is unused, you probably would not care.

having 78G of cache used means oracle hardly uses more than that in terms of disk access on the filesystem. it looks like the performance problem is elsewhere

btw gosh what the hell is that machine ? how big and intensely used a database must be to need this ?

you may want to run iostat and post information regarding the disk layout if you want to know why oracle is slow.


back to your question, you can disable write cache using -o sync in fstab. i do not know of a simple way to deactivate read cache. anyway, it is pretty foolish to even try.

any decent rdbm including oracle can use directio both for tablespaces and logs which effectively bypasses the caches for what oracle uses. you can also give oracle raw disks.
Top Expert 2015

OracleDB can skip disk cache using direct-io, are you sure its a DBA?


i forgot to mentioned .... these are ASM ( oracle disks )

i dont see these asm disks @ /etc/fstab

any ideas ?
Top Expert 2015

So they use direct io and kernel aio. There is no OS buffer in between.


well , i realized that linux doesnt know and doesnt care about asm disks.
I will still have to disable write cache on the linux level
Top Expert 2015

No, there is no linux write cache in the way of ASM

Only thing you can change is IO scheduler per disk or system default (tip - elevator=deadline is default on UEK)
That will change order aio requests are served, namely deadline makes sure they ground asap.
asm disks are by nature synchronous so "-o sync" from my previous comment does not apply, my bad

you'll find below a bunch of raw information but no real answer

using unbuffered IO is a matter of using directio or not which is something for the application to decide, not the OS. it is usually a parameter to the open system call and can be changed using ioctl on an open file descriptor. on linux there is also a family of functions (io_setup, io_getevent...) that will always be buffered by nature.

as far as i know, linux does not actually have a buffer cache separated from the filesystem's ( any one can confirm/infirm ? )

anyway, there is virtually no way that buffered IO will be slower than directIO. this is espetially true when you have that much free RAM. most OSes will use unused RAM for their buffer cache. this enhances disk writes speed a lot, and possibly writes with the use of prefetch.

there are high chances that the cached memory you see is totally unrelated to asm anyway. possibly your redo logs and the likes are on a regular filesystem ?

or oracle is configured to do asynchronous read/writes on asm (i have no idea wether this would appear as "cached" in top... if someone knows, thanks for sheding some light on the matter)

this links details different oracle behaviour in conjunction with asm or fiesystems. you'll probably find it usefull.

hope some of this helps

ps : maybe if you give more details about the actual performance problem (and run an iostat when something seems wrong), we could give much better hints regarding what to work on. disabling the buffer cache does not seem like a good candidate. if your dba has a theory, i'll be happy to know about it
Top Expert 2015

One can apply disk scheduler to block device backing raw device (actually there are heaps of tunable parameters, but "deadline" is what good disk firmware does, or noop if you trust disk/raid firmware, probably better test which does better - you can apply them to live partition, so if oracle admin has clean room benchmark let them help you)

That would operate on "buffers" seen in "free" (you cannot avoid those buffers, it is where data of disk request goes before or from disk)
Then there comes various filesystem cached, out of scope when handling raw devices, essentially free RAM will be used as filesystem cache.

If you want you can try disabling disk's/raid's internal cache (at the (huge) cost of performance)
So the consensus of the experts on this thread appears to be that none of us can understand why your DBA wants the cache disabled as it doesnt make any sense.

Yes the cache useage looks large, but there is a huge amount of RAM still free, so the system is not memory constrained.

And although you CAN disable the write cache, your whole system is going to suffer the performance degradation this will bring.

So lets see the DBA's justification of why disabling the caching on this system is going to solve whatever the original issue is.
Top Expert 2015

I got it (maybe)
Is your DBA complaining about oracle being swapped out? Do you use AMM too?
how could anything be swapped on a machine that has that much free ram ? if i'm not mistaken, even if a process could grow that big, it would be oem-killed before it would be swapped.
Top Expert 2015

It happens. there is cache pressure and swappiness settings.
Common scenario is oracle making buffered writes to backup (extfs or nfs) and verifying, to make sure less used /dev/shm pages go to swap file.
@gheist : yes, these settings do exist, but how would a backup + backup check swap out anything in this case ? espetially shared pages which are unswappable most of the time (if not always on linux ?)


@c_lockland : i think we're moving away from your problem

there seems to be a mixup between buffers and ansynchronous writes in this thread :

when writing through a filesystem, you write through the kernel's VFS so there is no way you can bypass buffers, whatever you do

even with synchronous writes, buffers get written to. that does not mean these buffers are dirty (other than transiently during the write operation). there might be a way to force them to be deleted but the system will do so by itself whenever an app requires some ram before it will swap anything


when accessing disks directly, you bypass the vfs, and as far as i gather there is no separate buffer cache (well not that big anyway) on linux so the buffers are of no concern in this case


do you use asm for everything ?
does asm perform caching ?
do you use directIO for whatever is not managed by asm (logs ?) ?

what do you expect ?
Top Expert 2015

AMM uses tmpfs and not systemV ipc and SGA can go to swap....

Do more with

Expert Office
Submit tech questions to Ask the Experts™ at any time to receive solutions, advice, and new ideas from leading industry professionals.

Start 7-Day Free Trial