Improve company productivity with a Business Account.Sign Up

x
  • Status: Solved
  • Priority: Medium
  • Security: Public
  • Views: 5778
  • Last Modified:

Linux : disable disk caching

I need to diable disk caching on RHEL

# free
             total       used       free     shared    buffers     cached
Mem:     528493876   86554832  441939044          0     930332   78327364
-/+ buffers/cache:    7297136  521196740
Swap:     16777208          0   16777208

in the output above, the system is using 86554832  bytes of memory, but almost all of it is disk cache, 78327364 bytes

I want to disable it entirely, and reboot.

how can i do this ?
0
c_hockland
Asked:
c_hockland
  • 7
  • 4
  • 3
  • +2
1 Solution
 
Chris SandriniSenior System EngineerCommented:
There is no way you can directly disable this feature. But there is a few kernel parameters that help you shape its behaviour

http://unixadminschool.com/blog/2013/10/rhel-6-controlling-cache-memory-page-cache-size/
0
 
Gerald ConnollyCommented:
Why do you want to disable it?  What problem are you trying to solve?
0
 
c_hocklandAuthor Commented:
My DBA has asked me to disable it because he thinks the performance issues he is having on his Oracle DB might be related to the cache value that i posted.
0
A proven path to a career in data science

At Springboard, we know how to get you a job in data science. With Springboard’s Data Science Career Track, you’ll master data science  with a curriculum built by industry experts. You’ll work on real projects, and get 1-on-1 mentorship from a data scientist.

 
Gerald ConnollyCommented:
What on the basis that it is taking up RAM that Oracle could use?
0
 
skullnobrainsCommented:
given the fact that 441G out of 528G are free, that would be weird to say the least

if his point is that the ram is effectively divided by 2 because the os and the rdbm both cache the same data, this is true if oracle's tablespaces are on the filesystem, but since most of the ram is unused, you probably would not care.

having 78G of cache used means oracle hardly uses more than that in terms of disk access on the filesystem. it looks like the performance problem is elsewhere

btw gosh what the hell is that machine ? how big and intensely used a database must be to need this ?

you may want to run iostat and post information regarding the disk layout if you want to know why oracle is slow.

----

back to your question, you can disable write cache using -o sync in fstab. i do not know of a simple way to deactivate read cache. anyway, it is pretty foolish to even try.

any decent rdbm including oracle can use directio both for tablespaces and logs which effectively bypasses the caches for what oracle uses. you can also give oracle raw disks.
0
 
gheistCommented:
OracleDB can skip disk cache using direct-io, are you sure its a DBA?

http://www.oracle-base.com/articles/misc/direct-and-asynchronous-io.php
0
 
c_hocklandAuthor Commented:
i forgot to mentioned .... these are ASM ( oracle disks )

i dont see these asm disks @ /etc/fstab

any ideas ?
0
 
gheistCommented:
So they use direct io and kernel aio. There is no OS buffer in between.
0
 
c_hocklandAuthor Commented:
well , i realized that linux doesnt know and doesnt care about asm disks.
I will still have to disable write cache on the linux level
0
 
gheistCommented:
No, there is no linux write cache in the way of ASM

Only thing you can change is IO scheduler per disk or system default (tip - elevator=deadline is default on UEK)
That will change order aio requests are served, namely deadline makes sure they ground asap.
0
 
skullnobrainsCommented:
asm disks are by nature synchronous so "-o sync" from my previous comment does not apply, my bad

you'll find below a bunch of raw information but no real answer

using unbuffered IO is a matter of using directio or not which is something for the application to decide, not the OS. it is usually a parameter to the open system call and can be changed using ioctl on an open file descriptor. on linux there is also a family of functions (io_setup, io_getevent...) that will always be buffered by nature.

as far as i know, linux does not actually have a buffer cache separated from the filesystem's ( any one can confirm/infirm ? )

anyway, there is virtually no way that buffered IO will be slower than directIO. this is espetially true when you have that much free RAM. most OSes will use unused RAM for their buffer cache. this enhances disk writes speed a lot, and possibly writes with the use of prefetch.

there are high chances that the cached memory you see is totally unrelated to asm anyway. possibly your redo logs and the likes are on a regular filesystem ?

or oracle is configured to do asynchronous read/writes on asm (i have no idea wether this would appear as "cached" in top... if someone knows, thanks for sheding some light on the matter)

this links details different oracle behaviour in conjunction with asm or fiesystems. you'll probably find it usefull. https://access.redhat.com/site/documentation/en-US/Red_Hat_Enterprise_Linux/5/html/Tuning_and_Optimizing_Red_Hat_Enterprise_Linux_for_Oracle_9i_and_10g_Databases/chap-Oracle_9i_and_10g_Tuning_Guide-Configuring_IO_for_Raw_Partitions.html

hope some of this helps

ps : maybe if you give more details about the actual performance problem (and run an iostat when something seems wrong), we could give much better hints regarding what to work on. disabling the buffer cache does not seem like a good candidate. if your dba has a theory, i'll be happy to know about it
0
 
gheistCommented:
One can apply disk scheduler to block device backing raw device (actually there are heaps of tunable parameters, but "deadline" is what good disk firmware does, or noop if you trust disk/raid firmware, probably better test which does better - you can apply them to live partition, so if oracle admin has clean room benchmark let them help you)

That would operate on "buffers" seen in "free" (you cannot avoid those buffers, it is where data of disk request goes before or from disk)
Then there comes various filesystem cached, out of scope when handling raw devices, essentially free RAM will be used as filesystem cache.

If you want you can try disabling disk's/raid's internal cache (at the (huge) cost of performance)
0
 
Gerald ConnollyCommented:
So the consensus of the experts on this thread appears to be that none of us can understand why your DBA wants the cache disabled as it doesnt make any sense.

Yes the cache useage looks large, but there is a huge amount of RAM still free, so the system is not memory constrained.

And although you CAN disable the write cache, your whole system is going to suffer the performance degradation this will bring.

So lets see the DBA's justification of why disabling the caching on this system is going to solve whatever the original issue is.
0
 
gheistCommented:
I got it (maybe)
Is your DBA complaining about oracle being swapped out? Do you use AMM too?
0
 
skullnobrainsCommented:
how could anything be swapped on a machine that has that much free ram ? if i'm not mistaken, even if a process could grow that big, it would be oem-killed before it would be swapped.
0
 
gheistCommented:
It happens. there is cache pressure and swappiness settings.
Common scenario is oracle making buffered writes to backup (extfs or nfs) and verifying, to make sure less used /dev/shm pages go to swap file.
0
 
skullnobrainsCommented:
@gheist : yes, these settings do exist, but how would a backup + backup check swap out anything in this case ? espetially shared pages which are unswappable most of the time (if not always on linux ?)

---

@c_lockland : i think we're moving away from your problem

there seems to be a mixup between buffers and ansynchronous writes in this thread :

when writing through a filesystem, you write through the kernel's VFS so there is no way you can bypass buffers, whatever you do

even with synchronous writes, buffers get written to. that does not mean these buffers are dirty (other than transiently during the write operation). there might be a way to force them to be deleted but the system will do so by itself whenever an app requires some ram before it will swap anything

-

when accessing disks directly, you bypass the vfs, and as far as i gather there is no separate buffer cache (well not that big anyway) on linux so the buffers are of no concern in this case

---

do you use asm for everything ?
does asm perform caching ?
do you use directIO for whatever is not managed by asm (logs ?) ?

what do you expect ?
0
 
gheistCommented:
AMM uses tmpfs and not systemV ipc and SGA can go to swap....
0
Question has a verified solution.

Are you are experiencing a similar issue? Get a personalized answer when you ask a related question.

Have a better answer? Share it in a comment.

Join & Write a Comment

Featured Post

Build your data science skills into a career

Are you ready to take your data science career to the next step, or break into data science? With Springboard’s Data Science Career Track, you’ll master data science topics, have personalized career guidance, weekly calls with a data science expert, and a job guarantee.

  • 7
  • 4
  • 3
  • +2
Tackle projects and never again get stuck behind a technical roadblock.
Join Now