Go Premium for a chance to win a PS4. Enter to Win

x
?
Solved

Linux : disable disk caching

Posted on 2014-02-27
18
Medium Priority
?
5,042 Views
Last Modified: 2014-03-13
I need to diable disk caching on RHEL

# free
             total       used       free     shared    buffers     cached
Mem:     528493876   86554832  441939044          0     930332   78327364
-/+ buffers/cache:    7297136  521196740
Swap:     16777208          0   16777208

in the output above, the system is using 86554832  bytes of memory, but almost all of it is disk cache, 78327364 bytes

I want to disable it entirely, and reboot.

how can i do this ?
0
Comment
Question by:c_hockland
  • 7
  • 4
  • 3
  • +2
18 Comments
 
LVL 11

Expert Comment

by:Chris Sandrini
ID: 39892380
There is no way you can directly disable this feature. But there is a few kernel parameters that help you shape its behaviour

http://unixadminschool.com/blog/2013/10/rhel-6-controlling-cache-memory-page-cache-size/
0
 
LVL 17

Expert Comment

by:Gerald Connolly
ID: 39894254
Why do you want to disable it?  What problem are you trying to solve?
0
 

Author Comment

by:c_hockland
ID: 39894859
My DBA has asked me to disable it because he thinks the performance issues he is having on his Oracle DB might be related to the cache value that i posted.
0
Independent Software Vendors: We Want Your Opinion

We value your feedback.

Take our survey and automatically be enter to win anyone of the following:
Yeti Cooler, Amazon eGift Card, and Movie eGift Card!

 
LVL 17

Expert Comment

by:Gerald Connolly
ID: 39896676
What on the basis that it is taking up RAM that Oracle could use?
0
 
LVL 27

Expert Comment

by:skullnobrains
ID: 39896962
given the fact that 441G out of 528G are free, that would be weird to say the least

if his point is that the ram is effectively divided by 2 because the os and the rdbm both cache the same data, this is true if oracle's tablespaces are on the filesystem, but since most of the ram is unused, you probably would not care.

having 78G of cache used means oracle hardly uses more than that in terms of disk access on the filesystem. it looks like the performance problem is elsewhere

btw gosh what the hell is that machine ? how big and intensely used a database must be to need this ?

you may want to run iostat and post information regarding the disk layout if you want to know why oracle is slow.

----

back to your question, you can disable write cache using -o sync in fstab. i do not know of a simple way to deactivate read cache. anyway, it is pretty foolish to even try.

any decent rdbm including oracle can use directio both for tablespaces and logs which effectively bypasses the caches for what oracle uses. you can also give oracle raw disks.
0
 
LVL 62

Expert Comment

by:gheist
ID: 39897884
OracleDB can skip disk cache using direct-io, are you sure its a DBA?

http://www.oracle-base.com/articles/misc/direct-and-asynchronous-io.php
0
 

Author Comment

by:c_hockland
ID: 39900812
i forgot to mentioned .... these are ASM ( oracle disks )

i dont see these asm disks @ /etc/fstab

any ideas ?
0
 
LVL 62

Expert Comment

by:gheist
ID: 39901208
So they use direct io and kernel aio. There is no OS buffer in between.
0
 

Author Comment

by:c_hockland
ID: 39901283
well , i realized that linux doesnt know and doesnt care about asm disks.
I will still have to disable write cache on the linux level
0
 
LVL 62

Expert Comment

by:gheist
ID: 39901640
No, there is no linux write cache in the way of ASM

Only thing you can change is IO scheduler per disk or system default (tip - elevator=deadline is default on UEK)
That will change order aio requests are served, namely deadline makes sure they ground asap.
0
 
LVL 27

Expert Comment

by:skullnobrains
ID: 39901744
asm disks are by nature synchronous so "-o sync" from my previous comment does not apply, my bad

you'll find below a bunch of raw information but no real answer

using unbuffered IO is a matter of using directio or not which is something for the application to decide, not the OS. it is usually a parameter to the open system call and can be changed using ioctl on an open file descriptor. on linux there is also a family of functions (io_setup, io_getevent...) that will always be buffered by nature.

as far as i know, linux does not actually have a buffer cache separated from the filesystem's ( any one can confirm/infirm ? )

anyway, there is virtually no way that buffered IO will be slower than directIO. this is espetially true when you have that much free RAM. most OSes will use unused RAM for their buffer cache. this enhances disk writes speed a lot, and possibly writes with the use of prefetch.

there are high chances that the cached memory you see is totally unrelated to asm anyway. possibly your redo logs and the likes are on a regular filesystem ?

or oracle is configured to do asynchronous read/writes on asm (i have no idea wether this would appear as "cached" in top... if someone knows, thanks for sheding some light on the matter)

this links details different oracle behaviour in conjunction with asm or fiesystems. you'll probably find it usefull. https://access.redhat.com/site/documentation/en-US/Red_Hat_Enterprise_Linux/5/html/Tuning_and_Optimizing_Red_Hat_Enterprise_Linux_for_Oracle_9i_and_10g_Databases/chap-Oracle_9i_and_10g_Tuning_Guide-Configuring_IO_for_Raw_Partitions.html

hope some of this helps

ps : maybe if you give more details about the actual performance problem (and run an iostat when something seems wrong), we could give much better hints regarding what to work on. disabling the buffer cache does not seem like a good candidate. if your dba has a theory, i'll be happy to know about it
0
 
LVL 62

Expert Comment

by:gheist
ID: 39901783
One can apply disk scheduler to block device backing raw device (actually there are heaps of tunable parameters, but "deadline" is what good disk firmware does, or noop if you trust disk/raid firmware, probably better test which does better - you can apply them to live partition, so if oracle admin has clean room benchmark let them help you)

That would operate on "buffers" seen in "free" (you cannot avoid those buffers, it is where data of disk request goes before or from disk)
Then there comes various filesystem cached, out of scope when handling raw devices, essentially free RAM will be used as filesystem cache.

If you want you can try disabling disk's/raid's internal cache (at the (huge) cost of performance)
0
 
LVL 17

Expert Comment

by:Gerald Connolly
ID: 39902152
So the consensus of the experts on this thread appears to be that none of us can understand why your DBA wants the cache disabled as it doesnt make any sense.

Yes the cache useage looks large, but there is a huge amount of RAM still free, so the system is not memory constrained.

And although you CAN disable the write cache, your whole system is going to suffer the performance degradation this will bring.

So lets see the DBA's justification of why disabling the caching on this system is going to solve whatever the original issue is.
0
 
LVL 62

Expert Comment

by:gheist
ID: 39903140
I got it (maybe)
Is your DBA complaining about oracle being swapped out? Do you use AMM too?
0
 
LVL 27

Expert Comment

by:skullnobrains
ID: 39906230
how could anything be swapped on a machine that has that much free ram ? if i'm not mistaken, even if a process could grow that big, it would be oem-killed before it would be swapped.
0
 
LVL 62

Expert Comment

by:gheist
ID: 39906376
It happens. there is cache pressure and swappiness settings.
Common scenario is oracle making buffered writes to backup (extfs or nfs) and verifying, to make sure less used /dev/shm pages go to swap file.
0
 
LVL 27

Accepted Solution

by:
skullnobrains earned 2000 total points
ID: 39908771
@gheist : yes, these settings do exist, but how would a backup + backup check swap out anything in this case ? espetially shared pages which are unswappable most of the time (if not always on linux ?)

---

@c_lockland : i think we're moving away from your problem

there seems to be a mixup between buffers and ansynchronous writes in this thread :

when writing through a filesystem, you write through the kernel's VFS so there is no way you can bypass buffers, whatever you do

even with synchronous writes, buffers get written to. that does not mean these buffers are dirty (other than transiently during the write operation). there might be a way to force them to be deleted but the system will do so by itself whenever an app requires some ram before it will swap anything

-

when accessing disks directly, you bypass the vfs, and as far as i gather there is no separate buffer cache (well not that big anyway) on linux so the buffers are of no concern in this case

---

do you use asm for everything ?
does asm perform caching ?
do you use directIO for whatever is not managed by asm (logs ?) ?

what do you expect ?
0
 
LVL 62

Expert Comment

by:gheist
ID: 39926204
AMM uses tmpfs and not systemV ipc and SGA can go to swap....
0

Featured Post

Veeam Task Manager for Hyper-V

Task Manager for Hyper-V provides critical information that allows you to monitor Hyper-V performance by displaying real-time views of CPU and memory at the individual VM-level, so you can quickly identify which VMs are using host resources.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

The purpose of this article is to show how we can create Linux Mint virtual machine using Oracle Virtual Box. To install Linux Mint we have to download the ISO file from its website i.e. http://www.linuxmint.com. Once you open the link you will see …
This article will show you step-by-step instructions to build your own NTP CentOS server.  The network diagram shows the best practice to setup the NTP server farm for redundancy.  This article also serves as your NTP server documentation.
Learn how to get help with Linux/Unix bash shell commands. Use help to read help documents for built in bash shell commands.: Use man to interface with the online reference manuals for shell commands.: Use man to search man pages for unknown command…
How to Install VMware Tools in Red Hat Enterprise Linux 6.4 (RHEL 6.4) Step-by-Step Tutorial
Suggested Courses
Course of the Month8 days, 13 hours left to enroll

877 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question