LVM Large File Performance Test

I ran tests to compare disk performance of LVM versus normal ext3 partitions
on a hardware Raid 10 consisting of six 600G 10k physical disks.  According to my
google search, I was expecting the performance to be about the same or a
tad bit slower for an LVM configuration.  

The LVM consisted of 6 logical volumes on one volume group using the whole Raid 10.
The non-LVM consisted of 6 ext3 partitions on the Raid 10.

Can anyone explain why the LVM performed considerably better?
I was seeing a peek of 400k sectors/second on the LVM and
a peek of 234k sectors/second on the non-LVM.  

Test Description
================
a) Test consists of concurrent I/O of 24 large 2G files
b) Memory cache is cleared before each test
c) Disk cache is allowed to quiesce before each test

=====================
Large File Write Test
=====================
229 seconds - Dell R710 (LVM)
389 seconds - Dell R710 (no LVM)

====================
Large File Read Test
====================
296 seconds - Dell R710 (LVM)
521 seconds - Dell R710 (no LVM)

The write test consisted of writing 24 2G files using the dd command.
The read test consisted of copying those 24 2G files to /dev/null.
otwaycAsked:
Who is Participating?
I wear a lot of hats...

"The solutions and answers provided on Experts Exchange have been extremely helpful to me over the last few years. I wear a lot of hats - Developer, Database Administrator, Help Desk, etc., so I know a lot of things but not a lot about one thing. Experts Exchange gives me answers from people who do know a lot about one thing, in a easy to use platform." -Todd S.

DeadRatRacerCommented:
First thought is that under the LVM the logical partition sectors were correctly aligned against the disk's physical layout and the ext3 partition was not -  just a guess though.

Lots of articles, here is but one on the topic
http://blogs.sun.com/dlutz/entry/partition_alignment_guidelines_for_unified
0

Experts Exchange Solution brought to you by

Your issues matter to us.

Facing a tech roadblock? Get the help and guidance you need from experienced professionals who care. Ask your question anytime, anywhere, with no hassle.

Start your 7-day free trial
DeadRatRacerCommented:
More obscure post, but showing a real-world 6x increase in speed under ext3 after proper alignment

http://aulix.com/partition-alignment
0
otwaycAuthor Commented:
Thanks.  Yes, it apparently is very sensitive to the parameters.  The Oracle OEL installation screens have a default LVM chunk size of 32M which performed quite a bit worse than the OEL Linux PVCREATE/VGCREATE default value of 4M (for this particular test).  I noticed the LVM configuration seemed to maintain a more even balance of I/O across the physical disks than a basic EXT3 configuration.  This could very well be due to the parameters as you suggest.
0
It's more than this solution.Get answers and train to solve all your tech problems - anytime, anywhere.Try it for free Edge Out The Competitionfor your dream job with proven skills and certifications.Get started today Stand Outas the employee with proven skills.Start learning today for free Move Your Career Forwardwith certification training in the latest technologies.Start your trial today
Storage Hardware

From novice to tech pro — start learning today.