Tiered storage in Linux?

I've been looking at this for a while, and not come up with anything satisfactory.  I don't know if it can be done, but I thought I'd solicit some opinions on how people think it might be achieved.  It may prove to be impossible...

I have a Linux server that runs 24/7, with a very light load.  It has a software RAID6 set with everything except /boot on it.  The drives never spin down because there are always log entries, database updates etc.  One of the small tasks assigned to this box, for example, is to poll a solar-power inverter every 5 minutes and record the result to a MySQL database.

How can I allow the drives to spin down?  I'm thinking perhaps using a USB flash drive as a buffer, like having a tiered drive, so that if the power goes out it's not lost, but every few hours it can be flushed to the drives.  Is that doable?

Failing that, is it sane to just write to the flash drive and rsync back to the spinning drives occasionally (for folders with logfiles and for database files)?

Other than that, how long can I reasonably stall output of the data within the machine so I don't fire up the drives very often?

My ideal would just be to pop in an SSD or other flash device and have it perform block-level buffering for the main drives, allowing the main drives to be updated in big chunks rather than piecemeal as they are just now.
Filthy_DefilerAsked:
Who is Participating?
I wear a lot of hats...

"The solutions and answers provided on Experts Exchange have been extremely helpful to me over the last few years. I wear a lot of hats - Developer, Database Administrator, Help Desk, etc., so I know a lot of things but not a lot about one thing. Experts Exchange gives me answers from people who do know a lot about one thing, in a easy to use platform." -Todd S.

Frank ContrepoisVP Technical SalesCommented:
why is it such a problem that your drives do not spin down ?
Good disks handle working 24/7 quite well

What you want is a filesystem cache on disk, AFAIK only ZFS can do that on linux (http://zfsonlinux.org/)

Another solution is to use the Unionfs

A bientot
0

Experts Exchange Solution brought to you by

Your issues matter to us.

Facing a tech roadblock? Get the help and guidance you need from experienced professionals who care. Ask your question anytime, anywhere, with no hassle.

Start your 7-day free trial
joolsCommented:
You might have an issue with latency, if you manage to spin all the disks down they may not all come up at the same time, not necessarily an issue for a single disk but a raid set? Not something I'd be happy relying on.

Is it worth spinning down on this system? You could move the log server to a system with ssd disks if you needed to conserve electricity etc but it really just might not ne worth it for this.
0
Duncan RoeSoftware DeveloperCommented:
I have been told more than once that the majority of wear of drive bearings occurs during spin-up and spin-down. So I would not worry that they never stop.
0
Ultimate Tool Kit for Technology Solution Provider

Broken down into practical pointers and step-by-step instructions, the IT Service Excellence Tool Kit delivers expert advice for technology solution providers. Get your free copy now.

Filthy_DefilerAuthor Commented:
To clarify, this "server" is a box at home with a stack of SATA drives in it.  It's my home filestore, runs the databases for my Squeezeboxes, XBMC and to monitor the inverter for my solar panels.  Latency during spin-up shouldn't be an issue (so long as Linux is happy that drives will take a few seconds to reappear).

The drives are just sitting there wasting power 99% of the time, and if I can spin them down then that's a few Wh saved.  They're also sitting there at 50+ degrees C, which I'd rather see cooled off a little.

The drives should be happy for 300,000 spin-ups, so if it's only happening every few hours it's not going to be an issue within the lifetime of the drives.

I'm not using ZFS (ext3 and XFS only), though it did look interesting.  Maybe next time!  I also stumbled across FlashCache and bcache, but they don't look terribly mature.

Any other good ideas?  I intend to gradually swap the drives out for WD Caviar Greens to save a little power, but that's longer term.
0
nociSoftware EngineerCommented:
Ok definitely consider SSD disk instead.
0
Filthy_DefilerAuthor Commented:
Well, looks like I'm out of luck just now.  I'll have a much closer look at ZFS and see if it fits the bill.  Could be a candidate for next time that box gets rebuilt.

Thanks!
0
It's more than this solution.Get answers and train to solve all your tech problems - anytime, anywhere.Try it for free Edge Out The Competitionfor your dream job with proven skills and certifications.Get started today Stand Outas the employee with proven skills.Start learning today for free Move Your Career Forwardwith certification training in the latest technologies.Start your trial today
Linux

From novice to tech pro — start learning today.

Question has a verified solution.

Are you are experiencing a similar issue? Get a personalized answer when you ask a related question.

Have a better answer? Share it in a comment.