Tiered storage in Linux?

Posted on 2011-10-11
Last Modified: 2012-05-12
I've been looking at this for a while, and not come up with anything satisfactory.  I don't know if it can be done, but I thought I'd solicit some opinions on how people think it might be achieved.  It may prove to be impossible...

I have a Linux server that runs 24/7, with a very light load.  It has a software RAID6 set with everything except /boot on it.  The drives never spin down because there are always log entries, database updates etc.  One of the small tasks assigned to this box, for example, is to poll a solar-power inverter every 5 minutes and record the result to a MySQL database.

How can I allow the drives to spin down?  I'm thinking perhaps using a USB flash drive as a buffer, like having a tiered drive, so that if the power goes out it's not lost, but every few hours it can be flushed to the drives.  Is that doable?

Failing that, is it sane to just write to the flash drive and rsync back to the spinning drives occasionally (for folders with logfiles and for database files)?

Other than that, how long can I reasonably stall output of the data within the machine so I don't fire up the drives very often?

My ideal would just be to pop in an SSD or other flash device and have it perform block-level buffering for the main drives, allowing the main drives to be updated in big chunks rather than piecemeal as they are just now.
Question by:Filthy_Defiler
    LVL 9

    Accepted Solution

    why is it such a problem that your drives do not spin down ?
    Good disks handle working 24/7 quite well

    What you want is a filesystem cache on disk, AFAIK only ZFS can do that on linux (

    Another solution is to use the Unionfs

    A bientot
    LVL 19

    Expert Comment

    You might have an issue with latency, if you manage to spin all the disks down they may not all come up at the same time, not necessarily an issue for a single disk but a raid set? Not something I'd be happy relying on.

    Is it worth spinning down on this system? You could move the log server to a system with ssd disks if you needed to conserve electricity etc but it really just might not ne worth it for this.
    LVL 34

    Expert Comment

    by:Duncan Roe
    I have been told more than once that the majority of wear of drive bearings occurs during spin-up and spin-down. So I would not worry that they never stop.

    Author Comment

    To clarify, this "server" is a box at home with a stack of SATA drives in it.  It's my home filestore, runs the databases for my Squeezeboxes, XBMC and to monitor the inverter for my solar panels.  Latency during spin-up shouldn't be an issue (so long as Linux is happy that drives will take a few seconds to reappear).

    The drives are just sitting there wasting power 99% of the time, and if I can spin them down then that's a few Wh saved.  They're also sitting there at 50+ degrees C, which I'd rather see cooled off a little.

    The drives should be happy for 300,000 spin-ups, so if it's only happening every few hours it's not going to be an issue within the lifetime of the drives.

    I'm not using ZFS (ext3 and XFS only), though it did look interesting.  Maybe next time!  I also stumbled across FlashCache and bcache, but they don't look terribly mature.

    Any other good ideas?  I intend to gradually swap the drives out for WD Caviar Greens to save a little power, but that's longer term.
    LVL 39

    Expert Comment

    Ok definitely consider SSD disk instead.

    Author Closing Comment

    Well, looks like I'm out of luck just now.  I'll have a much closer look at ZFS and see if it fits the bill.  Could be a candidate for next time that box gets rebuilt.


    Write Comment

    Please enter a first name

    Please enter a last name

    We will never share this with anyone.

    Featured Post

    Threat Intelligence Starter Resources

    Integrating threat intelligence can be challenging, and not all companies are ready. These resources can help you build awareness and prepare for defense.

    Ever notice how you can't use a new drive in Windows without having Windows assigning a Disk Signature?  Ever have a signature collision problem (especially with Virtual Machines?)  This article is intended to help you understand what's going on and…
    Create your own, high-performance VM backup appliance by installing NAKIVO Backup & Replication directly onto a Synology NAS!
    This video teaches viewers how to encrypt an external drive that requires a password to read and edit the drive. All tasks are done in Disk Utility. Plug in the external drive you wish to encrypt: Make sure all previous data on the drive has been …
    This tutorial will walk an individual through the process of installing the necessary services and then configuring a Windows Server 2012 system as an iSCSI target. To install the necessary roles, go to Server Manager, and select Add Roles and Featu…

    758 members asked questions and received personalized solutions in the past 7 days.

    Join the community of 500,000 technology professionals and ask your questions.

    Join & Ask a Question

    Need Help in Real-Time?

    Connect with top rated Experts

    10 Experts available now in Live!

    Get 1:1 Help Now