Solved

designing a scalable Linux Storage System!

Posted on 2009-07-06
3
388 Views
Last Modified: 2013-11-14
Hello,

what is the best reasonable solution to build a scalable storage system using Linux with the following characters:

1- Open-Source based solution if possible (yet to be very trustful).
2- High availability (load balancing).
3- Good and secured Backup System.
4- Scalable and easy to add storage when needed.

the way it will work is like:
1- users will have access to the files through a special programmed client that we made (ssh based)
2- when the program upload and updates the files on the server it will keep revisions of the files.
3- each user will have a limited disk space.


my questions to help understand the case:
1- we need a centralized users server (user name and passwords)? how it will deal with multi servers?
2- Clustering?
3- ZFS (using Open Solaris)? for keeping file revisions?


we just need to have a full understanding first, and then decide.
it seems there is allot of ways to achieve the goal, and we look for the secure and realistic one for enterprise business grade,


cheers
0
Comment
Question by:Data-Base
3 Comments
 
LVL 2

Accepted Solution

by:
MarkSlingsby earned 500 total points
ID: 24846322
well.. thats quite a list of requirements!

Ok. From experience with these things and what we have been testing internally (as well as using) you have some options, but not all will meet our requirements 100%.

Solution 1: (cheap)
2 x File Server Systems running Nexenta with HA. I must admit that we havent tested the HA. But Nexenta runs a full web based GUI, has HA support as an optional addon.

Solution 2: (expensive)
Netapp with dual controller and CIFS license
The NetApp fully supports versioning, High Availability, snapshots and much more. It really is THE solution, but costly. (How much is your data worth)

Solution 3: (really cheap)
ZFS with rsync - Ubuntu has a port from OpenSolaris for Ubuntu. But dont think your versioning is going to work. ZFS is one of the best/most efficient file systems we have come across, its very similar to NetApp's WAFL file system. It gives you easy flexibility to add new disk to the pool and provision easily.

Summary:
It sounds to me like you need a frontend machine with backend flexible storage. Use a Linux box with quotas and map that to a NetApp via CIFS. OR you can give your users direct CIFS access on the NetApp (CIFS is Samba/Windows file sharing)

ZFS & WAFL give you snapshotting facilities and are great for backups.
0
 
LVL 16

Expert Comment

by:Gerald Connolly
ID: 24959258
You havent told us what kind of performance you want!
But for ultimate scaleable performance, have you looked at things like Panasas (proprietary) or Lustre (Opensource (ish))
Have a look at this collection of file-systems  http://en.wikipedia.org/wiki/List_of_file_systems to really blow your mind.
0
 

Author Closing Comment

by:Data-Base
ID: 31600061
Good info that we are going to test soon :-)

Thanks
0

Featured Post

Microsoft Certification Exam 74-409

Veeam® is happy to provide the Microsoft community with a study guide prepared by MVP and MCT, Orin Thomas. This guide will take you through each of the exam objectives, helping you to prepare for and pass the examination.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

The article will include the best Data Recovery Tools along with their Features, Capabilities, and their Download Links. Hope you’ll enjoy it and will choose the one as required by you.
This article aims to explain the working of CircularLogArchiver. This tool was designed to solve the buildup of log file in cases where systems do not support circular logging or where circular logging is not enabled
This video teaches viewers how to encrypt an external drive that requires a password to read and edit the drive. All tasks are done in Disk Utility. Plug in the external drive you wish to encrypt: Make sure all previous data on the drive has been …
This tutorial will walk an individual through the steps necessary to install and configure the Windows Server Backup Utility. Directly connect an external storage device such as a USB drive, or CD\DVD burner: If the device is a USB drive, ensure i…

867 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

21 Experts available now in Live!

Get 1:1 Help Now