Solved

designing a scalable Linux Storage System!

Posted on 2009-07-06
3
395 Views
Last Modified: 2013-11-14
Hello,

what is the best reasonable solution to build a scalable storage system using Linux with the following characters:

1- Open-Source based solution if possible (yet to be very trustful).
2- High availability (load balancing).
3- Good and secured Backup System.
4- Scalable and easy to add storage when needed.

the way it will work is like:
1- users will have access to the files through a special programmed client that we made (ssh based)
2- when the program upload and updates the files on the server it will keep revisions of the files.
3- each user will have a limited disk space.


my questions to help understand the case:
1- we need a centralized users server (user name and passwords)? how it will deal with multi servers?
2- Clustering?
3- ZFS (using Open Solaris)? for keeping file revisions?


we just need to have a full understanding first, and then decide.
it seems there is allot of ways to achieve the goal, and we look for the secure and realistic one for enterprise business grade,


cheers
0
Comment
Question by:Data-Base
[X]
Welcome to Experts Exchange

Add your voice to the tech community where 5M+ people just like you are talking about what matters.

  • Help others & share knowledge
  • Earn cash & points
  • Learn & ask questions
3 Comments
 
LVL 2

Accepted Solution

by:
MarkSlingsby earned 500 total points
ID: 24846322
well.. thats quite a list of requirements!

Ok. From experience with these things and what we have been testing internally (as well as using) you have some options, but not all will meet our requirements 100%.

Solution 1: (cheap)
2 x File Server Systems running Nexenta with HA. I must admit that we havent tested the HA. But Nexenta runs a full web based GUI, has HA support as an optional addon.

Solution 2: (expensive)
Netapp with dual controller and CIFS license
The NetApp fully supports versioning, High Availability, snapshots and much more. It really is THE solution, but costly. (How much is your data worth)

Solution 3: (really cheap)
ZFS with rsync - Ubuntu has a port from OpenSolaris for Ubuntu. But dont think your versioning is going to work. ZFS is one of the best/most efficient file systems we have come across, its very similar to NetApp's WAFL file system. It gives you easy flexibility to add new disk to the pool and provision easily.

Summary:
It sounds to me like you need a frontend machine with backend flexible storage. Use a Linux box with quotas and map that to a NetApp via CIFS. OR you can give your users direct CIFS access on the NetApp (CIFS is Samba/Windows file sharing)

ZFS & WAFL give you snapshotting facilities and are great for backups.
0
 
LVL 17

Expert Comment

by:Gerald Connolly
ID: 24959258
You havent told us what kind of performance you want!
But for ultimate scaleable performance, have you looked at things like Panasas (proprietary) or Lustre (Opensource (ish))
Have a look at this collection of file-systems  http://en.wikipedia.org/wiki/List_of_file_systems to really blow your mind.
0
 

Author Closing Comment

by:Data-Base
ID: 31600061
Good info that we are going to test soon :-)

Thanks
0

Featured Post

MIM Survival Guide for Service Desk Managers

Major incidents can send mastered service desk processes into disorder. Systems and tools produce the data needed to resolve these incidents, but your challenge is getting that information to the right people fast. Check out the Survival Guide and begin bringing order to chaos.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Suggested Solutions

This article is an update and follow-up of my previous article:   Storage 101: common concepts in the IT enterprise storage This time, I expand on more frequently used storage concepts.
A Bare Metal Image backup allows for the restore of an entire system to a similar or dissimilar hardware. They are highly useful for migrations and disaster recovery. Bare Metal Image backups support Full and Incremental backups. Differential backup…
This tutorial will walk an individual through locating and launching the BEUtility application to properly change the service account username and\or password in situation where it may be necessary or where the password has been inadvertently change…
This tutorial will walk an individual through the process of installing of Data Protection Manager on a server running Windows Server 2012 R2, including the prerequisites. Microsoft .Net 3.5 is required. To install this feature, go to Server Manager…

737 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question