Solved

designing a scalable Linux Storage System!

Posted on 2009-07-06
3
384 Views
Last Modified: 2013-11-14
Hello,

what is the best reasonable solution to build a scalable storage system using Linux with the following characters:

1- Open-Source based solution if possible (yet to be very trustful).
2- High availability (load balancing).
3- Good and secured Backup System.
4- Scalable and easy to add storage when needed.

the way it will work is like:
1- users will have access to the files through a special programmed client that we made (ssh based)
2- when the program upload and updates the files on the server it will keep revisions of the files.
3- each user will have a limited disk space.


my questions to help understand the case:
1- we need a centralized users server (user name and passwords)? how it will deal with multi servers?
2- Clustering?
3- ZFS (using Open Solaris)? for keeping file revisions?


we just need to have a full understanding first, and then decide.
it seems there is allot of ways to achieve the goal, and we look for the secure and realistic one for enterprise business grade,


cheers
0
Comment
Question by:Data-Base
3 Comments
 
LVL 2

Accepted Solution

by:
MarkSlingsby earned 500 total points
ID: 24846322
well.. thats quite a list of requirements!

Ok. From experience with these things and what we have been testing internally (as well as using) you have some options, but not all will meet our requirements 100%.

Solution 1: (cheap)
2 x File Server Systems running Nexenta with HA. I must admit that we havent tested the HA. But Nexenta runs a full web based GUI, has HA support as an optional addon.

Solution 2: (expensive)
Netapp with dual controller and CIFS license
The NetApp fully supports versioning, High Availability, snapshots and much more. It really is THE solution, but costly. (How much is your data worth)

Solution 3: (really cheap)
ZFS with rsync - Ubuntu has a port from OpenSolaris for Ubuntu. But dont think your versioning is going to work. ZFS is one of the best/most efficient file systems we have come across, its very similar to NetApp's WAFL file system. It gives you easy flexibility to add new disk to the pool and provision easily.

Summary:
It sounds to me like you need a frontend machine with backend flexible storage. Use a Linux box with quotas and map that to a NetApp via CIFS. OR you can give your users direct CIFS access on the NetApp (CIFS is Samba/Windows file sharing)

ZFS & WAFL give you snapshotting facilities and are great for backups.
0
 
LVL 16

Expert Comment

by:Gerald Connolly
ID: 24959258
You havent told us what kind of performance you want!
But for ultimate scaleable performance, have you looked at things like Panasas (proprietary) or Lustre (Opensource (ish))
Have a look at this collection of file-systems  http://en.wikipedia.org/wiki/List_of_file_systems to really blow your mind.
0
 

Author Closing Comment

by:Data-Base
ID: 31600061
Good info that we are going to test soon :-)

Thanks
0

Featured Post

Enabling OSINT in Activity Based Intelligence

Activity based intelligence (ABI) requires access to all available sources of data. Recorded Future allows analysts to observe structured data on the open, deep, and dark web.

Join & Write a Comment

A Bare Metal Image backup allows for the restore of an entire system to a similar or dissimilar hardware. They are highly useful for migrations and disaster recovery. Bare Metal Image backups support Full and Incremental backups. Differential backup…
Are you looking to recover an email message or a contact you just deleted mistakenly? Or you are searching for a contact that you erased from your MS Outlook ‘Contacts’ folder and now realized that it was important.
To efficiently enable the rotation of USB drives for backups, storage pools need to be created. This way no matter which USB drive is installed, the backups will successfully write without any administrative intervention. Multiple USB devices need t…
This tutorial will show how to configure a single USB drive with a separate folder for each day of the week. This will allow each of the backups to be kept separate preventing the previous day’s backup from being overwritten. The USB drive must be s…

760 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

20 Experts available now in Live!

Get 1:1 Help Now