File Server: Will this do the job?

Posted on 2005-04-11
Last Modified: 2010-04-03
We are about to ramp up some business and we need to add some storage servers to our company. We process data for our clients. Our processing application resides on approximately 100 processing computers. They grab the files to process from the current file server which is about 1 TB. It is a Dell dual processor w/ 4gb of RAM Windows 2003 Server Standard. The 1 TB is an attached DAS unit.---SCSI

The units we are looking at adding are 4.8 TB servers with Windows 2003 Server as well Dual Processor 4 GB RAM. We wanted to have multiple of these types of units as file servers where we could dedicate one unit to larger clients. We were looking at 4 of these units with 4.8 TB. I know this sounds like overkill but we expect to get a lot of data and we won't be able to get the jobs unless the new clients think we can handle it. Presentation is everything.

Anyway, these units are 3U's with 12 400 GB SATA drives. We were going to setup RAID 5.

How is this setup going to compare to getting one big unit with say 12 TB rather than several smaller ones like the above example. We liked having multiple units that we could also double up as an extra SQL Server and in case another unit is ever down. We don't want to have all of our eggs in one basket.

Also, is the SATA going to give us the performance we need.

Please let me know if we are doing something crazy.

Question by:moonzappa
    LVL 87

    Accepted Solution

    Sata should be no problem. Having many drives in a raid 5 system also improves the throughput, so the speed of the individual drive would show up that much (Sata 10000 Rpm max at the moment, SCSI 15000 Rpm).

    I would certainly suggest you do it the way you have outlined. A couple of smaller devices is better than one large system, exactly for the reason you mentioned, redundancy. You could probably set one such system up as a spare so if one of the life ones fail you'd be able to get the spare one running in very little time. £Having the large as a spare would probably be overkill and too expensive.
    LVL 9

    Expert Comment

    hi, there

    Approximately one big large raid 5 over 5 disk. Ex.

    The more disk you have in raid the bigger potential of 2 disk failiure at the same time.

    You might better list a plan for the future growth and not shoot yourself in the foot. Your backup will take longer since you have many places to get your data from.

    I'll properly make the plan as this.

    What is the data storage size in 5 years 5TB

    What is the maximum time in hours where i have my backup TIME this is very important. Lets make a time frame of 6 hours

    5TB = 5 hours What kinda equipment can handle this.
    1TB for future growth. Or at this moment for maintenance Driver updates, Service packs or else.

    for the 5TB of storage a SAN/NAS box is the only right solution get your data one place "Ease of backup + performance" Could be a EMC CX300 product max capacity 13.4TB.

    I assume you do data storage for many clients now here is the trick. On EMC other SAN product's each "Slide of a disk" is Acccesible with just using a HBA on each server and it is secure.

    LVL 4

    Expert Comment

    Another point to keep in mind, SATA is fine unless you have a large number of transactions, not necessarily transfers. SATA has horrbile latency compared to SCSI. So for file servers and most other storage SATA is fine.

    Author Comment

    The data that will go on the file servers will be for short term storage while we are processing it. Once the processing is completed we will take it off the "active" storage unit and put it on an external hard drive to redeliver to the client or put aside for unlikely future use. We would keep it for them for possibly up to a year unless they asked us to keep it longer or told us it was no longer necessary to keep and to return it to them...but there not be a reason to keep it on these active servers.

    We liked having the multiple units for another reason as well, which is for our clients we have thought about dedicating this server space for their use. As they are not that expensive and the clients like having dedicated space. We are not really hosting the data for them as much as processing it, storing it temporarily, delivering the processed data, and returning the data. After backing it up on an external for offline storage we can pretty much delete it from these units.

    If we go SAN/NAS then if it goes down we are kind of out of luck as all of our eggs will be in one basket.

    This way we are getting 20 TB for 30 grand but we will rarely need more than 1 tb for one job and when it was completed we would most likely remove it from the server on external drives.

    QUESTION: When we get the data from our client, we:

    1) load into onto the file server
    2) from the processing computers our software looks at the location of the file server to grab data to process locally-there are lots of processing machines
    3) It then writes to a SQL database on a separate machine information about the files processed

    Currently we are using the Dell mentioned above as the domain controller, SQL server, and file server. That is why these extra units are so important.

    How should we spread out the different components of our process. We are kind of on a limited budget and our IT person, doesn't work full time.

    Must we have a backup domain controller? How about SQL server, can we double up on one of the file servers that we are buying if it has Windows 2003 Server on it?

    And a really basic question but bare with me...all of the new servers are going to have Windows 2003 Server-It will not be a problem adding them to the LAN as we will set them up as file servers instead of domain controllers-Is there a way to have one of these as a backup domain controller?

    I know I jumbled a bunch of stuff in there so please feel free to break it down into separate answers or issues.

    LVL 4

    Expert Comment

    any other win2k3 servers should automatically be backup domain controllers.. however you can force them to be broadcasters..

    the SQL server is only an issue if you have ALOT of transactions.. how big is your sql database

    i dont think you'd need 30tb if the avg client is under 1tb
    NAS would be a very easy way to go and the transfer speed is fine, they can mirror as well for redundancy .. dunno if striped raid is what you actually 'need'


    Author Comment

    We have approximately 100 machines hitting the SQL database as fast as they can so I guess it could be 100+per second.

    We have 16 tables per project with 3 variable tables ranging between 30k and 5million rows depending on data being processed.

    Would one of the units above be good for the SQL server or how about the DELL with 1TB already here where it already is located.

    As for the TB:
    Yes your right about the TB size except we will have multiple jobs from the same client at the sam time. All could be up to 1 TB or less...or more so if we cut it too close then they will never give us the job in the first place. They like to come out and check out our facility. They don't like it too much when we say that we will buy more when we need it.

    Author Comment

    Thanks guys. Sorry for the delay in responding. We have been setting up everything and so far so good.

    Thanks again.
    LVL 87

    Expert Comment

    Thanks, too.

    Write Comment

    Please enter a first name

    Please enter a last name

    We will never share this with anyone.

    Featured Post

    Looking for New Ways to Advertise?

    Engage with tech pros in our community with native advertising, as a Vendor Expert, and more.

    Solid State Drive Performance Tips: Solid state storage technology is now a standard.  After testing and using several different brands and revisions of SSD's over the years I have put together a collection of tips,tools and suggestions that I ha…
    We all have limited time to study long and complicated information about RAID theories, but you may be interested as to how RAID 5 works. We made it simple for you by providing the shortest and easiest explanation ever.   First we need to remind …
    This tutorial will walk an individual through the process of installing the necessary services and then configuring a Windows Server 2012 system as an iSCSI target. To install the necessary roles, go to Server Manager, and select Add Roles and Featu…
    This Micro Tutorial will teach you how to reformat your flash drive. Sometimes your flash drive may have issues carrying files so this will completely restore it to manufacturing settings. Make sure to backup all files before reformatting. This w…

    737 members asked questions and received personalized solutions in the past 7 days.

    Join the community of 500,000 technology professionals and ask your questions.

    Join & Ask a Question

    Need Help in Real-Time?

    Connect with top rated Experts

    19 Experts available now in Live!

    Get 1:1 Help Now