Can MS SQL server handle "Big Data"?

Can MS SQL server 2008 handle "Big Data"?

I am talking about big data, 100  to 1000TB database, can MS SQL handle it? can MS SQL 2008 handle nop RDBMS model database?

Normally, how big (max) MS SQL 2008 can handle?

I've heard MS SQL 2012 can handle big data, what is the max for MS SQL 2012 to handle?
Who is Participating?

[Product update] Infrastructure Analysis Tool is now available with Business Accounts.Learn More

I wear a lot of hats...

"The solutions and answers provided on Experts Exchange have been extremely helpful to me over the last few years. I wear a lot of hats - Developer, Database Administrator, Help Desk, etc., so I know a lot of things but not a lot about one thing. Experts Exchange gives me answers from people who do know a lot about one thing, in a easy to use platform." -Todd S.

Adam BrownSr Solutions ArchitectCommented: has all the limitations of SQL server 2012. It can handle databases of up to 524,272TB, so yeah. It'll handle 1000TB. If you go to the top of that page and click "Other Versions" you can view limitations of other version of SQL.
Scott PletcherSenior DBACommented:
Yes, with the proper configuration and database design.

You should go with the DataCenter Edition, and MS will lead the way for you.
Anthony PerkinsCommented:
Can MS SQL server 2008 handle "Big Data"?
Of course.  The question should however be phrased:
Can you and/or your team handle "Big Data" with MS SQL server 2008?
Acronis Data Cloud 7.8 Enhances Cyber Protection

A closer look at five essential enhancements that benefit end-users and help MSPs take their cloud data protection business further.

Capturing 100  to 1000TB of data is easy.  Getting value out of that data is much harder.  Companies have coined the terms 'Data Scientist' and 'Actionable Analytics'.  The first to delve into all their data and try to make sense of what they have and the second to decide what if anything they should do about it...

But to address the original question. Which does deserve a fair answer.

This is MY VIEW of what happened...  Sql Server first got its base in serving the thousands of small companies that did not have the pocket depth (or the needs for) an enterprise solution offered by the likes of Oracle, IBM or Terradata.  And they made millions (Billions?) servicing this niche.  But they were pretty much laughed at in the world of enterprise.  And I expect it angered the executives of Sql Server. After all, THEY were Microsoft AND they ahd the money base.  So they decided to play with the big guys and devoted the money to develop their product into something that COULD compete with the other big guys.  And it does, but some of its base is still (replication, broker services) still feels a little behind to me.. (Again , my personal opinion)

But overall, I think most companies are way too concerned on collecting information and way too unconcerned on understanding the real complexities involved in making automated decisions instantly.  People tend to want easy answers. Lets do this! And everyone heads in one direction.  If its hard, lets ignore that issue. We need it simpler.  Well, that pretty much got us the banking nightmare.  But I digress....

Sql Server has a broadshare of the market now and the money to support its product.  But its pricing itself above what many smaller companies are happy with.  I think the market is ready for another smaller player to provide these companies with what SqlServer used to and that may eat into their profit base.  Oracle, IBM and Terradata on the other hand are firmly entrenched in Enterprises and as US companies start to come out of the recession may take back the ground they lost.  but only time will tell if SqlServer made a mistake by entering the 'big boys' playground...

Again, all this is only my opinion.  I  have been pleasantly surprised by how much I have come to love working with Sql Server and feel that as long as they have people like Buck Woody on their team, they have a chance of staying grounded in reality and we are in good hands. But if they loose people like him, I'll wish I'd stayed in a shop with Oracle and UDB...

Clear as Mud?
wasabi3689Author Commented:
Ok, my next question is

which is better to handle "Big Data" MS SQL 2008 or 2012 to Hbase, hadoop...If you have an option, do you want to choose MS SQL 2008 to handle big data or Hbase.

What are the advantage and disadvantage by using MS SQL 2008 or 2012 to handle big data comparing some free system like hbase?
I think you need to develop the list of your needs and weigh them.  THEN compare them against the alternatives.  If you need guaranteed stability, support, disaster recovery,etc. then a platform such as Hbase may not want to make the guarantees you need.  You would have to talk to their team to see.  But if your organization did not require these things, then it might be a good fit for you.

I have no experience with Hbase, but have read that Facebook uses it for its volume data.  But a lost facebook post would be different than a lost financial transaction or other transaction required to be kept for legal purposes.  I would not abse my corporations core business processes on such a platform, but for an application for a high tolerance for some data loss it would be acceptable if there were a performance advantage or money was a key factor.

There is not one right answer.  there is usually a complex set of requirements (price, performance under different conditions, simple recoveries, disaster recoveries, support, feature set, company stability, integration with other preferred software, trained developers, cost of said developers,...) that feed into the decisions.

Do make up your matrix and start putting answers against each.  You may find that your answer solves itself as you fill out the sheet and one vendor becomes a clear winner....
Re Hbase.  it appears to have been created to help handle a specific problem type of large data.  If your large data matches theirs, then you too could benefit.  You would need to research the specific type.

Re.  Sql Server 2008 or 2012.  if you are just getting started and choose SqlServer, go with 2012 enterprise unless you are so under a deadline that you cannot afford the 6months it will take 2012 to stabalize.  Why?  SqlServer is rampinmg up for large data. I beleieve the changes made in 2012 will enhance its ability to do what you will want it to.  but if you want an expert answer, I suggest you try and get an answer from Buck Woody.
Scott PletcherSenior DBACommented:
If you really have 100TB of data (really??), then I wouldn't consider less than Datacenter for SQL Server, which no doubt will cost more than just Enterprise.  So adjust your cost models accordingly.

But do you want to try to handle issues with 100TB of data on a non-enterprise-level db with no direct support?  To me, that's just not realistic in a business situation.
It would depend on the type of data I was collecting; the purpose for which it was used; the ease in which I could recollect the data if lost; my sla's and the tolerance my customers had for down time.

So many times we look at the collection of the data itself as the holy grail.  but it is a means to an end.

Suppose it is being used by mathemeticians and was being fed into complex models, the results of which tooks months for them to analyze and issue their reports to corporations.  After which, they would get new requests for business and possibly have to change the analysis to cope with the new requests. I expect that they would look at the data and start slicing and dicing, probably keeping intermediary tables or cubes as they went?  on a limited budjet, which would be more important?  The income of an assistant or a robust database engine?  I propose that this is a business decision.

Being a DBA, I personally would not want to use freeware for a database. but then at times the databases feel like my children...   It is hard for me to be objective and look at them as strickly repositories for transient data that has a limited lifespan and probably needing clearing out on a much more frequent basis than what we information hoarding humans normally do....  Nor do we have to treat every database as if it supported life on a distant planet... (not to mention each artifically determined deadline...)

So do a cost / benefit analysis and live with the decision.
Having said all that rational cost/benefit rationalization. i would not be able to support the application using the freeware database.  Because, I personally would not be able to sleep at night....   But that, I fear, is a flaw in my personality. I am the kind of person that likes to have two alternatives for recovery AND a guaranteed support person on the line if the impossible happens and both should fail....  So the last entry in my spreadsheet is always could I live with the decision or should I get my resume prepared...

Experts Exchange Solution brought to you by

Your issues matter to us.

Facing a tech roadblock? Get the help and guidance you need from experienced professionals who care. Ask your question anytime, anywhere, with no hassle.

Start your 7-day free trial
It's more than this solution.Get answers and train to solve all your tech problems - anytime, anywhere.Try it for free Edge Out The Competitionfor your dream job with proven skills and certifications.Get started today Stand Outas the employee with proven skills.Start learning today for free Move Your Career Forwardwith certification training in the latest technologies.Start your trial today
Microsoft SQL Server

From novice to tech pro — start learning today.