Solved

Can MS SQL server handle "Big Data"?

Posted on 2012-04-03
10
907 Views
Last Modified: 2016-03-23
Can MS SQL server 2008 handle "Big Data"?

I am talking about big data, 100  to 1000TB database, can MS SQL handle it? can MS SQL 2008 handle nop RDBMS model database?

Normally, how big (max) MS SQL 2008 can handle?

I've heard MS SQL 2012 can handle big data, what is the max for MS SQL 2012 to handle?
0
Comment
Question by:wasabi3689
[X]
Welcome to Experts Exchange

Add your voice to the tech community where 5M+ people just like you are talking about what matters.

  • Help others & share knowledge
  • Earn cash & points
  • Learn & ask questions
10 Comments
 
LVL 40

Assisted Solution

by:Adam Brown
Adam Brown earned 20 total points
ID: 37803930
http://msdn.microsoft.com/en-us/library/ms143432%28v=sql.110%29.aspx has all the limitations of SQL server 2012. It can handle databases of up to 524,272TB, so yeah. It'll handle 1000TB. If you go to the top of that page and click "Other Versions" you can view limitations of other version of SQL.
0
 
LVL 69

Assisted Solution

by:Scott Pletcher
Scott Pletcher earned 40 total points
ID: 37803962
Yes, with the proper configuration and database design.

You should go with the DataCenter Edition, and MS will lead the way for you.
0
 
LVL 75

Assisted Solution

by:Anthony Perkins
Anthony Perkins earned 20 total points
ID: 37804209
Can MS SQL server 2008 handle "Big Data"?
Of course.  The question should however be phrased:
Can you and/or your team handle "Big Data" with MS SQL server 2008?
0
Simplifying Server Workload Migrations

This use case outlines the migration challenges that organizations face and how the Acronis AnyData Engine supports physical-to-physical (P2P), physical-to-virtual (P2V), virtual to physical (V2P), and cross-virtual (V2V) migration scenarios to address these challenges.

 
LVL 6

Assisted Solution

by:SJCFL-Admin
SJCFL-Admin earned 100 total points
ID: 37806336
Capturing 100  to 1000TB of data is easy.  Getting value out of that data is much harder.  Companies have coined the terms 'Data Scientist' and 'Actionable Analytics'.  The first to delve into all their data and try to make sense of what they have and the second to decide what if anything they should do about it...

But to address the original question. Which does deserve a fair answer.

This is MY VIEW of what happened...  Sql Server first got its base in serving the thousands of small companies that did not have the pocket depth (or the needs for) an enterprise solution offered by the likes of Oracle, IBM or Terradata.  And they made millions (Billions?) servicing this niche.  But they were pretty much laughed at in the world of enterprise.  And I expect it angered the executives of Sql Server. After all, THEY were Microsoft AND they ahd the money base.  So they decided to play with the big guys and devoted the money to develop their product into something that COULD compete with the other big guys.  And it does, but some of its base is still (replication, broker services) still feels a little behind to me.. (Again , my personal opinion)

But overall, I think most companies are way too concerned on collecting information and way too unconcerned on understanding the real complexities involved in making automated decisions instantly.  People tend to want easy answers. Lets do this! And everyone heads in one direction.  If its hard, lets ignore that issue. We need it simpler.  Well, that pretty much got us the banking nightmare.  But I digress....

Sql Server has a broadshare of the market now and the money to support its product.  But its pricing itself above what many smaller companies are happy with.  I think the market is ready for another smaller player to provide these companies with what SqlServer used to and that may eat into their profit base.  Oracle, IBM and Terradata on the other hand are firmly entrenched in Enterprises and as US companies start to come out of the recession may take back the ground they lost.  but only time will tell if SqlServer made a mistake by entering the 'big boys' playground...

Again, all this is only my opinion.  I  have been pleasantly surprised by how much I have come to love working with Sql Server and feel that as long as they have people like Buck Woody on their team, they have a chance of staying grounded in reality and we are in good hands. But if they loose people like him, I'll wish I'd stayed in a shop with Oracle and UDB...

Clear as Mud?
0
 

Author Comment

by:wasabi3689
ID: 37806651
Ok, my next question is

which is better to handle "Big Data" MS SQL 2008 or 2012 to Hbase, hadoop...If you have an option, do you want to choose MS SQL 2008 to handle big data or Hbase.

What are the advantage and disadvantage by using MS SQL 2008 or 2012 to handle big data comparing some free system like hbase?
0
 
LVL 6

Assisted Solution

by:SJCFL-Admin
SJCFL-Admin earned 100 total points
ID: 37806767
I think you need to develop the list of your needs and weigh them.  THEN compare them against the alternatives.  If you need guaranteed stability, support, disaster recovery,etc. then a platform such as Hbase may not want to make the guarantees you need.  You would have to talk to their team to see.  But if your organization did not require these things, then it might be a good fit for you.

I have no experience with Hbase, but have read that Facebook uses it for its volume data.  But a lost facebook post would be different than a lost financial transaction or other transaction required to be kept for legal purposes.  I would not abse my corporations core business processes on such a platform, but for an application for a high tolerance for some data loss it would be acceptable if there were a performance advantage or money was a key factor.

There is not one right answer.  there is usually a complex set of requirements (price, performance under different conditions, simple recoveries, disaster recoveries, support, feature set, company stability, integration with other preferred software, trained developers, cost of said developers,...) that feed into the decisions.

Do make up your matrix and start putting answers against each.  You may find that your answer solves itself as you fill out the sheet and one vendor becomes a clear winner....
0
 
LVL 6

Assisted Solution

by:SJCFL-Admin
SJCFL-Admin earned 100 total points
ID: 37806805
Re Hbase.  it appears to have been created to help handle a specific problem type of large data.  If your large data matches theirs, then you too could benefit.  You would need to research the specific type.

Re.  Sql Server 2008 or 2012.  if you are just getting started and choose SqlServer, go with 2012 enterprise unless you are so under a deadline that you cannot afford the 6months it will take 2012 to stabalize.  Why?  SqlServer is rampinmg up for large data. I beleieve the changes made in 2012 will enhance its ability to do what you will want it to.  but if you want an expert answer, I suggest you try and get an answer from Buck Woody.
0
 
LVL 69

Assisted Solution

by:Scott Pletcher
Scott Pletcher earned 40 total points
ID: 37807041
If you really have 100TB of data (really??), then I wouldn't consider less than Datacenter for SQL Server, which no doubt will cost more than just Enterprise.  So adjust your cost models accordingly.

But do you want to try to handle issues with 100TB of data on a non-enterprise-level db with no direct support?  To me, that's just not realistic in a business situation.
0
 
LVL 6

Assisted Solution

by:SJCFL-Admin
SJCFL-Admin earned 100 total points
ID: 37807715
It would depend on the type of data I was collecting; the purpose for which it was used; the ease in which I could recollect the data if lost; my sla's and the tolerance my customers had for down time.

So many times we look at the collection of the data itself as the holy grail.  but it is a means to an end.

Suppose it is being used by mathemeticians and was being fed into complex models, the results of which tooks months for them to analyze and issue their reports to corporations.  After which, they would get new requests for business and possibly have to change the analysis to cope with the new requests. I expect that they would look at the data and start slicing and dicing, probably keeping intermediary tables or cubes as they went?  on a limited budjet, which would be more important?  The income of an assistant or a robust database engine?  I propose that this is a business decision.

Being a DBA, I personally would not want to use freeware for a database. but then at times the databases feel like my children...   It is hard for me to be objective and look at them as strickly repositories for transient data that has a limited lifespan and probably needing clearing out on a much more frequent basis than what we information hoarding humans normally do....  Nor do we have to treat every database as if it supported life on a distant planet... (not to mention each artifically determined deadline...)

So do a cost / benefit analysis and live with the decision.
0
 
LVL 6

Accepted Solution

by:
SJCFL-Admin earned 100 total points
ID: 37807742
Having said all that rational cost/benefit rationalization. i would not be able to support the application using the freeware database.  Because, I personally would not be able to sleep at night....   But that, I fear, is a flaw in my personality. I am the kind of person that likes to have two alternatives for recovery AND a guaranteed support person on the line if the impossible happens and both should fail....  So the last entry in my spreadsheet is always could I live with the decision or should I get my resume prepared...
0

Featured Post

Online Training Solution

Drastically shorten your training time with WalkMe's advanced online training solution that Guides your trainees to action. Forget about retraining and skyrocket knowledge retention rates.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Why is this different from all of the other step by step guides?  Because I make a living as a DBA and not as a writer and I lived through this experience. Defining the name: When I talk to people they say different names on this subject stuff l…
For both online and offline retail, the cross-channel business is the most recent pattern in the B2C trade space.
Using examples as well as descriptions, and references to Books Online, show the different Recovery Models available in SQL Server and explain, as well as show how full, differential and transaction log backups are performed
Viewers will learn how to use the SELECT statement in SQL to return specific rows and columns, with various degrees of sorting and limits in place.

726 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question