Load entire database into tables in memory

Posted on 2005-04-08
Last Modified: 2010-04-16
Anyone know how to do this? I have a friend who is working on a huge database and wants to do this so that they're not incurring lots of over head duing database transactions.

Anyone got some information on how to do this?

Or why it may or may not be a good idea?
(besides the obvious data failure problems)
Question by:jayrod
    LVL 41

    Assisted Solution

    Uh, well... for one thing, if it's really a huge database, that would mean that the client PC would have to have enough RAM to hold the entire database... otherwise the client PC would start "paging"... and things would slow to a crawl.

    You can persist the database (or parts of it) to files on the client PC...  but somehow I don't think that's what you're after
    LVL 7

    Accepted Solution

    Ok, this is a pretty serious design question.

    If you load the whole thing into datasets you are going to need some serious hardware like graye said. (If this is a large database)

    What might be better is to build your middle layers out into a framework that accesses the data as neccessary. A sql call should not be taking all that much time. If you commonly need to access the same information over and over again and they do not change frequently then just plop that into XML outside of the database and bind to that. You can even create some services to keep that up to date.

    The solution to this would depend completely on the design and usage of the system.

    you can use the System.DAta.SqlClient namespace to give you what you need to load the tables. But it will be slow.
    LVL 3

    Author Comment

    well he does have a few large tables. And this is for a web application so the server is a beast. But I think i'll just tell him that he's better off doing it on a table by table basis. He does have one table that's got several hundred thousand rows that don't ever change. maybe this table would be great to put on an "island"

    Write Comment

    Please enter a first name

    Please enter a last name

    We will never share this with anyone.

    Featured Post

    Free Trending Threat Insights Every Day

    Enhance your security with threat intelligence from the web. Get trending threat insights on hackers, exploits, and suspicious IP addresses delivered to your inbox with our free Cyber Daily.

    Introduction This article series is supposed to shed some light on the use of IDisposable and objects that inherit from it. In essence, a more apt title for this article would be: using (IDisposable) {}. I’m just not sure how many people would ge…
    It was really hard time for me to get the understanding of Delegates in C#. I went through many websites and articles but I found them very clumsy. After going through those sites, I noted down the points in a easy way so here I am sharing that unde…
    Migrating to Microsoft Office 365 is becoming increasingly popular for organizations both large and small. If you have made the leap to Microsoft’s cloud platform, you know that you will need to create a corporate email signature for your Office 365…
    This video is in connection to the article "The case of a missing mobile phone (". It will help one to understand clearly the steps to track a lost android phone.

    779 members asked questions and received personalized solutions in the past 7 days.

    Join the community of 500,000 technology professionals and ask your questions.

    Join & Ask a Question

    Need Help in Real-Time?

    Connect with top rated Experts

    19 Experts available now in Live!

    Get 1:1 Help Now