Performance Issue In Caching Large Amounts of Data

I have a .NET application that will need to make quite a few calls the database based upon users interaction.

I'm quite concerned with the number of times the database will be hit with user interactions, so I have decided to cache the data stored in the database (around 3500 records) into .NET cache. So I am querying my data though a generic list using LINQ within the application.

What I have noticed is that when the cache expires, the website takes a long time to load whilst the cache is populated with data.

Is there a better and more efficient way to cache large amounts of data whilst minimising calls to the database?
Who is Participating?
I wear a lot of hats...

"The solutions and answers provided on Experts Exchange have been extremely helpful to me over the last few years. I wear a lot of hats - Developer, Database Administrator, Help Desk, etc., so I know a lot of things but not a lot about one thing. Experts Exchange gives me answers from people who do know a lot about one thing, in a easy to use platform." -Todd S.

Jacques Bourgeois (James Burger)PresidentCommented:
Do you really need to cache the 3500 records? If the users consult them one at a time, keeping them all in memory is overkill. Hitting the database each time a user needs a record might be better.

Or balance the thing as Amazon does. From what I learned about their system, it seems that they keep in cache only the items that are hot sellers and that they know will be requested repetitively during a given period of time. For any other item, they retrieve it from the database each time it is requested.

Experts Exchange Solution brought to you by

Your issues matter to us.

Facing a tech roadblock? Get the help and guidance you need from experienced professionals who care. Ask your question anytime, anywhere, with no hassle.

Start your 7-day free trial
It's more than this solution.Get answers and train to solve all your tech problems - anytime, anywhere.Try it for free Edge Out The Competitionfor your dream job with proven skills and certifications.Get started today Stand Outas the employee with proven skills.Start learning today for free Move Your Career Forwardwith certification training in the latest technologies.Start your trial today

From novice to tech pro — start learning today.