Solved

performance - sql server select vs. select on in memory datatable

Posted on 2011-02-23
8
732 Views
Last Modified: 2012-08-13
Hi,
I’m looking for the best way to do the following thing in terms of performance time/memory usage.
I’ve a windows app. build with C#. the machine is win server 2003 with sql server 2005 DB.

I’ve a DB with a few tables, some have 1000's rows and some 10,000's rows.
in my application I need to do many select commands.
there're 2 approaches I’m aware of and I’m looking for the best one in terms of performance:
1. load the tables into datatables at the beginning of the app. run using SqlDataAdapters and then use datatable.select(xxx) to find the rows I need (no primary key selects usually select on text).
in this method I approach sql server only in the beginning of the app., loads the data to the memory and query on the datatables.
2. each time I’ve to query use a stored procedure on the DB. which means 10000's of queries on the DB.

from test I’ve made the 2nd method looks faster, but on task manager it looks like the sqlservr.exe is working very hard and takes a lot of CPU.

so what is the best way? is there another solution which is better than my solutions?
what about LINQ?

Thanks,
Assaf.
0
Comment
Question by:AssafST
[X]
Welcome to Experts Exchange

Add your voice to the tech community where 5M+ people just like you are talking about what matters.

  • Help others & share knowledge
  • Earn cash & points
  • Learn & ask questions
  • 4
  • 2
  • 2
8 Comments
 
LVL 53

Expert Comment

by:Dhaest
ID: 34967709
Some remarks:

1. Loading the data at startup - what if data has changed ?
2. How big is the return ? Do you return 10 rows, 100 rows, 10000 rows, ... The higher you go, the inneffecient your design is made. Perhaps you can use paging to return the data in smaller pieces ...

0
 

Author Comment

by:AssafST
ID: 34967742
1. the data is changed/updated at the end of the App. run - this App. is used to update the data in the DB, so there's no fear that the data isn't correct.
2. i need all the tables data in order to run the App., so there're 6 tables with ~10,000 rows each.
0
 
LVL 53

Accepted Solution

by:
Dhaest earned 250 total points
ID: 34967781
>> 2. i need all the tables data in order to run the App., so there're 6 tables with ~10,000 rows each.

Do you mean that you update all the tables, all the records ?
0
Independent Software Vendors: We Want Your Opinion

We value your feedback.

Take our survey and automatically be enter to win anyone of the following:
Yeti Cooler, Amazon eGift Card, and Movie eGift Card!

 

Author Comment

by:AssafST
ID: 34967820
no. i need all the data in order to update 3 tables (not all the records).
i don't do the update in memory but create a long string of SP commands and at the end of the run update the DB.
something like:
string updateDBstr = "exec SP1 p1, p2,p3; exec SP1 p5, p6,p7; exec SP2 p1, p2,p3;" etc...
objCommO.CommandText = updateDBstr;
objCommO.ExecuteNonQuery();
0
 
LVL 40

Expert Comment

by:lcohan
ID: 34970787
If you have enough memory on the SQL box you should not worry about that as SQL will put those tables in cache and keep them as long as they are needed (especially on not so large tables as yours). Until SQL 2005 you could have use DBCC PINTABLE sql command to force SQL keep those tables in memory but was discontinued.

you should check your page life expectancy/SQL Server Buffer Cache Hit Ratio and if less than 90% you may need better indexing/stats or more RAM on the box
0
 

Author Comment

by:AssafST
ID: 34971946
1. what do you mean by "put those tables in cache"? put them in datatables as i do?
2. "you should check your page life expectancy/SQL Server Buffer Cache Hit Ratio"
how do i check this?

10x
0
 
LVL 40

Assisted Solution

by:lcohan
lcohan earned 250 total points
ID: 34973237
SQL server will put the data in cache and as your tables are small is likely they will be entirely cached on servers with sufficient RAM.

You can run command below to check the values in SQL:


select * from sys.sysperfinfo
   where counter_name in ('Buffer cache hit ratio','Page life expectancy' )
order by counter_name



0
 

Author Closing Comment

by:AssafST
ID: 35139243
not a final solution
0

Featured Post

Stressed Out?

Watch some penguins on the livecam!

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

It was really hard time for me to get the understanding of Delegates in C#. I went through many websites and articles but I found them very clumsy. After going through those sites, I noted down the points in a easy way so here I am sharing that unde…
The article shows the basic steps of integrating an HTML theme template into an ASP.NET MVC project
In this video, viewers are given an introduction to using the Windows 10 Snipping Tool, how to quickly locate it when it's needed and also how make it always available with a single click of a mouse button, by pinning it to the Desktop Task Bar. Int…
Michael from AdRem Software outlines event notifications and Automatic Corrective Actions in network monitoring. Automatic Corrective Actions are scripts, which can automatically run upon discovery of a certain undesirable condition in your network.…

691 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question