Solved

I need help writing a C# console app.

Posted on 2011-03-10
5
343 Views
Last Modified: 2013-12-17
Hi Experts,
I need help writing a C# console application with SQL Server 2005 DB.  The table I will be querying has thousands of records so It needs to be multi-threaded.  The program needs two functions.  
One that calculates the Product of all ItemCost values by ItemCategory.
The second function needs to calculate the Median value of ItemCost by ItemCategory.

The name of my SQL table is  INVENTORY with the fields ItemCost (float), ItemCategory (varchar).

How can I do this?

Thanks in advance,
mrotor
0
Comment
Question by:mainrotor
5 Comments
 
LVL 75

Expert Comment

by:käµfm³d 👽
ID: 35102040
Why does this need to be multi-threaded? "Thousands" is not very large in terms of DB processing.
0
 

Author Comment

by:mainrotor
ID: 35102634
It has to be multi-threaded because it will eventually grow to millions of records.
0
 
LVL 33

Accepted Solution

by:
Todd Gerbert earned 250 total points
ID: 35109713
I will offer some broad advice, if you're interested...

Collect all the unique ItemCategory values using SELECT DISTINCT ItemCategory FROM INVENTORY, use these results to populate a pair of List<string>'s (one for product, one for median).  Create two Dictionary<string, float>'s - one each for the product results, and the median results.

Spawn two threads, one to calculate the product and one to find the median.

In each of the threads, grab the next 5 ItemCategory values and remove them from the List (make sure to use "lock" keyword, or some other thread synchronization mechanism).  For each of those 5 ItemCategory values spawn another thread that does the actual work - perhaps passing this thread a Delegate to be executed on completion to provide the result.

Of course, I have no idea how efficient multi-threading like this will be since I'm not sure if opening too many simultaneous connections to the SQL server will hurt performance, and ultimately all the threads are gonna be hitting the same SQL server anyway.

You should also tweak the number of threads that are running simultaneously to match the number of processor cores in your system (i.e. on a single processor system only one thread will be running at any given time anyway, so creating additional threads doesn't do much in the way of a performance increase; however, a system with two quad-core processors can run 8 threads simultaneously).
0
 
LVL 33

Expert Comment

by:Todd Gerbert
ID: 35109757
...and also, if it's a remote SQL server, all the connections will be using the same network connection.  So if it's a lot of data being transferred, that might present a bottleneck too.  
0
 
LVL 8

Assisted Solution

by:Volox
Volox earned 250 total points
ID: 35118517
Is your SQL server decently powered?  And is there any reason to do this outside of SQL server?

The reason I ask the above questions is because if you are talking about wanting to scale to millions of rows, the absolutely least efficient thing you can do is pull millions of rows OUT of SQL to do a calculation on them that you could do inside of SQL server.  Just the network consumption alone of pulling that much data out of SQL server is a waste of resources and time.

I'm not sure I'm clear on what you meant by a 'product' by category...?  Are you saying you have quantity and a price per quantity and have to calc a total price and then sum it?  Which would look like =>
SELECT SUM(Quantity * Price), Category FROM Items GROUP BY Category
If it's something different you are after, then please describe and I'm sure someone can help you come up with a solution.

Here is an article on how one can calculate medians within SQL server
http://sqlblog.com/blogs/adam_machanic/archive/2006/12/18/medians-row-numbers-and-performance.aspx 

And I'd also mention that if you have the disk space on SQL and you query for the total price more often than you change the per item price or quantity, then you might give consideration to using a computed column so that the total price is calculated for you within the table.  But be sure you read about the performance impacts on both sides of the equation before you implement.
0

Featured Post

3 Use Cases for Connected Systems

Our Dev teams are like yours. They’re continually cranking out code for new features/bugs fixes, testing, deploying, testing some more, responding to production monitoring events and more. It’s complex. So, we thought you’d like to see what’s working for us.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Calculating holidays and working days is a function that is often needed yet it is not one found within the Framework. This article presents one approach to building a working-day calculator for use in .NET.
The article shows the basic steps of integrating an HTML theme template into an ASP.NET MVC project
This Micro Tutorial demonstrates using Microsoft Excel pivot tables, how to reverse engineer competitors' marketing strategies through backlinks.

809 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question