Solved

I need help writing a C# console app.

Posted on 2011-03-10
5
341 Views
Last Modified: 2013-12-17
Hi Experts,
I need help writing a C# console application with SQL Server 2005 DB.  The table I will be querying has thousands of records so It needs to be multi-threaded.  The program needs two functions.  
One that calculates the Product of all ItemCost values by ItemCategory.
The second function needs to calculate the Median value of ItemCost by ItemCategory.

The name of my SQL table is  INVENTORY with the fields ItemCost (float), ItemCategory (varchar).

How can I do this?

Thanks in advance,
mrotor
0
Comment
Question by:mainrotor
5 Comments
 
LVL 74

Expert Comment

by:käµfm³d 👽
Comment Utility
Why does this need to be multi-threaded? "Thousands" is not very large in terms of DB processing.
0
 

Author Comment

by:mainrotor
Comment Utility
It has to be multi-threaded because it will eventually grow to millions of records.
0
 
LVL 33

Accepted Solution

by:
Todd Gerbert earned 250 total points
Comment Utility
I will offer some broad advice, if you're interested...

Collect all the unique ItemCategory values using SELECT DISTINCT ItemCategory FROM INVENTORY, use these results to populate a pair of List<string>'s (one for product, one for median).  Create two Dictionary<string, float>'s - one each for the product results, and the median results.

Spawn two threads, one to calculate the product and one to find the median.

In each of the threads, grab the next 5 ItemCategory values and remove them from the List (make sure to use "lock" keyword, or some other thread synchronization mechanism).  For each of those 5 ItemCategory values spawn another thread that does the actual work - perhaps passing this thread a Delegate to be executed on completion to provide the result.

Of course, I have no idea how efficient multi-threading like this will be since I'm not sure if opening too many simultaneous connections to the SQL server will hurt performance, and ultimately all the threads are gonna be hitting the same SQL server anyway.

You should also tweak the number of threads that are running simultaneously to match the number of processor cores in your system (i.e. on a single processor system only one thread will be running at any given time anyway, so creating additional threads doesn't do much in the way of a performance increase; however, a system with two quad-core processors can run 8 threads simultaneously).
0
 
LVL 33

Expert Comment

by:Todd Gerbert
Comment Utility
...and also, if it's a remote SQL server, all the connections will be using the same network connection.  So if it's a lot of data being transferred, that might present a bottleneck too.  
0
 
LVL 8

Assisted Solution

by:Volox
Volox earned 250 total points
Comment Utility
Is your SQL server decently powered?  And is there any reason to do this outside of SQL server?

The reason I ask the above questions is because if you are talking about wanting to scale to millions of rows, the absolutely least efficient thing you can do is pull millions of rows OUT of SQL to do a calculation on them that you could do inside of SQL server.  Just the network consumption alone of pulling that much data out of SQL server is a waste of resources and time.

I'm not sure I'm clear on what you meant by a 'product' by category...?  Are you saying you have quantity and a price per quantity and have to calc a total price and then sum it?  Which would look like =>
SELECT SUM(Quantity * Price), Category FROM Items GROUP BY Category
If it's something different you are after, then please describe and I'm sure someone can help you come up with a solution.

Here is an article on how one can calculate medians within SQL server
http://sqlblog.com/blogs/adam_machanic/archive/2006/12/18/medians-row-numbers-and-performance.aspx

And I'd also mention that if you have the disk space on SQL and you query for the total price more often than you change the per item price or quantity, then you might give consideration to using a computed column so that the total price is calculated for you within the table.  But be sure you read about the performance impacts on both sides of the equation before you implement.
0

Featured Post

Find Ransomware Secrets With All-Source Analysis

Ransomware has become a major concern for organizations; its prevalence has grown due to past successes achieved by threat actors. While each ransomware variant is different, we’ve seen some common tactics and trends used among the authors of the malware.

Join & Write a Comment

Many of us here at EE write code. Many of us write exceptional code; just as many of us write exception-prone code. As we all should know, exceptions are a mechanism for handling errors which are typically out of our control. From database errors, t…
A long time ago (May 2011), I have written an article showing you how to create a DLL using Visual Studio 2005 to be hosted in SQL Server 2005. That was valid at that time and it is still valid if you are still using these versions. You can still re…
It is a freely distributed piece of software for such tasks as photo retouching, image composition and image authoring. It works on many operating systems, in many languages.
This tutorial demonstrates a quick way of adding group price to multiple Magento products.

743 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

16 Experts available now in Live!

Get 1:1 Help Now