• Status: Solved
  • Priority: Medium
  • Security: Public
  • Views: 353
  • Last Modified:

I need help writing a C# console app.

Hi Experts,
I need help writing a C# console application with SQL Server 2005 DB.  The table I will be querying has thousands of records so It needs to be multi-threaded.  The program needs two functions.  
One that calculates the Product of all ItemCost values by ItemCategory.
The second function needs to calculate the Median value of ItemCost by ItemCategory.

The name of my SQL table is  INVENTORY with the fields ItemCost (float), ItemCategory (varchar).

How can I do this?

Thanks in advance,
mrotor
0
mainrotor
Asked:
mainrotor
2 Solutions
 
käµfm³d 👽Commented:
Why does this need to be multi-threaded? "Thousands" is not very large in terms of DB processing.
0
 
mainrotorAuthor Commented:
It has to be multi-threaded because it will eventually grow to millions of records.
0
 
Todd GerbertIT ConsultantCommented:
I will offer some broad advice, if you're interested...

Collect all the unique ItemCategory values using SELECT DISTINCT ItemCategory FROM INVENTORY, use these results to populate a pair of List<string>'s (one for product, one for median).  Create two Dictionary<string, float>'s - one each for the product results, and the median results.

Spawn two threads, one to calculate the product and one to find the median.

In each of the threads, grab the next 5 ItemCategory values and remove them from the List (make sure to use "lock" keyword, or some other thread synchronization mechanism).  For each of those 5 ItemCategory values spawn another thread that does the actual work - perhaps passing this thread a Delegate to be executed on completion to provide the result.

Of course, I have no idea how efficient multi-threading like this will be since I'm not sure if opening too many simultaneous connections to the SQL server will hurt performance, and ultimately all the threads are gonna be hitting the same SQL server anyway.

You should also tweak the number of threads that are running simultaneously to match the number of processor cores in your system (i.e. on a single processor system only one thread will be running at any given time anyway, so creating additional threads doesn't do much in the way of a performance increase; however, a system with two quad-core processors can run 8 threads simultaneously).
0
 
Todd GerbertIT ConsultantCommented:
...and also, if it's a remote SQL server, all the connections will be using the same network connection.  So if it's a lot of data being transferred, that might present a bottleneck too.  
0
 
VoloxCommented:
Is your SQL server decently powered?  And is there any reason to do this outside of SQL server?

The reason I ask the above questions is because if you are talking about wanting to scale to millions of rows, the absolutely least efficient thing you can do is pull millions of rows OUT of SQL to do a calculation on them that you could do inside of SQL server.  Just the network consumption alone of pulling that much data out of SQL server is a waste of resources and time.

I'm not sure I'm clear on what you meant by a 'product' by category...?  Are you saying you have quantity and a price per quantity and have to calc a total price and then sum it?  Which would look like =>
SELECT SUM(Quantity * Price), Category FROM Items GROUP BY Category
If it's something different you are after, then please describe and I'm sure someone can help you come up with a solution.

Here is an article on how one can calculate medians within SQL server
http://sqlblog.com/blogs/adam_machanic/archive/2006/12/18/medians-row-numbers-and-performance.aspx 

And I'd also mention that if you have the disk space on SQL and you query for the total price more often than you change the per item price or quantity, then you might give consideration to using a computed column so that the total price is calculated for you within the table.  But be sure you read about the performance impacts on both sides of the equation before you implement.
0

Featured Post

Never miss a deadline with monday.com

The revolutionary project management tool is here!   Plan visually with a single glance and make sure your projects get done.

Tackle projects and never again get stuck behind a technical roadblock.
Join Now