High performance sorted list in C# (200,000+ items)

I am writing a charting application that receives real time updates - financial quotes to be specific. I need to keep track of every quote that comes in and in the order it occurred.  I get these quotes out of order, and I have to put them in order.  I have used a SortedList, but found it is WAY too slow.  What is the absolute fastest way I could keep track of these items and keep them sorted?  The SortedList hangs up the application while it is managing incoming data - should I implement something that uses multithreading to perform the sorting?


Who is Participating?
I wear a lot of hats...

"The solutions and answers provided on Experts Exchange have been extremely helpful to me over the last few years. I wear a lot of hats - Developer, Database Administrator, Help Desk, etc., so I know a lot of things but not a lot about one thing. Experts Exchange gives me answers from people who do know a lot about one thing, in a easy to use platform." -Todd S.

Solution #1 - (Id really recommend do this one since its the easiest)
Dont dismiss a relation database such as MS SQL Server. MS SQL Server has the concept of RAM only based temporary tables.
Just create a table with a time stamp column, and index the timestamp column. Then, select * from table order by timestamp. This way there will be almost zero cost related to sorting the list of information.  You will also find that you can do a few thousand inserts per second into the relational database, even WITH the indexed column, hopefully this is fast enough. Also, consider installing MSDE or sql server on the machine that receives the data so that there is no network bandwidth latency. Here is an example of a memory (ram) temporary table :

DECLARE @OrderCounts TABLE(ProductID int, OrderCount int)

INSERT INTO @OrderCounts
FROM [Order Details]

SELECT COUNT(ProductID), OrderCount
FROM @OrderCounts
GROUP BY OrderCount

Solution #2
I would use an unsorted collection, or array for capturing the data as it comes in, and then at the exact moment you need it, sort the list into a new collection/array (using the quicksort algorithm) and then use the new sorted array /collection. Using this method you wont have any overhead in capturing the data.  And you will only have to sort the data the most minimal number of times.

Solution #3
*Gulp* Get ready for a huge programming / debugging session.
To get the highest performance out of your application and minimize the cost of inserting data, and minimize the cost of retrieving sorted information, I would recommend creating a new class that inherits from system.collections and make it use a self balancing binary tree algorithm for storing data internally.

Also note this statment from microsoft visual studio help:
"Enumerating through a collection is intrinsically not a thread-safe procedure. Even when a collection is synchronized, other threads could still modify the collection, which causes the enumerator to throw an exception. To guarantee thread safety during enumeration, you can either lock the collection during the entire enumeration or catch the exceptions resulting from changes made by other threads."
Based on that I wouldnt try to use a collection if you *must* make it multi-threaded to get more performance.

Also, note that relational database indexes are a dirivative of Self balancing binary trees. I believe they are Self balancing compressed hashing N-Trees algorithms.

Experts Exchange Solution brought to you by

Your issues matter to us.

Facing a tech roadblock? Get the help and guidance you need from experienced professionals who care. Ask your question anytime, anywhere, with no hassle.

Start your 7-day free trial
ericdotnetAuthor Commented:
djwillms, Thank you!  That was alot of very useful information, I very much appreciate it.


It's more than this solution.Get answers and train to solve all your tech problems - anytime, anywhere.Try it for free Edge Out The Competitionfor your dream job with proven skills and certifications.Get started today Stand Outas the employee with proven skills.Start learning today for free Move Your Career Forwardwith certification training in the latest technologies.Start your trial today

From novice to tech pro — start learning today.

Question has a verified solution.

Are you are experiencing a similar issue? Get a personalized answer when you ask a related question.

Have a better answer? Share it in a comment.