Question

Custom Hashtable C#

Asked by: CallConnection

Hello Experts!

I would like to reasonably quickly lookup objects by NON-UNIQUE values in a collection.

Naturally I tried at Hashtables and Dictionaries, but because they rely on key's uniqueness, I couldn't really use them.

&By the way, if you know an alternative way, I don't mind what "collection" to use.  Any collection that would be quick for storing extra large collection (1 million objects), and looking them up. The type of each VALUE to lookup is long.

Hence the idea of custom Hashtable..  I thought of implementing a custom hashable solution which would in effect use chaining and not rehashing (http://en.wikipedia.org/wiki/Hash_table)

I wanted to ask if you know of a good (fast!?) implementation of this, or if not, if you could look at my code and suggest how it can be improved/completed please?

If you think it's a good idea, how would you implement the rest?

It implements just one nonunique key lookup (but there may be more).



Thank you,
Dmitri

// ************************************************************
//  This is just a section which is supposed to generate the "CustomHash" collection.
//
// The general idea is that you'd follow this path to get back an object:
// HashListHeader  ->  Bucket_keys -> BucketObjects
// By the way, I have been using the following to test the speed of this block:
/*
            public static System.Diagnostics.Stopwatch s = new System.Diagnostics.Stopwatch();
            s.Reset();
            s.Start();
            for (int i = 0; i < 10; i++)
            {
                HashListOfLists(1000000);
                
            }
            s.Stop();
            textBox1.Text += ("result = " + s.Elapsed + Environment.NewLine);
*/
// ************************************************************
 
        // 4294967298 - the value required to get hash int from hash long  (you can also get it by maxlong / maxint!)
        // 7700 - trial and error figure - seems to provide optimum balance for hash collisions, between the length of collision list and length of hash list
        static long long2int_denominator = (4294967298 * 7700);
 
        public static int GetNewArraySize(int OldSize, int nextIndex)
        {
            double incrementer = 0.0;
            if (OldSize < 100) incrementer = 2;
            else incrementer = 1.5;
 
            int newSize = (int)((double)OldSize + ((double)OldSize * incrementer));
 
            if (nextIndex >= newSize)
            {
                newSize = (int)((double)nextIndex + ((double)nextIndex * incrementer));
            }
            return newSize;
        }
 
        public static void HashListOfLists(int new_terations)
        {
            int[] HashListHeader = new int[278894];  // the size of this would never go higher than 278894
                                                                               // because 2147483647 (aka int!) / 7700 = 278893
                                                                               // see long2int_denominator defenition...
            List<List<long>> Bucket_keys = new List<List<long>>();
            List<List<string>> Bucket_objects = new List<List<string>>();   // to be replaced with the object...
            
            //populate the list with new lists.
            int while_loop1 = 0;
            while (while_loop1 < new_terations)
            {
                Bucket_keys.Add(new List<long>());
                Bucket_objects.Add(new List<string>());
                while_loop1++;
            }
 
            Random r = new Random();
            int current_list;           // create some random stuff to test this
            long value;
            int int_hash_value;
            for (int i = 0; i < new_terations; i++)
            {
                value = (long)((r.NextDouble() * 2.0 - 1.0) * long.MaxValue);  // create some random value to test
                int_hash_value = (int)(Math.Abs(value) / long2int_denominator); // get the hashed value ready to be inserted into the hash list header.
 
                current_list = HashListHeader[int_hash_value];
                if (current_list == null || current_list == 0)
                {
                    HashListHeader[int_hash_value] = i;
                    current_list = i;
                }
                Bucket_objects[current_list].Add(i.ToString());
                Bucket_keys[current_list].Add(value);
            }
        }

                                  
1:
2:
3:
4:
5:
6:
7:
8:
9:
10:
11:
12:
13:
14:
15:
16:
17:
18:
19:
20:
21:
22:
23:
24:
25:
26:
27:
28:
29:
30:
31:
32:
33:
34:
35:
36:
37:
38:
39:
40:
41:
42:
43:
44:
45:
46:
47:
48:
49:
50:
51:
52:
53:
54:
55:
56:
57:
58:
59:
60:
61:
62:
63:
64:
65:
66:
67:
68:
69:
70:
71:
72:
73:
74:
75:

Select allOpen in new window

This Question has been solved and asker verified All Experts Exchange premium technology solutions are available to subscription members.

Subscribe now for full access to Experts Exchange and get

Instant Access to this Solution

  • Plus...
  • 30 Day FREE access, no risk, no obligation
  • Collaborate with the world's top tech experts
  • Unlimited access to our exclusive solution database
  • Never be left without tech help again

Subscribe Now

Asked On
2009-04-29 at 04:15:40ID24365017
Tags

.Net

,

C#

,

Hashtable implementation

,

Custom Hashtable

Topics

C# Programming Language

,

Microsoft Visual C#.Net

,

Programming Languages

Participating Experts
3
Points
500
Comments
21

Trusted by hundreds of thousands everyday for fast, accurate and reliable tech support.

  • "The time we save is the biggest benefit of Experts Exchange to Warner Bros. What could take multiple guys 2 hours or more each to find is accessed in around 15 minutes on Experts Exchange." Mike Kapnisakis, Warner Bros.
  • "Our team likes having a resource that is more secure than just using Google and most experts using this service really know their stuff. It's nice to look here first versus using Google." Dayna Sellner, Lockheed Martin
  • "Anytime that I've been stumped with a problem, 9 out of 10 times Experts Exchange has either the accepted solution or an open discussion of the potential solution to the problem." Kenny Red, eBay Inc.

See what Experts Exchange can do for you.

Got a question?

We've got the answer.

Experts Exchange has been collecting answers to technology questions since 1996…3 million and counting! If you have a question, chances are we already have your answer.

Screenshot of Experts Exchange Knowledgebase

Need individual assistance?

Our experts are ready to help.

If you can't find the exact answer you're looking for, ask our exclusive community of 50,000 experts. You’ll get a personalized answer from a trusted professional.

Screenshot of Experts Exchange Knowledgebase

Want to learn from the best?

Read articles from industry experts.

Thousands of free tech tips, tricks, how-to’s and tutorials are available in our peer reviewed articles section. See for yourself how smart our experts are, no login required.

Screenshot of an Article

Working on a long term project?

Store your work and research.

Save solutions to your questions, answers you’ve discovered through searching plus helpful articles in your personal knowledgebase for easy future access.

Screenshot of Experts Exchange Knowledgebase

Access the answers to your technology questions today.

Subscribe Now

30-day free trial. Register in 60 seconds.

What Makes Experts Exchange Unique?

Members of the expert community talk about why the experience at Experts Exchange is different than what you will find anywhere else.

Trusted by the world's most respected brands.

image of each brand's logo

Faithfully serving IT professionals since 1996.

Experts Exchange Logo

Try it out and discover for yourself.

Subscribe Now

30-day free trial. Register in 60 seconds.

Related Solutions

  1. HashTable ...
    Hi experts, I am studying hashtable ... is there any written hashtable class in STL or anywhere in C++ ? or I need to make one myself ? thanks.
  2. Dictionary capacity in VB.Net 2.0
    Is there a known maximum capacity for Dictionaries? We are adding entries to a dictionary, and we're finding that at 1 864 135 records, the dictionary consistantly drags in performance and appears to no longer add new records. The dictionary is appended each loop of a cyc...
  3. HashTable
    Hi, I am writing VBA code that interfaces with a bespoke API. This API returns data from an LDAP search as a hashtable. As far as I know, there is not hashtable data type in VBA. Can anyone please give me some advice on how to work around this and successfully capture the...

Free Tech Articles

  1. WARNING: 5 Reasons why you should NEVER fix a computer for free.
    It is in our nature to love the puzzle. We are obsessed. The lot of us. We love puzzles. We love the challenge. We thrive on finding the answer. We hate disarray. It bothers us deep in our soul. W...
  2. SCCM OSD Basic troubleshooting
    SCCM 2007 OSD is a fantastic way to deploy operating systems, however, like most things SCCM issues can sometimes be difficult to resolve due to the sheer volume of logs to sift through and the dispe...
  3. Migrate Small Business Server 2003 to Exchange 2010 and Windows 2008 R2
    This guide is intended to provide step by step instructions on how to migrate from Small Business Server 2003 to Windows 2008 R2 with Exchange 2010. For this migration to work you will need the fo...
  4. Create a Win7 Gadget
    This article shows you how to create a simple "Gadget" -- a sort of mini-application supported by Windows 7 and Vista. Gadgets can be dropped anywhere on the desktop to provide instant information, ...
  5. Outlook continually prompting for username and password
    There have been a lot of questions recently regarding Outlook prompting for a username and password whilst using Exchange 2007. There are a few reasons why this would happen and I will try to cover t...
  6. Backup Exchange 2010 Information Store using Windows Backup
    There seems to be quite a lot of confusion around the ability to backup Exchange 2010 using the built in Windows Backup feature. This stems from the omission of this feature prior to Exchange 2007 s...

Cloud Class Webinars

  1. Avoiding Bugs in Microsoft Access
    Alison Balter takes and in-depth look at avoiding bugs in Access. In this webinar you will learn about using the immediate window to debug your applications, invoking the debugger, using breakpoints to troubleshoot, stepping through code, setting the next statement to execute, ...
  2. Top 10 Best New Features in Visio 2010
    Scott Helmers gives live demonstrations of the top 10 new features in Visio 2010. This webinar will teach you how to create compelling diagrams by adding shapes to the page with a single click, linking the shapes in a diagram to data in Excel (or SQL Server, or SharePoint), ...
  3. IT Consultant Business Secrets Revealed
    Michael Munger, Experts Exchange tech pro and IT consultant, pulls back the curtain on his very successful businesses and answers question on every IT consultant and business owner should know about. He shares secrets on what he did to solve the 5 most common problems in IT, ...
  4. Disaster Recovery and Business Continuity
    Quest CTO, Mike Billon, gives an overview of the steps involved in building a dunamic disaster recovery plan. Through case studies and an examination of software/hardware tooles for monitoring and testing, you'll gain a better understandin of where you are, where you want ...
  5. Organize Your Visio Diagrams with Containers and Lists
    Scott Helmers uses cross functional flowcharts, wireframe diagrams, data graphic legends and seating charts to teach you: how to ustilize all three new structured diagram components in Visio 2010, the best practices for organizeing shapes in previous version of Visio, how to organize ...
  6. How to Us Objects, Properties, Events and Methods in Microsoft Access
    Alison Dalter gives an in-depbth look at objects, properties, events and methods in Microsoft Access. In this webinar you will learn about using the object browser, referring to objects, working with properties and methods, working with object variables, understanding the ...

Join the Community

Give a Little. Get a Lot.

Join the community of experts here and help other tech pros by answering question in your area of expertise. You can earn FREE access to all Experts Exchange's premium features and resources.

Join the Community

Answers

 

by: ragi0017Posted on 2009-04-29 at 04:39:14ID: 24259231

you are talking about a million objects and in this scenario a database will be tbe best option because a million objects in dictionary/hashtable will consume a lot of memory and will decrease the performance

 

by: CallConnectionPosted on 2009-04-29 at 04:45:33ID: 24259282

True, and I know that, but database is not an option in my case. But forgetting million, how would you deal with a large collection?

 

by: ragi0017Posted on 2009-04-29 at 05:00:18ID: 24259392

i will prefer a dictionary over hashtable because at least the overhead of boxing and unboxing will not occur

The Dictionary class has the same functionality as the Hashtable class. A Dictionary of a specific type (other than Object) has better performance than a Hashtable for value types because the elements of Hashtable are of type Object and, therefore, boxing and unboxing typically occur if storing or retrieving a value type. - this from the following link
http://msdn.microsoft.com/en-us/library/4yh14awz(VS.80).aspx

 

by: CallConnectionPosted on 2009-04-29 at 05:07:54ID: 24259450

Thank you ragi.  As I mentioned, I tried Hashtables and Dictionaries, that's why I suggested a custom solution. How would you deal with duplicates in Key field?

 

by: alex_pavenPosted on 2009-04-29 at 06:27:53ID: 24260152

I remembered that there was a specialized collection that does pretty much what you described in Collections.Specialized, namely NameValueCollection, but it's no good since it uses only strings.

A quick search turned up this, don't know if it will help you or not, but may be another take on the idea (you could easily modify it to accept generic keys, I think).

 

by: gregoryyoungPosted on 2009-04-29 at 09:15:25ID: 24262024

a million objects is nothing to keep in memory ...

What is your duplciation like as a percentage? whats the highest count of dduplications you expect?

Greg

 

by: gregoryyoungPosted on 2009-04-29 at 09:22:25ID: 24262106

rags: just to be clear in this example a million (gasp objects) as say ints would take up a whopping 4 mb of memory! even once we add the overhead of the hash table's extra size (fill factor) we might be at 6? Oh Noez. Even with small objects say an address we would still be < 100 mb which again is nothing.

Not surprisingly the cost to make an out of process call to the database is about a million times larger than checking our value in memory.

there are certainly some archtitectural situations where you would want to keep the hash in memory.

 

by: gregoryyoungPosted on 2009-04-29 at 10:04:47ID: 24262473

oh well no response so here are the answers.

If your number of duplicates is low just make a Dictionary<whatever, List<whatevervalue>> almost as efficient as chainging and avoids the overhead of writing your own Dictionary (though writing one isn't that hard just meet the Idictionary<T,V> interface (I am sure we all wrote one at some point)..

If you expect a high number of duplicates then chaining or using the list<> as above would really suck instead use a Dictionary<T, Dictionary<T, V>> where the internal dictionary has a different hashing algorithm than the first one.

It really depends how many duplicates you expect to be seeing on average and whether or not the checking of the duplicates in a chain/list has a higher cost than running a hash function on your key.

 

by: ragi0017Posted on 2009-04-29 at 13:14:37ID: 24264280

the aurhor has never mentioned what kind of objects are being delt with - so if they are going to contain even a single stirng object the its a bad design to keep it in memory

as dictionary is also a key value pair i totally agree with your approach of keeping the key in the dictionayr and values in a generic list

 

by: gregoryyoungPosted on 2009-04-30 at 15:18:09ID: 24274942

"the aurhor has never mentioned what kind of objects are being delt with - so if they are going to contain even a single stirng object the its a bad design to keep it in memory"

Not at all.

You seem to be woefully ignorant of this thing called context. As I explained previously keeping them in memory allows for searches that are many order of magnitude faster than hitting an out of process call. If you need fast searching then its perfectly fine to be doing as a design decision.

Also keeping millions of objects in memory really is not that big of a deal. As an example I created some time ago a program that kept the entire state of the Toronto Stock Exchange in memory (orders and all). This was 3-5 million objects in memory at any given point (and they were not simple objects). The application ran in about a gig of memory but do to our other requirements (like needing to process 20k+ changes to the data per second) using something like a database would not have been a cost effective solution.

It is in fact quite common to put millions of objects in memory for performance reasons (with a horizontal partitioning plan) for when you run out of memory on a given machine. For reference see a "prevalence layer"

Be more careful in the future when you just call things "bad design" without having a context.

Cheers,

Greg

 

by: CallConnectionPosted on 2009-05-06 at 08:25:04ID: 24315921

Thank you all for your replies!!!

I'm really encouraged by your estimates for size, and also the fact that you do agree that it's a reasonable approach to take.
Makes me feel like I'm not alone in my quest, and that, hopefully, I'll be able to solve it.

Unfortunately I don't know exactly how many duplicates I'm going to get, but I would guess that it would constitute aroud 5% of total.
It's important to say that the keys might duplicate, but not whole obejcts ( - a bit similar to postcode (key) vs person (value) scenaio), so I cannot throw any "duplicates" away.

gregoryyoung:

I have looked into doing a Dictionary> structure; in fact as I was looking to implement *multiple* keys,
so I attempeted to use List>> ..

I have attached the code I used for Dictionary approach... The thing was that it seemed quite slow. I suspect it might be because on adding each object I tried to translate the "object" to "keys". What do you think?

In the mean time I have tried to persue my original goal of custom hashtable - I have attached the file with my second attempt to solve it...
It seems reasonably quick to insert, but it seems a bit slow to lookup... Is it not worth persuing?

Cheers again for your answers - looking forward to hearing from you!

Regards,
Dmitri

using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Reflection;
 
namespace DataTypeSpeedTests
{
    public class MKD<K, V>
    {
        //internal readonly List<Dictionary<K, V>> DictionaryList = new List<Dictionary<K, V>>();
        //internal readonly List<Dictionary<K, V>> DictionaryList = new List<Dictionary<K, V>>();
        //internal readonly List<Dictionary<K, Dictionary<K, V>>> ListOfDictionaries = new List<Dictionary<K, Dictionary<K, V>>>();
        //internal readonly Dictionary<int, string> TopLevelInt2String = new Dictionary<int, string>();
        //internal readonly List<K> 
        //internal int TopLevelCount = 0;
 
        //internal readonly Dictionary<string, Dictionary<K, List<V>>> MultiKeyDictionary = new Dictionary<string, Dictionary<K, List<V>>>();
 
        internal readonly List<Dictionary<K, List<V>>> MultiKeyDictionary = new List<Dictionary<K, List<V>>>();
        public readonly List<string> TopLevelKeyNameList = new List<string>();
        public readonly Dictionary<string, int> TopLevelKeyNoList = new Dictionary<string, int>();
        
 
        //internal readonly PropertyInfo[] ValueProperties = typeof(V).GetProperties();
        //internal readonly List<K> TopLevelKeyObjectList = new List<K>();
 
 
        readonly object lockObject = new object();
 
        /// <summary>
        /// Gets or sets the Value.
        /// </summary>
        /// <value>The Value.</value>
        public V Value
        {
            get;
            set;
        }
 
        /// <summary>
        /// Gets or sets the Key.
        /// </summary>
        /// <value>The Key.</value>
        public K Key
        {
            get;
            set;
        }
 
        /// <summary>
        /// Gets the value from Keys and List Index
        /// </summary>
        /// <remarks></remarks>
        /// <param name="TopLevelDicName">Top Level Dictionary's Name</param>
        /// <param name="key">Bottom Level Key</param>
        /// <param name="ListPosition">Position In The Value List.</param>
        public V this[string TopLevelDicName, K BottomLevelKey, int ListPosition]
        {
            get
            {
                V value;
                if (TryGetValue(TopLevelDicName, BottomLevelKey, ListPosition, out value))
                    return value;
                else
                    throw new KeyNotFoundException("Not found: " + TopLevelDicName.ToString() + " " + BottomLevelKey.ToString() + " " + ListPosition.ToString());
            }
        }
 
        /// <summary>
        /// Gets the value from Keys and List Index
        /// </summary>
        /// <remarks></remarks>
        /// <param name="TopLevelDicNo">Top Level Dictionary's Number</param>
        /// <param name="key">Bottom Level Key</param>
        /// <param name="ListPosition">Position In The Value List.</param>
        public V this[int TopLevelDicNo, K BottomLevelKey, int ListPosition]
        {
            get
            {
                V value;
                if (TryGetValue(TopLevelDicNo, BottomLevelKey, ListPosition, out value))
                    return value;
                else
                    throw new KeyNotFoundException("Not found: " + TopLevelDicNo.ToString() + " " + BottomLevelKey.ToString() + " " + ListPosition.ToString());
            }
        }
 
        /// <summary>
        /// Gets the list of values from Keys and List Index
        /// </summary>
        /// <remarks></remarks>
        /// <param name="TopLevelDicName">Top Level Dictionary's Name</param>
        /// <param name="key">Bottom Level Key</param>
        public List<V> this[string TopLevelDicName, K BottomLevelKey]
        {
            get
            {
                List<V> ListValue;
                if (TryGetValueList(TopLevelDicName, BottomLevelKey, out ListValue))
                    return ListValue;
                else
                    throw new KeyNotFoundException("Not found: " + TopLevelDicName.ToString() + " " + BottomLevelKey.ToString());
            }
        }
 
        /// <summary>
        /// Gets the list of values from Keys and List Index
        /// </summary>
        /// <remarks></remarks>
        /// <param name="TopLevelDicNo">Top Level Dictionary's Number</param>
        /// <param name="key">Bottom Level Key</param>
        public List<V> this[int TopLevelDicNo, K BottomLevelKey]
        {
            get
            {
                List<V> ListValue;
                if (TryGetValueList(TopLevelDicNo, BottomLevelKey, out ListValue))
                    return ListValue;
                else
                    throw new KeyNotFoundException("Not found: " + TopLevelDicNo.ToString() + " " + BottomLevelKey.ToString());
            }
        }
 
        /// <summary>
        /// MKD Class constructor
        /// </summary>
        /// <remarks></remarks>
        /// <param name="Keys">The list of keys to be used; </param>
        /// <param name="newClassValue">An instance of the item - does not need to contain a value; just important to establish the type</param>
        /// <typeparam name="V">
        /// The type of the object that should be stored
        /// </typeparam>
        /// <typeparam name="K">
        /// All the keys must be of the same type (K).
        /// </typeparam>
        public MKD(List<K> Keys, V newClassValue) //string Property2Dictionary
        {
            int i = 0;
            foreach (K key in Keys)
            {
                if (newClassValue.GetType().GetProperty(key.ToString()) != null)
                {
                    try
                    {
                        MultiKeyDictionary.Add(new Dictionary<K, List<V>>());
                        //TopLevelKeyObjectList.Add(key);
                        TopLevelKeyNameList.Add(key.ToString());
                        TopLevelKeyNoList.Add(key.ToString(),i);
                        i++;
                        //TopLevelInt2String.Add(MultiKeyDictionary.Count, key.ToString());
                    }
                    catch (Exception e)
                    {
                        throw e;
                    }
                }
                else
                    throw new System.Exception(key.ToString() + " is not in the " + newClassValue.ToString() + " class; correct the list which is used in the constructor.");
            }
        }
 
        /// <summary>
        /// Get Top Level Dictionary Name By It's "Number"
        /// </summary>
        /// <param name="TopLevelDicNo">Top Level Dictionary's Number</param>
        public string GetTopLevelDictionaryName(int TopLevelDicNo)
        {
            return TopLevelKeyNameList[TopLevelDicNo];
        }
 
        /// <summary>
        /// Verify That Top Level Dictionary Exists By It's Name
        /// </summary>
        /// <param name="TopLevelDicName">Top Level Dictionary's Name</param>
        public bool ContainsTopLevelKey(string TopLevelDicName)
        {
            return TopLevelKeyNameList.Contains(TopLevelDicName);
                //MultiKeyDictionary.ContainsKey(TopLevelDicName);
        }
 
        /// <summary>
        /// Verify That Top Level Dictionary Exists By It's Number
        /// </summary>
        /// <param name="TopLevelDicNo">Top Level Dictionary's Number</param>
        public bool ContainsTopLevelKey(uint TopLevelDicNo)
        {
            return (TopLevelDicNo <= TopLevelKeyNameList.Count);
                //MultiKeyDictionary.ContainsKey(TopLevelKeyNameList[TopLevelDicNo]);
        }
 
        /// <summary>
        /// Verify That Bottom Level Dictionary Exists
        /// </summary>
        /// <param name="TopLevelDicName">Top Level Dictionary's Name</param>
        /// <param name="key">Bottom Level Key</param>
        public bool ContainsBottomLevelKey(string TopLevelDicName, K key)
        {
            lock (lockObject)
            {
                if (ContainsTopLevelKey(TopLevelDicName)) //MultiKeyDictionary.ContainsKey(TopLevelDicName)
                {
                    if (MultiKeyDictionary[TopLevelKeyNoList[TopLevelDicName]].ContainsKey(key))
                    {
                        //return true; -- return at the end to unlock the object
                    }
                    else
                        return false;
                }
                else
                    return false;
            }
            return true;
        }
 
        /// <summary>
        /// Verify That Bottom Level Dictionary Exists
        /// </summary>
        /// <param name="TopLevelDicNo">Top Level Dictionary's Number</param>
        /// <param name="key">Bottom Level Key</param>
        public bool ContainsBottomLevelKey(int TopLevelDicNo, K key)
        {
            return ContainsBottomLevelKey(TopLevelKeyNameList[TopLevelDicNo], key);
        }
 
        /// <summary>
        /// Verify That Dictionary Contains Value
        /// </summary>
        /// <remarks>
        /// It's a bit more efficient than other override alternatives.
        /// </remarks>
        /// <param name="TopLevelDicName">Top Level Dictionary's Name</param>
        /// <param name="key">Bottom Level Key</param>
        /// <param name="value">The Value (object).</param>
        public bool ContainsValue(string TopLevelDicName, K key, V value)
        {
            lock (lockObject)
            {
                if (ContainsTopLevelKey(TopLevelDicName))
                {
                    if (MultiKeyDictionary[TopLevelKeyNoList[TopLevelDicName]].ContainsKey(key))
                    {
                        if (MultiKeyDictionary[TopLevelKeyNoList[TopLevelDicName]][key].Contains(value))
                        {
                            //return true; -- return at the end to unlock the object
                        }
                        else
                            return false;
                    }
                    else
                    {
                        return false;
                    }
                }
                else
                {
                    return false;
                }
            }
            return true;
        }
 
        /// <summary>
        /// Verify That Dictionary Contains Value
        /// </summary>
        /// <param name="TopLevelDicNo">Top Level Dictionary's Number</param>
        /// <param name="key">Bottom Level Key</param>
        /// <param name="value">The Value (object).</param>
        public bool ContainsValue(int TopLevelDicNo, K key, V value)
        {
            return ContainsValue(TopLevelKeyNameList[TopLevelDicNo], key, value);
        }
 
        /// <summary>
        /// Verify That Dictionary Contains Value
        /// </summary>
        /// <param name="TopLevelDicName">Top Level Dictionary's Name</param>
        /// <param name="value">The Value (object).</param>
        public bool ContainsValue(string TopLevelDicName, V value)
        {
            lock (lockObject)
            {
                if (ContainsTopLevelKey(TopLevelDicName))
                {
                    foreach (KeyValuePair<K, List<V>> kvp in MultiKeyDictionary[TopLevelKeyNoList[TopLevelDicName]])
                    {
                        if (kvp.Value.Contains(value))
                            //return true; -- break instead and unlock the object...
                            break;
                    }
                    return false;
                }
                else
                    return false;
            }
            return true; // unreachable code is a fib, because of the break in the foreach loop.
        }
 
        /// <summary>
        /// Verify That Dictionary Contains Value
        /// </summary>
        /// <param name="TopLevelDicNo">Top Level Dictionary's Number</param>
        /// <param name="value">The Value (object).</param>
        public bool ContainsValue(int TopLevelDicNo, V value)
        {
            return ContainsValue(TopLevelKeyNameList[TopLevelDicNo], value);
        }
 
        
 
        /// <summary>
        /// Verify That Dictionary Contains Value; Get First Value - And The List Of Values.
        /// </summary>
        /// <remarks>
        /// Hopefully the overhead of doing this would not be too high...
        /// </remarks>
        /// <returns>The Count Of Values In The List</returns>
        /// <param name="TopLevelDicName">Top Level Dictionary's Name</param>
        /// <param name="key">Bottom Level Key</param>
        /// <param name="value">(output) - The Value (object).</param>
        /// <param name="ListVal">(output) - The List Of Values (object list).</param>
        public uint TryGetValueAndValueList(string TopLevelDicName, K key, out V value, out List<V> ListVal)
        {
            lock (lockObject)
            {
                if (ContainsTopLevelKey(TopLevelDicName))//.TryGetValue(TopLevelDicName, out Ds)
                {
                    if (MultiKeyDictionary[TopLevelKeyNoList[TopLevelDicName]].ContainsKey(key))
                    {
                        if (MultiKeyDictionary[TopLevelKeyNoList[TopLevelDicName]].TryGetValue(key, out ListVal))
                        {
                            value = ListVal[0];
                            return (uint)ListVal.Count;
                        }
                        else
                        {
                            ListVal = default(List<V>);
                            value = default(V);
                            return 0;
                        }
                    }
                    else
                    {
                        ListVal = default(List<V>);
                        value = default(V);
                        return 0;
                    }
                }
                else
                {
                    ListVal = default(List<V>);
                    value = default(V);
                    return 0;
                }
            }
        }
 
        /// <summary>
        /// Verify That Dictionary Contains Value; Get First Value - And The List Of Values.
        /// </summary>
        /// <remarks>
        /// Hopefully the overhead of doing this would not be too high...
        /// </remarks>
        /// <returns>The Count Of Values In The List</returns>
        /// <param name="TopLevelDicNo">Top Level Dictionary's Number</param>
        /// <param name="key">Bottom Level Key</param>
        /// <param name="value">(output) - The Value (object).</param>
        /// <param name="ListVal">(output) - The List Of Values (object list).</param>
        public uint TryGetValueAndValueList(int TopLevelDicNo, K key, out V value, out List<V> ListVal)
        {
            lock (lockObject)
            {
                if (TopLevelDicNo >= 0 && TopLevelDicNo <= TopLevelKeyNameList.Count)
                {
                    if (MultiKeyDictionary[TopLevelDicNo].ContainsKey(key))
                    {
                        if (MultiKeyDictionary[TopLevelDicNo].TryGetValue(key, out ListVal))
                        {
                            value = ListVal[0];
                            return (uint)ListVal.Count;
                        }
                        else
                        {
                            ListVal = default(List<V>);
                            value = default(V);
                            return 0;
                        }
                    }
                    else
                    {
                        ListVal = default(List<V>);
                        value = default(V);
                        return 0;
                    }
                }
                else
                {
                    ListVal = default(List<V>);
                    value = default(V);
                    return 0;
                }
            }
        }
 
        /// <summary>
        /// Verify That Dictionary Contains Value and Get Value.
        /// </summary>
        /// <param name="TopLevelDicName">Top Level Dictionary's Name</param>
        /// <param name="key">Bottom Level Key</param>
        /// <param name="ListPosition">Position In The Value List.</param>
        /// <param name="value">(output) - The Value (object).</param>
        public bool TryGetValue(string TopLevelDicName, K key, int ListPosition, out V value)
        {
            lock (lockObject)
            {
                if (ListPosition >= 0 && ContainsTopLevelKey(TopLevelDicName))//.TryGetValue(TopLevelDicName, out Ds)
                {
                    if (MultiKeyDictionary[TopLevelKeyNoList[TopLevelDicName]].ContainsKey(key))
                    {
                        if (ListPosition <= MultiKeyDictionary[TopLevelKeyNoList[TopLevelDicName]][key].Count) // .TryGetValue(key, out ListVal))
                        {
                            value = MultiKeyDictionary[TopLevelKeyNoList[TopLevelDicName]][key][ListPosition];
                            return true; // return true at the end to unlock object???
                        }
                        else
                        {
                            value = default(V);
                            return false;
                        }
                    }
                    else
                    {
                        value = default(V);
                        return false;
                    }
                }
                else
                {
                    value = default(V);
                    return false;
                }
            }
        }
 
        /// <summary>
        /// Verify That Dictionary Contains Value and Get The Value.
        /// </summary>
        /// <param name="TopLevelDicNo">Top Level Dictionary's Number</param>
        /// <param name="key">Bottom Level Key</param>
        /// <param name="ListPosition">Position In The Value List.</param>
        /// <param name="value">(output) - The Value (object).</param>
        public bool TryGetValue(int TopLevelDicNo, K key, int ListPosition, out V value)
        {
            lock (lockObject)
            {
                if (ListPosition >= 0 && ListPosition <= TopLevelKeyNameList.Count)//.TryGetValue(TopLevelDicName, out Ds)
                {
                    if (MultiKeyDictionary[TopLevelDicNo].ContainsKey(key))
                    {
                        if (ListPosition <= MultiKeyDictionary[TopLevelDicNo][key].Count) // .TryGetValue(key, out ListVal))
                        {
                            value = MultiKeyDictionary[TopLevelDicNo][key][ListPosition];
                            return true; // return true at the end to unlock object???
                        }
                        else
                        {
                            value = default(V);
                            return false;
                        }
                    }
                    else
                    {
                        value = default(V);
                        return false;
                    }
                }
                else
                {
                    value = default(V);
                    return false;
                }
            }
            //return TryGetValue(TopLevelKeyNameList[TopLevelDicNo], key, ListPosition, out  value);
        }
 
        /// <summary>
        /// Verify That Dictionary Contains Value and Get First Value.
        /// </summary>
        /// <remarks>
        /// Major Weakness - there may be more elements in the list...
        /// </remarks>
        /// <param name="TopLevelDicName">Top Level Dictionary's Name</param>
        /// <param name="key">Bottom Level Key</param>
        /// <param name="ListPosition">Position In The Value List.</param>
        /// <param name="value">(output) - The Value (object).</param>
        public bool TryGetFirstValue(string TopLevelDicName, K key, out V value)
        {
            return TryGetValue(TopLevelDicName, key, 0, out value);
        }
 
        /// <summary>
        /// Verify That Dictionary Contains Value and Get Value List.
        /// </summary>
        /// <param name="TopLevelDicName">Top Level Dictionary's Name</param>
        /// <param name="key">Bottom Level Key</param>
        /// <param name="ListVal">(output) - The Value List (List Of Value Objects).</param>
        public bool TryGetValueList(string TopLevelDicName, K key, out List<V> ListVal)
        {
            lock (lockObject)
            {
                if (ContainsTopLevelKey(TopLevelDicName))//.TryGetValue(TopLevelDicName, out Ds)
                {
                    if (MultiKeyDictionary[TopLevelKeyNoList[TopLevelDicName]].ContainsKey(key))
                    {
                        if (MultiKeyDictionary[TopLevelKeyNoList[TopLevelDicName]].TryGetValue(key, out ListVal))
                        {
                            return true; // return true at the end to unlock object???
                        }
                        else
                        {
                            ListVal = default(List<V>);
                            return false;
                        }
                    }
                    else
                    {
                        ListVal = default(List<V>);
                        return false;
                    }
                }
                else
                {
                    ListVal = default(List<V>);
                    return false;
                }
            }
        }
 
        /// <summary>
        /// Verify That Dictionary Contains Value and Get Value List.
        /// </summary>
        /// <param name="TopLevelDicNo">Top Level Dictionary's Number</param>
        /// <param name="key">Bottom Level Key</param>
        /// <param name="ListVal">(output) - The Value List (List Of Value Objects).</param>
        public bool TryGetValueList(int TopLevelDicNo, K key, out List<V> ListVal)
        {
            lock (lockObject)
            {
                if (TopLevelDicNo >= 0 && TopLevelDicNo <= TopLevelKeyNameList.Count)
                {
                    if (MultiKeyDictionary[TopLevelDicNo].ContainsKey(key))
                    {
                        if (MultiKeyDictionary[TopLevelDicNo].TryGetValue(key, out ListVal))
                        {
                            return true; // return true at the end to unlock object???
                        }
                        else
                        {
                            ListVal = default(List<V>);
                            return false;
                        }
                    }
                    else
                    {
                        ListVal = default(List<V>);
                        return false;
                    }
                }
                else
                {
                    ListVal = default(List<V>);
                    return false;
                }
            }
        }
 
        /// <summary>
        /// Add Value To All Dictionaries.
        /// </summary>
        /// <param name="value">The Value (Of Type V) To Be Added.</param>
        public void AddItem(V value)
        {
            lock (lockObject)
            {
                for (int i = 0; i < TopLevelKeyNameList.Count; i++)
                {
                    List<V> ListValues;
                    K KeyFromValue = default(K);
                    object o = value.GetType().GetProperty(TopLevelKeyNameList[i]).GetValue(value, null);
                    if (o != null)
                    {
                        KeyFromValue = (K)o;
                        if (MultiKeyDictionary[i].TryGetValue(KeyFromValue, out ListValues)) //ContainsKey(KeyFromValue))
                        {
                            ListValues.Add(value);
                        }
                        else
                        {
                            ListValues = new List<V>();
                            ListValues.Add(value);
                            MultiKeyDictionary[i].Add(KeyFromValue, ListValues);
                        }
                    }
                }
            }
        }
 
 
 
        /// <summary>
        /// Add Value To All Dictionaries.
        /// </summary>
        /// <param name="value">The Value (Of Type V) To Be Added.</param>
        public void AddItemNew(V value)
        {
            for (int i = 0; i < TopLevelKeyNameList.Count; i++)
            {
                try
                {
                    K KeyFromValue = (K)value.GetType().GetProperty(TopLevelKeyNameList[i]).GetValue(value, null);
                    try
                    {
                        MultiKeyDictionary[i].Add(KeyFromValue, new List<V>(new V[]{value}));
                    }
                    catch
                    {
                        // the key already exists...
                        try
                        {
                            MultiKeyDictionary[i][KeyFromValue].Add(value);
                        }
                        catch
                        {
                            throw;
                        }
                    }
                }
                catch
                {
                    // no object in value, ignore...
                }
            }
        }
 
 
 
 
        /// <summary>
        /// Remove Value From All Dictionaries.
        /// </summary>
        /// <param name="value">The Value (Of Type V) To Be Removed.</param>
        public void RemoveItem(V value)
        {
            lock (lockObject)
            {
                for (int i = 0; i < TopLevelKeyNameList.Count; i++)
                {
                    List<V> ListValues = default(List<V>);
                    K KeyFromValue = default(K);
                    object o = value.GetType().GetProperty(TopLevelKeyNameList[i]).GetValue(value, null);
                    if (o != null)
                    {
                        KeyFromValue = (K)o;
                        if (MultiKeyDictionary[i].TryGetValue(KeyFromValue, out ListValues)) //ContainsKey(KeyFromValue))
                        {
                            int cnt_before_removing = ListValues.Count;
                            ListValues.Remove(value); // todo - need to check that removing the value in THIS list does not remove it in OTHER lists.
                            // if this was the only value in this dictionary, remove key from dictionary too.
                            if (cnt_before_removing == 1)
                                MultiKeyDictionary[i].Remove(KeyFromValue);
                        }
                        else
                        {
                            //do nothing...
                        }
                    }
                }
                value = default(V);
            }
        }
 
 
        /// <summary>
        /// Removes All Keys And Values From MKD Dictionary.
        /// </summary>
        public void Clear()
        {
            lock (lockObject) 
                MultiKeyDictionary.Clear();
        }
 
 
 
    }
}
                                              
1:
2:
3:
4:
5:
6:
7:
8:
9:
10:
11:
12:
13:
14:
15:
16:
17:
18:
19:
20:
21:
22:
23:
24:
25:
26:
27:
28:
29:
30:
31:
32:
33:
34:
35:
36:
37:
38:
39:
40:
41:
42:
43:
44:
45:
46:
47:
48:
49:
50:
51:
52:
53:
54:
55:
56:
57:
58:
59:
60:
61:
62:
63:
64:
65:
66:
67:
68:
69:
70:
71:
72:
73:
74:
75:
76:
77:
78:
79:
80:
81:
82:
83:
84:
85:
86:
87:
88:
89:
90:
91:
92:
93:
94:
95:
96:
97:
98:
99:
100:
101:
102:
103:
104:
105:
106:
107:
108:
109:
110:
111:
112:
113:
114:
115:
116:
117:
118:
119:
120:
121:
122:
123:
124:
125:
126:
127:
128:
129:
130:
131:
132:
133:
134:
135:
136:
137:
138:
139:
140:
141:
142:
143:
144:
145:
146:
147:
148:
149:
150:
151:
152:
153:
154:
155:
156:
157:
158:
159:
160:
161:
162:
163:
164:
165:
166:
167:
168:
169:
170:
171:
172:
173:
174:
175:
176:
177:
178:
179:
180:
181:
182:
183:
184:
185:
186:
187:
188:
189:
190:
191:
192:
193:
194:
195:
196:
197:
198:
199:
200:
201:
202:
203:
204:
205:
206:
207:
208:
209:
210:
211:
212:
213:
214:
215:
216:
217:
218:
219:
220:
221:
222:
223:
224:
225:
226:
227:
228:
229:
230:
231:
232:
233:
234:
235:
236:
237:
238:
239:
240:
241:
242:
243:
244:
245:
246:
247:
248:
249:
250:
251:
252:
253:
254:
255:
256:
257:
258:
259:
260:
261:
262:
263:
264:
265:
266:
267:
268:
269:
270:
271:
272:
273:
274:
275:
276:
277:
278:
279:
280:
281:
282:
283:
284:
285:
286:
287:
288:
289:
290:
291:
292:
293:
294:
295:
296:
297:
298:
299:
300:
301:
302:
303:
304:
305:
306:
307:
308:
309:
310:
311:
312:
313:
314:
315:
316:
317:
318:
319:
320:
321:
322:
323:
324:
325:
326:
327:
328:
329:
330:
331:
332:
333:
334:
335:
336:
337:
338:
339:
340:
341:
342:
343:
344:
345:
346:
347:
348:
349:
350:
351:
352:
353:
354:
355:
356:
357:
358:
359:
360:
361:
362:
363:
364:
365:
366:
367:
368:
369:
370:
371:
372:
373:
374:
375:
376:
377:
378:
379:
380:
381:
382:
383:
384:
385:
386:
387:
388:
389:
390:
391:
392:
393:
394:
395:
396:
397:
398:
399:
400:
401:
402:
403:
404:
405:
406:
407:
408:
409:
410:
411:
412:
413:
414:
415:
416:
417:
418:
419:
420:
421:
422:
423:
424:
425:
426:
427:
428:
429:
430:
431:
432:
433:
434:
435:
436:
437:
438:
439:
440:
441:
442:
443:
444:
445:
446:
447:
448:
449:
450:
451:
452:
453:
454:
455:
456:
457:
458:
459:
460:
461:
462:
463:
464:
465:
466:
467:
468:
469:
470:
471:
472:
473:
474:
475:
476:
477:
478:
479:
480:
481:
482:
483:
484:
485:
486:
487:
488:
489:
490:
491:
492:
493:
494:
495:
496:
497:
498:
499:
500:
501:
502:
503:
504:
505:
506:
507:
508:
509:
510:
511:
512:
513:
514:
515:
516:
517:
518:
519:
520:
521:
522:
523:
524:
525:
526:
527:
528:
529:
530:
531:
532:
533:
534:
535:
536:
537:
538:
539:
540:
541:
542:
543:
544:
545:
546:
547:
548:
549:
550:
551:
552:
553:
554:
555:
556:
557:
558:
559:
560:
561:
562:
563:
564:
565:
566:
567:
568:
569:
570:
571:
572:
573:
574:
575:
576:
577:
578:
579:
580:
581:
582:
583:
584:
585:
586:
587:
588:
589:
590:
591:
592:
593:
594:
595:
596:
597:
598:
599:
600:
601:
602:
603:
604:
605:
606:
607:
608:
609:
610:
611:
612:
613:
614:
615:
616:
617:
618:
619:
620:
621:
622:
623:
624:
625:
626:
627:
628:
629:
630:
631:
632:
633:
634:
635:
636:
637:
638:
639:
640:
641:
642:
643:
644:
645:
646:
647:
648:
649:
650:
651:
652:
653:
654:
655:
656:
657:
658:
659:
660:
661:
662:
663:
664:
665:
666:
667:
668:
669:
670:
671:
672:
673:
674:
675:
676:
677:
678:
679:
680:
681:
682:
683:
684:
685:
686:
687:
688:
689:
690:
691:
692:
693:
694:
695:

Select allOpen in new window

 

by: CallConnectionPosted on 2009-05-06 at 08:27:37ID: 24315959

..I have noticed that the formatting went a bit wrong once I submitted the post..

The attached attempt covers List > Dictionary > List

Thanks,
  Dmitri

 

by: gregoryyoungPosted on 2009-05-06 at 08:46:16ID: 24316210

I am not suprised wtf are you using reflection in it?

Why is this a responsibility of your MKD at all? Why is your MKD handling locking etc? This is the responsibility of other code.

Start simple ... profile the add/get of a "MKD" using a Dictionary<T, List<V>> I will even write it for you. (included). Filling it out should not take long.


Cheers,

Greg

public MKD<T, V>
   private readonly Dictionary<T, List<V>> dictionary;
 
   public MKD() { dictionary = new Dictionary<T, List<V>>(); }
 
   public Add(T key, V item) {
       List<V> entry;
       if(!dictionary.TryGetValue(key, out entry)) { 
           entry = new List<V>();
           dictionary.Add(entry);
       }
       entry.Add(entry);
   }
 
   public void Remove(T key) {
       dictionary.Remove(key);
   }
 
   public void Clear() {
       dictionary.Clear();
   }
 
   public void Remove(T key, K value) {
       List<V> entry;
       if(dictionary.TryGetValue(key, out entry)) { 
            entry.Remove(value);
       }
   }
 
 
   public IEnumerable<V> Get(T key) {
       List<V> entry;
       if(!dictionary.TryGetValue(key, out entry)) throw new Exception("Bad key");
       return entry; //might want to use .ToArray() here in multi threaded code
   }
}
                                              
1:
2:
3:
4:
5:
6:
7:
8:
9:
10:
11:
12:
13:
14:
15:
16:
17:
18:
19:
20:
21:
22:
23:
24:
25:
26:
27:
28:
29:
30:
31:
32:
33:
34:
35:
36:

Select allOpen in new window

 

by: CallConnectionPosted on 2009-05-06 at 11:33:09ID: 24318079

Hey Greg,

Well, to be honest I am not hugely experienced on this front (and most certainly not comparing to some guys here).

But this is why I'm here - and experience is really what gives *you* the edge (and might explains some questions you had)..

I used reflection to map object fields to keys, which I agree is not really necessary.

I took the object locking idea from the article which did a similar job  
( http://www.codeproject.com/KB/recipes/multikey-dictionary.aspx ), but I can also see it's not needed.

Thank you for time, and I'll give it a go soon...

Cheers,
  Dmitri

 

by: CallConnectionPosted on 2009-05-06 at 13:55:13ID: 24319562

Hi Greg,

Well, I believe you've got it.. I updated the code very very slightly - for the benefit of others who might read this..

I will try it a bit more before I close up the post, but I really appreciate what you did.

Thanks again,
  Dmitri

    public class MKD2<T, V>
    {
        private readonly Dictionary<T, List<V>> dictionary;
 
        public MKD2() { dictionary = new Dictionary<T, List<V>>(); }
 
        public void Add(T key, V item) 
        {
           List<V> entry;
           if(!dictionary.TryGetValue(key, out entry)) 
           { 
               entry = new List<V>();
               dictionary.Add(key, entry);
           }
           entry.Add(item);
        }
 
        public void Remove(T key) 
        {
           dictionary.Remove(key);
        }
 
        public void Clear() 
        {
           dictionary.Clear();
        }
 
        public void Remove(T key, V value) 
        {
           List<V> entry;
           if(dictionary.TryGetValue(key, out entry)) { 
                entry.Remove(value);
           }
        }
 
        public IEnumerable<V> Get(T key) 
        {
           List<V> entry;
           if(!dictionary.TryGetValue(key, out entry)) throw new Exception("Bad key");
           return entry; //might want to use .ToArray() here in multi threaded code
        }
    }

                                              
1:
2:
3:
4:
5:
6:
7:
8:
9:
10:
11:
12:
13:
14:
15:
16:
17:
18:
19:
20:
21:
22:
23:
24:
25:
26:
27:
28:
29:
30:
31:
32:
33:
34:
35:
36:
37:
38:
39:
40:
41:
42:

Select allOpen in new window

 

by: gregoryyoungPosted on 2009-05-06 at 14:01:47ID: 24319636

no worries, sorry for any typos I just typed it in here.

 

by: CallConnectionPosted on 2009-05-07 at 08:32:31ID: 24327002

**** Brilliant ****

I have successfully tested the structure and it's quick for simple and even reasonably complex objects, both for storing, enumerating (hurray!).

Testing conditions: I have roughly 1 gb of free memory.

Using simple objects I managed to store up to 5.9 mil (with one key dymention).
Using fairly large objects I managed to store up to 1.5 mil (with one key dymention).
Using fairly large objects, with 10 key dymentions I managed to store up to 900 thousand.

Times it took:
Adding small obejects 1 dim, 1 mil - next to no time (less than 2 seconds)
Adding Large objects 1 dim, 1.5 mil - 17 seconds
Adding Large objects 2 dim, 1.5 mil - 28 seconds
Adding Large objects 10 dim - 1min 14 seconds



    public class MKD2<K, V>
    {
        private readonly Dictionary<K, List<V>>[] dictionary;
 
        public byte DymentionCount
        {
            get;
            set;
        }
 
        public MKD2() 
        {
            DymentionCount = 1;
            dictionary = new Dictionary<K, List<V>>[1];
            dictionary[0] = new Dictionary<K, List<V>>(); 
        }
 
        public MKD2(byte dymentionCount)
        {
            if (dymentionCount == 0) dymentionCount = 1;
            DymentionCount = dymentionCount;
            dictionary = new Dictionary<K, List<V>>[DymentionCount];
            for (int i = 0; i < DymentionCount; i++)
            {
                dictionary[i] = new Dictionary<K, List<V>>();
            }
        }
 
        public int GetDimentionLength(byte dymention)
        {
            if (dictionary[dymention] == null)
                return 0;
            else
                return dictionary[dymention].Count;
        }
 
        public void Add(K key, V item)
        {
            List<V> entry;
            if (!dictionary[0].TryGetValue(key, out entry))
            {
                entry = new List<V>();
                dictionary[0].Add(key, entry);
            }
            entry.Add(item);
        }
 
        public void Add(K key, V item, byte dymention)
        {
            List<V> entry;
            if (!dictionary[dymention].TryGetValue(key, out entry))
            {
                entry = new List<V>();
                dictionary[dymention].Add(key, entry);
            }
            entry.Add(item);
        }
 
        public void Remove(K key)
        {
            dictionary[0].Remove(key);
        }
 
        public void Remove(K key, byte dymention)
        {
            dictionary[dymention].Remove(key);
        }
 
        public void Clear()
        {
            dictionary[0].Clear();
        }
 
        public void Clear(byte dymention)
        {
            dictionary[dymention].Clear();
        }
 
        public void Remove(K key, V value)
        {
            List<V> entry;
            if (dictionary[0].TryGetValue(key, out entry))
            {
                entry.Remove(value);
            }
        }
 
        public void Remove(K key, V value, byte dymention)
        {
            List<V> entry;
            if (dictionary[dymention].TryGetValue(key, out entry))
            {
                entry.Remove(value);
            }
        }
 
        public List<V> Get(K key) //IEnumerable<V>
        {
            List<V> entry;
            if (!dictionary[0].TryGetValue(key, out entry)) throw new Exception("Bad key");
            return entry; //might want to use .ToArray() here in multi threaded code
        }
 
        public List<V> Get(K key, byte dymention) //IEnumerable<V>
        {
            List<V> entry;
            if (!dictionary[dymention].TryGetValue(key, out entry)) throw new Exception("Bad key");
            return entry; //might want to use .ToArray() here in multi threaded code
        }
 
        public List<V> GetFlatDymentionValueList(byte dymention)
        {
            List<V> full_list = new List<V>();
            foreach (KeyValuePair<K,List<V>> kvp in dictionary[dymention])
	        {
                full_list.AddRange(kvp.Value);
	        }
            return full_list;
        }
 
        public List<K> GetFlatDimentionKeyList(byte dymention)
        {
            List<K> full_list = new List<K>();
            foreach (KeyValuePair<K, List<V>> kvp in dictionary[dymention])
            {
                full_list.Add(kvp.Key);
            }
            return full_list;
        }
    }

                                              
1:
2:
3:
4:
5:
6:
7:
8:
9:
10:
11:
12:
13:
14:
15:
16:
17:
18:
19:
20:
21:
22:
23:
24:
25:
26:
27:
28:
29:
30:
31:
32:
33:
34:
35:
36:
37:
38:
39:
40:
41:
42:
43:
44:
45:
46:
47:
48:
49:
50:
51:
52:
53:
54:
55:
56:
57:
58:
59:
60:
61:
62:
63:
64:
65:
66:
67:
68:
69:
70:
71:
72:
73:
74:
75:
76:
77:
78:
79:
80:
81:
82:
83:
84:
85:
86:
87:
88:
89:
90:
91:
92:
93:
94:
95:
96:
97:
98:
99:
100:
101:
102:
103:
104:
105:
106:
107:
108:
109:
110:
111:
112:
113:
114:
115:
116:
117:
118:
119:
120:
121:
122:
123:
124:
125:
126:
127:
128:
129:
130:

Select allOpen in new window

 

by: CallConnectionPosted on 2009-05-07 at 08:40:24ID: 24327085

..However one nagging problem is removing items from the collection; the memory doesn't appear to clear when I:
1) Add items to the collection (- a good 1.5 million)
2) Remove items;
3) Add some more items.

Debugging through the steps I can see hash collection is empty. So why doesn't the memory get released?

I have attached the code I used to remove the elelments...

Thanks,
  Dmitri

            foreach (KeyValuePair<long, List<ActivityDetails_HashedLib>> item in cht.GetKVPList(0))
            {
                foreach(ActivityDetails_HashedLib hl in item.Value)
                {
                    cht.Remove(item.Key, hl, 0);
                }
                cht.Remove(item.Key);
            }

                                              
1:
2:
3:
4:
5:
6:
7:
8:

Select allOpen in new window

 

by: gregoryyoungPosted on 2009-05-07 at 09:30:23ID: 24327632

This code should just be


            foreach (KeyValuePair<long, List<ActivityDetails_HashedLib>> item in cht.GetKVPList(0))
            {
                cht.Remove(item.Key);
            }

No?


How are you measuring that the "memory was not freed"?

Cheers,

Greg

 

by: CallConnectionPosted on 2009-05-08 at 07:11:11ID: 24336210

Yes, I tried that first of all, but it didn't work at the time..

It was because I used foreach instead of a for loop. I now get a list of keys first and iterate - and for loop works just fine. :)

Cheers,
  Dmitri

 

by: gregoryyoungPosted on 2009-05-08 at 08:24:45ID: 24336968

Glad it worked for you.

Good luck.

Greg

20120131-EE-VQP-002

3 Ways to Join

30-Day Free Trial

The Experts

98% positive feedback on 31,087 answers since March 2000. angeliii is a Microsoft Most Valuable Professional for his work with MS SQL Server & Develoment.

He has also proven his knowledge of Visual Basic Programming, PHP Scripting and Oracle Databases.

The Experts

97% positive feedback on 10,752 answers since July 2000. lrmoore has more than 18 years experience in the networking industry.

The six-time Mircosoft MVPs specialties include firewalls, virtual private networking, and network management.

Testimonials

"...and excellent source for support... Kind of like having your very own IT dept." Electriciansnet

Testimonials

"I was apprehensive at signing up at first. However... it has already made my life as an IT administrator much easier." JaCrews

Testimonials

"WOW! You guys have great, active, and knowledgeable people on here." moore50

Business Clients

Business Clients

In the Press

"If you’ve got a question... Experts Exchange can supply an answer.”

In the Press

"...an invaluable aid for both IT professionals and those who require tech support."

In the Press

"where IT professionals provide quick answers on just about any topic"

Business Account Plans

Loading Advertisement...