C# Searching through a large ARRAY

What is the most efficient way to search through a large ARRAY.  I have a array with about 20,000 rows and I need to find rows that have a certain value populated.  Is there a better way of doing this?  Would it be better if I used a datatable instead of an array?  This is ASPX C# program.

Here is how I am searching through the Array now:
var FilteredProducts = GetAllConvertedValue.Where(i => i.value1 == li.Text).FirstOrDefault();

Here is how the Array is defined:
List<FilteredValuesOfAttribure> GetAllConvertedValue = new List<FilteredValuesOfAttribure>();

public class FilteredValuesOfAttribure
    {
        public string attributeURL { get; set; }
        public string attributeTtitle { get; set; }
        public string value1 { get; set; }
        public string id { get; set; }
        public string name { get; set; }
    }
tmajor99Asked:
Who is Participating?
I wear a lot of hats...

"The solutions and answers provided on Experts Exchange have been extremely helpful to me over the last few years. I wear a lot of hats - Developer, Database Administrator, Help Desk, etc., so I know a lot of things but not a lot about one thing. Experts Exchange gives me answers from people who do know a lot about one thing, in a easy to use platform." -Todd S.

Fernando SotoRetiredCommented:
So why are you unhappy with that approach it looks fine to me?
0
tmajor99Author Commented:
I am asking if there is a better more efficient way to search through a large set of data.
0
käµfm³d 👽Commented:
Efficient how? In terms of speed? In terms of memory usage?

Generally speaking, a sorted array/collection can be searched through faster (on average) than an unsorted array/collection.
0
Cloud Class® Course: C++ 11 Fundamentals

This course will introduce you to C++ 11 and teach you about syntax fundamentals.

tmajor99Author Commented:
Yes in terms of speed.
0
käµfm³d 👽Commented:
Are you able to sort the array and keep it sorted during the lifetime of your application?
0
tmajor99Author Commented:
Yes.
0
käµfm³d 👽Commented:
Array.Sort has an interesting algorithm for sorting (described in the link). You could use that for the initial sort. Then there are various algorithms you could use for the actual searching. Binary search is pretty fast, and it's already implemented on the Array class. There are other algorithms that you could implement, but AFAIK only that one is implemented out of the box--someone else can correct me otherwise.
0
käµfm³d 👽Commented:
Also, if you're really using the List class, and not an actual array, then those two methods are also implemented on List as well:

0
it_saigeDeveloperCommented:
When dealing with 20000 records, that is not really a large set of data.  On average any method you choose will process the set of data in less than 1 ms.

However, when dealing with hundreds of thousand or millions records.  Then the method efficiency starts to get called into question.

So let's mock up one at 2 million records and get approximately the middle record:
using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Diagnostics;

namespace EE_Q28690669
{
	class Program
	{
		static List<FilteredValuesOfAttribute> attributes = new List<FilteredValuesOfAttribute>();
		static Stopwatch watch = new Stopwatch();

		static void Main(string[] args)
		{
			for (int i = 0; i < 2000000; i++)
				attributes.Add(new FilteredValuesOfAttribute() { ID = i.ToString(), Name = string.Format("Attribute{0}", i), Title = string.Format("Title{0}", i), URL = string.Format("URL{0}", i), Value = string.Format("Value{0}", i) });

			Console.WriteLine("Let's compare some times:");
			StandardForLoop();
			ImprovedForLoop();
			ForEachLoop();
			ImprovedForEachLoop();
			LinqBasedLookup();
			Console.ReadLine();
		}

		static FilteredValuesOfAttribute StandardForLoop()
		{
			watch.Reset();
			Console.WriteLine();
			Console.WriteLine("Finding a value using a standard for loop:");
			watch.Start();
			FilteredValuesOfAttribute result = default(FilteredValuesOfAttribute);
			for (int i = 0; i < attributes.Count - 1; i++)
			{
				if (attributes[i].Value == "Value-1")
					result = attributes[i];
			}
			watch.Stop();
			Console.WriteLine("Finding a value using a standard for loop took: {0} ms", watch.ElapsedMilliseconds);
			return result;
		}

		static FilteredValuesOfAttribute ImprovedForLoop()
		{
			watch.Reset();
			Console.WriteLine();
			Console.WriteLine("Finding a value using an improved for loop (we break on first value found):");
			watch.Start();
			FilteredValuesOfAttribute result = default(FilteredValuesOfAttribute);
			for (int i = 0; i < attributes.Count - 1; i++)
			{
				if (attributes[i].Value == "Value-1")
				{
					result = attributes[i];
					break;
				}
			}
			watch.Stop();
			Console.WriteLine("Finding a value using an improved for loop took: {0} ms", watch.ElapsedMilliseconds);
			return result;
		}

		static FilteredValuesOfAttribute ForEachLoop()
		{
			watch.Reset();
			Console.WriteLine();
			Console.WriteLine("Finding a value using a standard foreach loop:");
			watch.Start();
			FilteredValuesOfAttribute result = default(FilteredValuesOfAttribute);
			foreach (var attribute in attributes)
			{
				if (attribute.Value == "Value-1")
					result = attribute;
			}
			watch.Stop();
			Console.WriteLine("Finding a value using a standard foreach loop took: {0} ms", watch.ElapsedMilliseconds);
			return result;
		}

		static FilteredValuesOfAttribute ImprovedForEachLoop()
		{
			watch.Reset();
			Console.WriteLine();
			Console.WriteLine("Finding a value using an improved foreach loop (we break on first value found):");
			watch.Start();
			FilteredValuesOfAttribute result = default(FilteredValuesOfAttribute);
			foreach (var attribute in attributes)
			{
				if (attribute.Value == "Value-1")
				{
					result = attribute;
					break;
				}
			}
			watch.Stop();
			Console.WriteLine("Finding a value using an improved foreach loop took: {0} ms", watch.ElapsedMilliseconds);
			return result;
		}

		static FilteredValuesOfAttribute LinqBasedLookup()
		{
			watch.Reset();
			Console.WriteLine();
			Console.WriteLine("Finding a value using a linq based lookup:");
			watch.Start();
			var result = attributes.FirstOrDefault(attribute => attribute.Value == "Value-1");
			watch.Stop();
			Console.WriteLine("Finding a value using a linq based lookup took: {0} ms", watch.ElapsedMilliseconds);
			return result;
		}
	}

	class FilteredValuesOfAttribute
	{
		public string ID { get; set; }
		public string URL { get; set; }
		public string Title { get; set; }
		public string Value { get; set; }
		public string Name { get; set; }
	}
}

Open in new window

As shown here (results will vary) -Capture.JPGA linq based approach is quicker than an approach that iterates through the list (but not one that iterates to find the first record).  But what about the default (or non-existent record), will Linq find this quicker?Capture.JPGAs it turns out, it does not; matter of fact Linq is slower than your standard for loop or foreach loop.  Most likely the reason for this is that Linq has to create a default object to return when the lookup fails.

Edit: Note this does not take into account sorting as kaufmed has pointed out.

-saige-
0
AndyAinscowFreelance programmer / ConsultantCommented:
Depending on what you need the collection for - a dictionary is incredibly fast to find individual items when you know the key it is stored with.  (MUCH faster than a list)
0

Experts Exchange Solution brought to you by

Your issues matter to us.

Facing a tech roadblock? Get the help and guidance you need from experienced professionals who care. Ask your question anytime, anywhere, with no hassle.

Start your 7-day free trial
It's more than this solution.Get answers and train to solve all your tech problems - anytime, anywhere.Try it for free Edge Out The Competitionfor your dream job with proven skills and certifications.Get started today Stand Outas the employee with proven skills.Start learning today for free Move Your Career Forwardwith certification training in the latest technologies.Start your trial today
C#

From novice to tech pro — start learning today.