How can I improve the foreach to possibly a linq for bettter performance for large number of rows over 500000

Posted on 2013-10-03
Medium Priority
Last Modified: 2016-02-10
foreach (DataRow item in priceTable.Rows)
                  DataRow[] drow = dtPrices.Select("ID = '" + item["ID"].ToString() + "' AND " + "Date = '" + item["Date"].ToString() + "'");
                  drow[0]["TotalReturn"] = Convert.ToDecimal(item["TotalReturn"]);
Question by:countrymeister
LVL 40

Accepted Solution

Jacques Bourgeois (James Burger) earned 1500 total points
ID: 39545678
Always to as much of the job as you need in the database before fetching the data. Databases are optimized for that type of job.

You have 500000 rows in your computer memory, there is not much memory left. And you are doing a lot of concatenations, which are costly in time and resources.

Instead of looping and concatenating in your application, do the job in the query that fetch the data.
LVL 96

Expert Comment

by:Bob Learned
ID: 39546675
What are you doing with the data?  

Can you convert this to a function that returns IEnumerable<T>, and then you can use the yield return statement to return each element in the iterator individually?

Author Closing Comment

ID: 39594689
Just a general suggestion

Featured Post

Free Tool: ZipGrep

ZipGrep is a utility that can list and search zip (.war, .ear, .jar, etc) archives for text patterns, without the need to extract the archive's contents.

One of a set of tools we're offering as a way to say thank you for being a part of the community.

Question has a verified solution.

Are you are experiencing a similar issue? Get a personalized answer when you ask a related question.

Have a better answer? Share it in a comment.

Join & Write a Comment

Entity Framework is a powerful tool to help you interact with the DataBase but still doesn't help much when we have a Stored Procedure that returns more than one resultset. The solution takes some of out-of-the-box thinking; read on!
Real-time is more about the business, not the technology. In day-to-day life, to make real-time decisions like buying or investing, business needs the latest information(e.g. Gold Rate/Stock Rate). Unlike traditional days, you need not wait for a fe…
In response to a need for security and privacy, and to continue fostering an environment members can turn to for support, solutions, and education, Experts Exchange has created anonymous question capabilities. This new feature is available to our Pr…
If you are looking for an automated tool which can generate reports for Outlook emails and other items from PST file, then you can go for Kernel PST Reporter tool. The reports which are created by this tool are helpful to analyze and understand PST …

623 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question