Solved

How to speed up the creation of output records in a C# console application using C# with VS2010?

Posted on 2014-01-12
6
554 Views
Last Modified: 2014-01-16
I developed a C# Console application (see attachment) using VS2010 that reads an input file with no file extension and creates output records.
 
Do you know how I could improve the speed of this application since it takes almost 1 minute to write each output record.

Then the last file created (with the file extension ard.out) is written out about 30 minutes later.
ch.txt
0
Comment
Question by:zimmer9
6 Comments
 
LVL 40

Assisted Solution

by:Jacques Bourgeois (James Burger)
Jacques Bourgeois (James Burger) earned 300 total points
ID: 39775314
Using constants for recurrent string values such as "COMMENT: CHECK NUMBER #" sometimes help. It depends on what the optimizer does when compiling.

You are using a lot of variables that you recreate each time through the loop. Declaring them before entering the loop might already help you a bit.

But the main problem is probably the amount of concatenation. Concatenations are inherently slow. The most common way of saving time in such a case is to use a System.Text.StringBuilder object to build the string. Although at first sight it might see as it takes more code to write, because the way it works the difference is always enormous:
System.Text.StringBuilder builder; // Define the object before going in the loop
.....
builder.Clear();
builder.Append( "COMMENT: CHECK NUMBER #"); // Use your constants here if you go that route
builder.Append( CheckOrderNum);
builder.Append(Environment.NewLine);
builder.Append( "GROUP_FIELD_NAME:CheckNumber");
builder.Append("GROUP_FIELD_VALUE:")
builder.Append(checknum);
builder.Append(Environment.NewLine);
......
sw.WriteLine(builder.ToString()

Open in new window

Instead of creating a lot of variables, you are creating a single object. You are also writing the whole file with only one WriteLine.
0
 
LVL 35

Assisted Solution

by:Miguel Oz
Miguel Oz earned 150 total points
ID: 39775774
Adding to James comments you need to check how you interact with the following file:
string indFilePath = System.IO.Path.Combine(MyGlobals.BASE_DIR, MyGlobals.BASE_FILE_Name + ".ard.ind");
                            using (StreamWriter sw = new StreamWriter(indFilePath, true))

Currently this file is open in the loop, you are better off opening only once at the start of the loop.

Notes:
1) Keep in mind that you are using Ling to manipulate bytes and that may cause you some issues as well.
2) Use a Performance tool (VS 2010 and up Ultimate edition has one for free) to check which lines are performing badly or you can use the trial version of Red Gate performance profiler (commercial tool)
0
 
LVL 13

Accepted Solution

by:
Naman Goel earned 50 total points
ID: 39775953
agree with James and Mas, also please don't use

parentDirectory.GetFiles() inside foreach loop

line #86

do something like this:

FileInfo[] fileInfos= parentDirectory.GetFiles();

    foreach (FileInfo file in fileInfos)
0
Master Your Team's Linux and Cloud Stack!

The average business loses $13.5M per year to ineffective training (per 1,000 employees). Keep ahead of the competition and combine in-person quality with online cost and flexibility by training with Linux Academy.

 
LVL 40
ID: 39786975
Strange that the accepted solution is the one that got the smaller number of points.
0
 

Author Comment

by:zimmer9
ID: 39786981
my bad, can I correct the error of my ways
0
 
LVL 40
ID: 39787208
No problem.

I am here to give back, to help, not for the glory :-).

As a programmer who needs to concentrate on the détails, I just found it strange.
0

Featured Post

3 Use Cases for Connected Systems

Our Dev teams are like yours. They’re continually cranking out code for new features/bugs fixes, testing, deploying, testing some more, responding to production monitoring events and more. It’s complex. So, we thought you’d like to see what’s working for us.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Introduction Hi all and welcome to my first article on Experts Exchange. A while ago, someone asked me if i could do some tutorials on object oriented programming. I decided to do them on C#. Now you may ask me, why's that? Well, one of the re…
Entity Framework is a powerful tool to help you interact with the DataBase but still doesn't help much when we have a Stored Procedure that returns more than one resultset. The solution takes some of out-of-the-box thinking; read on!
Finds all prime numbers in a range requested and places them in a public primes() array. I've demostrated a template size of 30 (2 * 3 * 5) but larger templates can be built such 210  (2 * 3 * 5 * 7) or 2310  (2 * 3 * 5 * 7 * 11). The larger templa…

832 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question