• Status: Solved
  • Priority: Medium
  • Security: Public
  • Views: 613
  • Last Modified:

How to speed up the creation of output records in a C# console application using C# with VS2010?

I developed a C# Console application (see attachment) using VS2010 that reads an input file with no file extension and creates output records.
 
Do you know how I could improve the speed of this application since it takes almost 1 minute to write each output record.

Then the last file created (with the file extension ard.out) is written out about 30 minutes later.
ch.txt
0
zimmer9
Asked:
zimmer9
3 Solutions
 
Jacques Bourgeois (James Burger)Commented:
Using constants for recurrent string values such as "COMMENT: CHECK NUMBER #" sometimes help. It depends on what the optimizer does when compiling.

You are using a lot of variables that you recreate each time through the loop. Declaring them before entering the loop might already help you a bit.

But the main problem is probably the amount of concatenation. Concatenations are inherently slow. The most common way of saving time in such a case is to use a System.Text.StringBuilder object to build the string. Although at first sight it might see as it takes more code to write, because the way it works the difference is always enormous:
System.Text.StringBuilder builder; // Define the object before going in the loop
.....
builder.Clear();
builder.Append( "COMMENT: CHECK NUMBER #"); // Use your constants here if you go that route
builder.Append( CheckOrderNum);
builder.Append(Environment.NewLine);
builder.Append( "GROUP_FIELD_NAME:CheckNumber");
builder.Append("GROUP_FIELD_VALUE:")
builder.Append(checknum);
builder.Append(Environment.NewLine);
......
sw.WriteLine(builder.ToString()

Open in new window

Instead of creating a lot of variables, you are creating a single object. You are also writing the whole file with only one WriteLine.
0
 
Miguel OzSoftware EngineerCommented:
Adding to James comments you need to check how you interact with the following file:
string indFilePath = System.IO.Path.Combine(MyGlobals.BASE_DIR, MyGlobals.BASE_FILE_Name + ".ard.ind");
                            using (StreamWriter sw = new StreamWriter(indFilePath, true))

Currently this file is open in the loop, you are better off opening only once at the start of the loop.

Notes:
1) Keep in mind that you are using Ling to manipulate bytes and that may cause you some issues as well.
2) Use a Performance tool (VS 2010 and up Ultimate edition has one for free) to check which lines are performing badly or you can use the trial version of Red Gate performance profiler (commercial tool)
0
 
Naman GoelCommented:
agree with James and Mas, also please don't use

parentDirectory.GetFiles() inside foreach loop

line #86

do something like this:

FileInfo[] fileInfos= parentDirectory.GetFiles();

    foreach (FileInfo file in fileInfos)
0
Independent Software Vendors: We Want Your Opinion

We value your feedback.

Take our survey and automatically be enter to win anyone of the following:
Yeti Cooler, Amazon eGift Card, and Movie eGift Card!

 
Jacques Bourgeois (James Burger)Commented:
Strange that the accepted solution is the one that got the smaller number of points.
0
 
zimmer9Author Commented:
my bad, can I correct the error of my ways
0
 
Jacques Bourgeois (James Burger)Commented:
No problem.

I am here to give back, to help, not for the glory :-).

As a programmer who needs to concentrate on the d├ętails, I just found it strange.
0

Featured Post

Independent Software Vendors: We Want Your Opinion

We value your feedback.

Take our survey and automatically be enter to win anyone of the following:
Yeti Cooler, Amazon eGift Card, and Movie eGift Card!

Tackle projects and never again get stuck behind a technical roadblock.
Join Now