Solved

How to speed up the creation of output records in a C# console application using C# with VS2010?

Posted on 2014-01-12
6
540 Views
Last Modified: 2014-01-16
I developed a C# Console application (see attachment) using VS2010 that reads an input file with no file extension and creates output records.
 
Do you know how I could improve the speed of this application since it takes almost 1 minute to write each output record.

Then the last file created (with the file extension ard.out) is written out about 30 minutes later.
ch.txt
0
Comment
Question by:zimmer9
6 Comments
 
LVL 40

Assisted Solution

by:Jacques Bourgeois (James Burger)
Jacques Bourgeois (James Burger) earned 300 total points
ID: 39775314
Using constants for recurrent string values such as "COMMENT: CHECK NUMBER #" sometimes help. It depends on what the optimizer does when compiling.

You are using a lot of variables that you recreate each time through the loop. Declaring them before entering the loop might already help you a bit.

But the main problem is probably the amount of concatenation. Concatenations are inherently slow. The most common way of saving time in such a case is to use a System.Text.StringBuilder object to build the string. Although at first sight it might see as it takes more code to write, because the way it works the difference is always enormous:
System.Text.StringBuilder builder; // Define the object before going in the loop
.....
builder.Clear();
builder.Append( "COMMENT: CHECK NUMBER #"); // Use your constants here if you go that route
builder.Append( CheckOrderNum);
builder.Append(Environment.NewLine);
builder.Append( "GROUP_FIELD_NAME:CheckNumber");
builder.Append("GROUP_FIELD_VALUE:")
builder.Append(checknum);
builder.Append(Environment.NewLine);
......
sw.WriteLine(builder.ToString()

Open in new window

Instead of creating a lot of variables, you are creating a single object. You are also writing the whole file with only one WriteLine.
0
 
LVL 35

Assisted Solution

by:Miguel Oz
Miguel Oz earned 150 total points
ID: 39775774
Adding to James comments you need to check how you interact with the following file:
string indFilePath = System.IO.Path.Combine(MyGlobals.BASE_DIR, MyGlobals.BASE_FILE_Name + ".ard.ind");
                            using (StreamWriter sw = new StreamWriter(indFilePath, true))

Currently this file is open in the loop, you are better off opening only once at the start of the loop.

Notes:
1) Keep in mind that you are using Ling to manipulate bytes and that may cause you some issues as well.
2) Use a Performance tool (VS 2010 and up Ultimate edition has one for free) to check which lines are performing badly or you can use the trial version of Red Gate performance profiler (commercial tool)
0
 
LVL 13

Accepted Solution

by:
Naman Goel earned 50 total points
ID: 39775953
agree with James and Mas, also please don't use

parentDirectory.GetFiles() inside foreach loop

line #86

do something like this:

FileInfo[] fileInfos= parentDirectory.GetFiles();

    foreach (FileInfo file in fileInfos)
0
Do You Know the 4 Main Threat Actor Types?

Do you know the main threat actor types? Most attackers fall into one of four categories, each with their own favored tactics, techniques, and procedures.

 
LVL 40
ID: 39786975
Strange that the accepted solution is the one that got the smaller number of points.
0
 

Author Comment

by:zimmer9
ID: 39786981
my bad, can I correct the error of my ways
0
 
LVL 40
ID: 39787208
No problem.

I am here to give back, to help, not for the glory :-).

As a programmer who needs to concentrate on the détails, I just found it strange.
0

Featured Post

Highfive + Dolby Voice = No More Audio Complaints!

Poor audio quality is one of the top reasons people don’t use video conferencing. Get the crispest, clearest audio powered by Dolby Voice in every meeting. Highfive and Dolby Voice deliver the best video conferencing and audio experience for every meeting and every room.

Join & Write a Comment

Introduction Hi all and welcome to my first article on Experts Exchange. A while ago, someone asked me if i could do some tutorials on object oriented programming. I decided to do them on C#. Now you may ask me, why's that? Well, one of the re…
Today I had a very interesting conundrum that had to get solved quickly. Needless to say, it wasn't resolved quickly because when we needed it we were very rushed, but as soon as the conference call was over and I took a step back I saw the correct …
Get a first impression of how PRTG looks and learn how it works.   This video is a short introduction to PRTG, as an initial overview or as a quick start for new PRTG users.
This tutorial demonstrates a quick way of adding group price to multiple Magento products.

707 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

15 Experts available now in Live!

Get 1:1 Help Now