Solved

How to speed up the creation of output records in a C# console application using C# with VS2010?

Posted on 2014-01-12
6
562 Views
Last Modified: 2014-01-16
I developed a C# Console application (see attachment) using VS2010 that reads an input file with no file extension and creates output records.
 
Do you know how I could improve the speed of this application since it takes almost 1 minute to write each output record.

Then the last file created (with the file extension ard.out) is written out about 30 minutes later.
ch.txt
0
Comment
Question by:zimmer9
[X]
Welcome to Experts Exchange

Add your voice to the tech community where 5M+ people just like you are talking about what matters.

  • Help others & share knowledge
  • Earn cash & points
  • Learn & ask questions
6 Comments
 
LVL 40

Assisted Solution

by:Jacques Bourgeois (James Burger)
Jacques Bourgeois (James Burger) earned 300 total points
ID: 39775314
Using constants for recurrent string values such as "COMMENT: CHECK NUMBER #" sometimes help. It depends on what the optimizer does when compiling.

You are using a lot of variables that you recreate each time through the loop. Declaring them before entering the loop might already help you a bit.

But the main problem is probably the amount of concatenation. Concatenations are inherently slow. The most common way of saving time in such a case is to use a System.Text.StringBuilder object to build the string. Although at first sight it might see as it takes more code to write, because the way it works the difference is always enormous:
System.Text.StringBuilder builder; // Define the object before going in the loop
.....
builder.Clear();
builder.Append( "COMMENT: CHECK NUMBER #"); // Use your constants here if you go that route
builder.Append( CheckOrderNum);
builder.Append(Environment.NewLine);
builder.Append( "GROUP_FIELD_NAME:CheckNumber");
builder.Append("GROUP_FIELD_VALUE:")
builder.Append(checknum);
builder.Append(Environment.NewLine);
......
sw.WriteLine(builder.ToString()

Open in new window

Instead of creating a lot of variables, you are creating a single object. You are also writing the whole file with only one WriteLine.
0
 
LVL 36

Assisted Solution

by:Miguel Oz
Miguel Oz earned 150 total points
ID: 39775774
Adding to James comments you need to check how you interact with the following file:
string indFilePath = System.IO.Path.Combine(MyGlobals.BASE_DIR, MyGlobals.BASE_FILE_Name + ".ard.ind");
                            using (StreamWriter sw = new StreamWriter(indFilePath, true))

Currently this file is open in the loop, you are better off opening only once at the start of the loop.

Notes:
1) Keep in mind that you are using Ling to manipulate bytes and that may cause you some issues as well.
2) Use a Performance tool (VS 2010 and up Ultimate edition has one for free) to check which lines are performing badly or you can use the trial version of Red Gate performance profiler (commercial tool)
0
 
LVL 13

Accepted Solution

by:
Naman Goel earned 50 total points
ID: 39775953
agree with James and Mas, also please don't use

parentDirectory.GetFiles() inside foreach loop

line #86

do something like this:

FileInfo[] fileInfos= parentDirectory.GetFiles();

    foreach (FileInfo file in fileInfos)
0
MS Dynamics Made Instantly Simpler

Make Your Microsoft Dynamics Investment Count  & Drastically Decrease Training Time by Providing Intuitive Step-By-Step WalkThru Tutorials.

 
LVL 40
ID: 39786975
Strange that the accepted solution is the one that got the smaller number of points.
0
 

Author Comment

by:zimmer9
ID: 39786981
my bad, can I correct the error of my ways
0
 
LVL 40
ID: 39787208
No problem.

I am here to give back, to help, not for the glory :-).

As a programmer who needs to concentrate on the détails, I just found it strange.
0

Featured Post

On Demand Webinar - Networking for the Cloud Era

This webinar discusses:
-Common barriers companies experience when moving to the cloud
-How SD-WAN changes the way we look at networks
-Best practices customers should employ moving forward with cloud migration
-What happens behind the scenes of SteelConnect’s one-click button

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Suggested Solutions

More often than not, we developers are confronted with a need: a need to make some kind of magic happen via code. Whether it is for a client, for the boss, or for our own personal projects, the need must be satisfied. Most of the time, the Framework…
This article shows how to deploy dynamic backgrounds to computers depending on the aspect ratio of display
Nobody understands Phishing better than an anti-spam company. That’s why we are providing Phishing Awareness Training to our customers. According to a report by Verizon, only 3% of targeted users report malicious emails to management. With compan…

726 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question