Solved

How to speed up the creation of output records in a C# console application using C# with VS2010?

Posted on 2014-01-12
6
547 Views
Last Modified: 2014-01-16
I developed a C# Console application (see attachment) using VS2010 that reads an input file with no file extension and creates output records.
 
Do you know how I could improve the speed of this application since it takes almost 1 minute to write each output record.

Then the last file created (with the file extension ard.out) is written out about 30 minutes later.
ch.txt
0
Comment
Question by:zimmer9
6 Comments
 
LVL 40

Assisted Solution

by:Jacques Bourgeois (James Burger)
Jacques Bourgeois (James Burger) earned 300 total points
ID: 39775314
Using constants for recurrent string values such as "COMMENT: CHECK NUMBER #" sometimes help. It depends on what the optimizer does when compiling.

You are using a lot of variables that you recreate each time through the loop. Declaring them before entering the loop might already help you a bit.

But the main problem is probably the amount of concatenation. Concatenations are inherently slow. The most common way of saving time in such a case is to use a System.Text.StringBuilder object to build the string. Although at first sight it might see as it takes more code to write, because the way it works the difference is always enormous:
System.Text.StringBuilder builder; // Define the object before going in the loop
.....
builder.Clear();
builder.Append( "COMMENT: CHECK NUMBER #"); // Use your constants here if you go that route
builder.Append( CheckOrderNum);
builder.Append(Environment.NewLine);
builder.Append( "GROUP_FIELD_NAME:CheckNumber");
builder.Append("GROUP_FIELD_VALUE:")
builder.Append(checknum);
builder.Append(Environment.NewLine);
......
sw.WriteLine(builder.ToString()

Open in new window

Instead of creating a lot of variables, you are creating a single object. You are also writing the whole file with only one WriteLine.
0
 
LVL 35

Assisted Solution

by:Miguel Oz
Miguel Oz earned 150 total points
ID: 39775774
Adding to James comments you need to check how you interact with the following file:
string indFilePath = System.IO.Path.Combine(MyGlobals.BASE_DIR, MyGlobals.BASE_FILE_Name + ".ard.ind");
                            using (StreamWriter sw = new StreamWriter(indFilePath, true))

Currently this file is open in the loop, you are better off opening only once at the start of the loop.

Notes:
1) Keep in mind that you are using Ling to manipulate bytes and that may cause you some issues as well.
2) Use a Performance tool (VS 2010 and up Ultimate edition has one for free) to check which lines are performing badly or you can use the trial version of Red Gate performance profiler (commercial tool)
0
 
LVL 13

Accepted Solution

by:
Naman Goel earned 50 total points
ID: 39775953
agree with James and Mas, also please don't use

parentDirectory.GetFiles() inside foreach loop

line #86

do something like this:

FileInfo[] fileInfos= parentDirectory.GetFiles();

    foreach (FileInfo file in fileInfos)
0
3 Use Cases for Connected Systems

Our Dev teams are like yours. They’re continually cranking out code for new features/bugs fixes, testing, deploying, testing some more, responding to production monitoring events and more. It’s complex. So, we thought you’d like to see what’s working for us.

 
LVL 40
ID: 39786975
Strange that the accepted solution is the one that got the smaller number of points.
0
 

Author Comment

by:zimmer9
ID: 39786981
my bad, can I correct the error of my ways
0
 
LVL 40
ID: 39787208
No problem.

I am here to give back, to help, not for the glory :-).

As a programmer who needs to concentrate on the détails, I just found it strange.
0

Featured Post

DevOps Toolchain Recommendations

Read this Gartner Research Note and discover how your IT organization can automate and optimize DevOps processes using a toolchain architecture.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

In my previous two articles we discussed Binary Serialization (http://www.experts-exchange.com/A_4362.html) and XML Serialization (http://www.experts-exchange.com/A_4425.html). In this article we will try to know more about SOAP (Simple Object Acces…
This document covers how to connect to SQL Server and browse its contents.  It is meant for those new to Visual Studio and/or working with Microsoft SQL Server.  It is not a guide to building SQL Server database connections in your code.  This is mo…
Windows 10 is mostly good. However the one thing that annoys me is how many clicks you have to do to dial a VPN connection. You have to go to settings from the start menu, (2 clicks), Network and Internet (1 click), Click VPN (another click) then fi…
Internet Business Fax to Email Made Easy - With eFax Corporate (http://www.enterprise.efax.com), you'll receive a dedicated online fax number, which is used the same way as a typical analog fax number. You'll receive secure faxes in your email, fr…

895 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

18 Experts available now in Live!

Get 1:1 Help Now