Solved

Reading and writig a large file in C# takes too long

Posted on 2007-03-26
2
1,943 Views
Last Modified: 2013-12-17
Hello experts:
I have a huge file (634 MB) that I want to read and then write, after changing the value of 3 fields.

The code is below. After the application ran for 2 hours(it was in the read records into Array section) I killed it.
 
 private void btnChange_Click(object sender, System.EventArgs e)
{
                         
  string OldValue = txtOldValue.Text;
   string NewValue = txtNewValue.Text;      

//Select input file
    openFileDialog1.ShowDialog();
   string MyFile = openFileDialog1.FileName;

//select output file
   SaveFileDialog1.ShowDialog();
   string OutFile = SaveFileDialog1.FileName;

// read records into Array
   StreamReader objReader = new StreamReader(MyFile);
   string sLine="";
   string Newline="";
   ArrayList arrText = new ArrayList();
   while (sLine != null)
   {
  sLine = objReader.ReadLine();
  if (sLine != null)
    arrText.Add(sLine);
   }
  objReader.Close();

//write records            
                               
  StreamWriter streamOut = new StreamWriter(OutFile);

  foreach (string sOutput in arrText)
  Newline = sOutput.Replace(OldValue, NewValue);      
  streamOut.WriteLine(Newline);

  streamOut.Close();      
  Application.Exit();                              
}

Any help or suggestion will be greatly appreciated.

Thanks

lancerxe
0
Comment
Question by:lancerxe
2 Comments
 
LVL 85

Accepted Solution

by:
Mike Tomlinson earned 500 total points
ID: 18796719
Instead of keeping the WHOLE thing in memory at once...

Open two files...source and temp target.
Read a line from source file and echo it to temp target with the changes.
When done, close both files.
Delete source.
Rename temp target.
Done.
0
 

Author Comment

by:lancerxe
ID: 18800325
Thanks Idle Mind
That did it.
0

Featured Post

Maximize Your Threat Intelligence Reporting

Reporting is one of the most important and least talked about aspects of a world-class threat intelligence program. Here’s how to do it right.

Join & Write a Comment

The object model of .Net can be overwhelming at times – so overwhelming that quite trivial tasks often take hours of research. In this case, the task at hand was to populate the datagrid from SQL Server database in Visual Studio 2008 Windows applica…
More often than not, we developers are confronted with a need: a need to make some kind of magic happen via code. Whether it is for a client, for the boss, or for our own personal projects, the need must be satisfied. Most of the time, the Framework…
Here's a very brief overview of the methods PRTG Network Monitor (https://www.paessler.com/prtg) offers for monitoring bandwidth, to help you decide which methods you´d like to investigate in more detail.  The methods are covered in more detail in o…
Polish reports in Access so they look terrific. Take yourself to another level. Equations, Back Color, Alternate Back Color. Write easy VBA Code. Tighten space to use less pages. Launch report from a menu, considering criteria only when it is filled…

758 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

21 Experts available now in Live!

Get 1:1 Help Now