Solved

Reading and writig a large file in C# takes too long

Posted on 2007-03-26
2
1,947 Views
Last Modified: 2013-12-17
Hello experts:
I have a huge file (634 MB) that I want to read and then write, after changing the value of 3 fields.

The code is below. After the application ran for 2 hours(it was in the read records into Array section) I killed it.
 
 private void btnChange_Click(object sender, System.EventArgs e)
{
                         
  string OldValue = txtOldValue.Text;
   string NewValue = txtNewValue.Text;      

//Select input file
    openFileDialog1.ShowDialog();
   string MyFile = openFileDialog1.FileName;

//select output file
   SaveFileDialog1.ShowDialog();
   string OutFile = SaveFileDialog1.FileName;

// read records into Array
   StreamReader objReader = new StreamReader(MyFile);
   string sLine="";
   string Newline="";
   ArrayList arrText = new ArrayList();
   while (sLine != null)
   {
  sLine = objReader.ReadLine();
  if (sLine != null)
    arrText.Add(sLine);
   }
  objReader.Close();

//write records            
                               
  StreamWriter streamOut = new StreamWriter(OutFile);

  foreach (string sOutput in arrText)
  Newline = sOutput.Replace(OldValue, NewValue);      
  streamOut.WriteLine(Newline);

  streamOut.Close();      
  Application.Exit();                              
}

Any help or suggestion will be greatly appreciated.

Thanks

lancerxe
0
Comment
Question by:lancerxe
2 Comments
 
LVL 85

Accepted Solution

by:
Mike Tomlinson earned 500 total points
ID: 18796719
Instead of keeping the WHOLE thing in memory at once...

Open two files...source and temp target.
Read a line from source file and echo it to temp target with the changes.
When done, close both files.
Delete source.
Rename temp target.
Done.
0
 

Author Comment

by:lancerxe
ID: 18800325
Thanks Idle Mind
That did it.
0

Featured Post

Master Your Team's Linux and Cloud Stack!

The average business loses $13.5M per year to ineffective training (per 1,000 employees). Keep ahead of the competition and combine in-person quality with online cost and flexibility by training with Linux Academy.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

The object model of .Net can be overwhelming at times – so overwhelming that quite trivial tasks often take hours of research. In this case, the task at hand was to populate the datagrid from SQL Server database in Visual Studio 2008 Windows applica…
In my previous two articles we discussed Binary Serialization (http://www.experts-exchange.com/A_4362.html) and XML Serialization (http://www.experts-exchange.com/A_4425.html). In this article we will try to know more about SOAP (Simple Object Acces…
This video shows how to quickly and easily add an email signature for all users on Exchange 2016. The resulting signature is applied on a server level by Exchange Online. The email signature template has been downloaded from: www.mail-signatures…
This video shows how to use Hyena, from SystemTools Software, to bulk import 100 user accounts from an external text file. View in 1080p for best video quality.

778 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question