Solved

Reading and writig a large file in C# takes too long

Posted on 2007-03-26
2
1,944 Views
Last Modified: 2013-12-17
Hello experts:
I have a huge file (634 MB) that I want to read and then write, after changing the value of 3 fields.

The code is below. After the application ran for 2 hours(it was in the read records into Array section) I killed it.
 
 private void btnChange_Click(object sender, System.EventArgs e)
{
                         
  string OldValue = txtOldValue.Text;
   string NewValue = txtNewValue.Text;      

//Select input file
    openFileDialog1.ShowDialog();
   string MyFile = openFileDialog1.FileName;

//select output file
   SaveFileDialog1.ShowDialog();
   string OutFile = SaveFileDialog1.FileName;

// read records into Array
   StreamReader objReader = new StreamReader(MyFile);
   string sLine="";
   string Newline="";
   ArrayList arrText = new ArrayList();
   while (sLine != null)
   {
  sLine = objReader.ReadLine();
  if (sLine != null)
    arrText.Add(sLine);
   }
  objReader.Close();

//write records            
                               
  StreamWriter streamOut = new StreamWriter(OutFile);

  foreach (string sOutput in arrText)
  Newline = sOutput.Replace(OldValue, NewValue);      
  streamOut.WriteLine(Newline);

  streamOut.Close();      
  Application.Exit();                              
}

Any help or suggestion will be greatly appreciated.

Thanks

lancerxe
0
Comment
Question by:lancerxe
2 Comments
 
LVL 85

Accepted Solution

by:
Mike Tomlinson earned 500 total points
ID: 18796719
Instead of keeping the WHOLE thing in memory at once...

Open two files...source and temp target.
Read a line from source file and echo it to temp target with the changes.
When done, close both files.
Delete source.
Rename temp target.
Done.
0
 

Author Comment

by:lancerxe
ID: 18800325
Thanks Idle Mind
That did it.
0

Featured Post

Is Your Active Directory as Secure as You Think?

More than 75% of all records are compromised because of the loss or theft of a privileged credential. Experts have been exploring Active Directory infrastructure to identify key threats and establish best practices for keeping data safe. Attend this month’s webinar to learn more.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

This document covers how to connect to SQL Server and browse its contents.  It is meant for those new to Visual Studio and/or working with Microsoft SQL Server.  It is not a guide to building SQL Server database connections in your code.  This is mo…
Real-time is more about the business, not the technology. In day-to-day life, to make real-time decisions like buying or investing, business needs the latest information(e.g. Gold Rate/Stock Rate). Unlike traditional days, you need not wait for a fe…
I designed this idea while studying technology in the classroom.  This is a semester long project.  Students are asked to take photographs on a specific topic which they find meaningful, it can be a place or situation such as travel or homelessness.…
This is a video that shows how the OnPage alerts system integrates into ConnectWise, how a trigger is set, how a page is sent via the trigger, and how the SENT, DELIVERED, READ & REPLIED receipts get entered into the internal tab of the ConnectWise …

947 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

20 Experts available now in Live!

Get 1:1 Help Now