Delete line from text file

I am trying to delete a line in a text
Here is the scenario:
there was an error like no connection to the database. so I have a streamwriter write the sql query to a text file - when there is a connection the code checks to see if the file has any lines - if it does it processes it . If the app successfully uploads that line of sql code I want to delete that line from the text file and continue to process the file till it is complete.
mycount is the count of lines in the text file
 if (mycount > 0)
            {
                string line = string.Empty;
                string newline = string.Empty;
                int lineNumber = 0;
                string sqlStr = string.Empty;
                string sqlfile = "\\sqlfile.txt";
               // string sqlfile1 = "\\sqlfilenew.txt";
                bool iWrote = false;
                DirectoryInfo di = new DirectoryInfo(Directory.GetCurrentDirectory());
                System.IO.StreamReader file = new System.IO.StreamReader(di.ToString() + sqlfile);
                while ((sqlStr = file.ReadLine()) != null)
                {
                    
                    try
                    {
                        iWrote = ss.UploadBackLog(sqlStr);
                        file.Close();
                        if (iWrote)
                        {
                            System.IO.StreamReader cutline = new System.IO.StreamReader(di.ToString() + sqlfile);
                            System.IO.StreamWriter sw = new System.IO.StreamWriter(di.ToString() + sqlfile);

                            while ((line = cutline.ReadLine()) != null)
                            {
                                if ((lineNumber + 1).Equals(lineNumber))
                                {
                                   // 
                                }
                                else
                                {
                                   // sw.WriteLine(line + Environment.NewLine);

                                }
                                lineNumber++;
                            }

                            file.Close();

                            
                        }
                    }
                    catch(Exception ex)
                    {
                        iWrote = false;
                    }
                }

Open in new window

LVL 6
r3nderAsked:
Who is Participating?

[Product update] Infrastructure Analysis Tool is now available with Business Accounts.Learn More

x
I wear a lot of hats...

"The solutions and answers provided on Experts Exchange have been extremely helpful to me over the last few years. I wear a lot of hats - Developer, Database Administrator, Help Desk, etc., so I know a lot of things but not a lot about one thing. Experts Exchange gives me answers from people who do know a lot about one thing, in a easy to use platform." -Todd S.

Duy PhamFreelance IT ConsultantCommented:
I think the most efficient way to do that is write remaining lines those haven't been processed into a new file, and then rename new file over the old one after reading through all the lines (and properly close the stream for reading old file).

However, if your text file is not that big, I suggest you do read all the lines from text file into memory, process them and save back the lines those haven't been processed. Sample code could be:
string[] allLines = File.ReadAllLines(Path.Combine(Directory.GetCurrentDirectory(), "sqlfile.txt"));
for (int i = allLines.Length - 1; i >= 0; i--)
{
     // TODO: process line number i here
     iWrote = ss.UploadBackLog(sqlStr);
     if (iWrote)
     {
           allLines.RemoteAt(i);
     }
}

// finally write remaining lines back to the file
File.WriteAllLines(Path.Combine(Directory.GetCurrentDirectory(), "sqlfile.txt"), allLines);

Open in new window

0
r3nderAuthor Commented:
thanks Duy
I need to process them 1 at a time because the file has the potential to be large - I have done this but can only get it to proccess 1 line  - how can I get it to process all lines
        private void processSQL(int mycount)
        {
            if (mycount > 0)
            {
                string line = string.Empty;
                string newline = string.Empty;
                string sqlStr = string.Empty;
                string sqlfile = "\\sqlfile.txt";
                string sqlfile1 = "\\sqlfilenew.txt";
                bool iWrote = false;
                DirectoryInfo di = new DirectoryInfo(Directory.GetCurrentDirectory());
                using (var newfile = File.Create(di.ToString() + sqlfile1))
                {
                    //only used this to close the connection to the file after it was created
                }

                System.IO.StreamReader file = new System.IO.StreamReader(di.ToString() + sqlfile);

                while ((sqlStr = file.ReadLine()) != null)
                {
                    try
                    {
                        iWrote = ss.UploadBackLog(sqlStr);
                        file.Close();
                        if (iWrote)
                        {
                            System.IO.StreamReader cutline = new System.IO.StreamReader(di.ToString() + sqlfile);
                            System.IO.StreamWriter sw = new System.IO.StreamWriter(di.ToString() + sqlfile1);
                            while ((line = cutline.ReadLine()) != null)
                            {
                                if (line != sqlStr)
                                {
                                    sw.WriteLine(line);
                                }

                            }
                            cutline.Close();
                            File.Delete(di.ToString() + sqlfile);
                            sw.Flush();
                            sw.Close();
                            File.Move(di.ToString() + sqlfile1, "sqlfile.txt");
                        }
                    }
                    catch (Exception ex)
                    {
                        iWrote = false;
                    }
                }
                file.Close();
            }
        }

Open in new window

0
r3nderAuthor Commented:
its like it goes through the while loop only once
0
Duy PhamFreelance IT ConsultantCommented:
In your above code, you did close the old file right after the first line is processed (line 24). It's not necessary, so remove that line.

Secondly, you should not rename the new file to old file each time you process each line, do it after processing all the lines. With that, you only need to delete old file and rename new file once, and it gives better performance.

Finally, you only need to write the lines which could not be processed to new file without needing any extra validation like if (line != sqlStr). Simply write line to new file when iWrote is false.

So a sample code could be:
        private void processSQL(int mycount)
        {
            if (mycount > 0)
            {
                string sqlStr = string.Empty;
                string sqlfile = "\\sqlfile.txt";
                string sqlfile1 = "\\sqlfilenew.txt";
                bool iWrote = false;
                DirectoryInfo di = new DirectoryInfo(Directory.GetCurrentDirectory());
                using (var sw = System.IO.StreamWriter(di.ToString() + sqlfile1))
                {
                    using (var file = new System.IO.StreamReader(di.ToString() + sqlfile))
                    {
                        while ((sqlStr = file.ReadLine()) != null)
                        {
                            try
                            {
                                iWrote = ss.UploadBackLog(sqlStr);
                            }
                            catch (Exception ex)
                            {
                                iWrote = false;
                            }
                                
                            if (!iWrote)
                            {
                                sw.WriteLine(sqlStr);
                            }
                        }
                    } // end using StreamReader
                } // end using StreamWriter

                // rename new file over the old file
                File.Delete(di.ToString() + sqlfile);

                // better sleep for a while to make sure old file is really deleted, could use a while loop if needed
                System.Threading.Thread.Sleep(2000); // sleep 2 seconds

                File.Move(di.ToString() + sqlfile1, di.ToString() + sqlfile);
            } // end if
        } // end void

Open in new window


But if your file is too large to load all lines into memory, removing processed lines from the file might not be a good idea. You might want to try to create a state file to store indices of the lines those have already been processed, and then ignore them when re-read through the file again. With this way, you don't have to spend effort on writing not-processed lines to new file and then renaming it to the old one.
Sample code:
        private void processSQL(int mycount)
        {
            if (mycount > 0)
            {
                int lineCounter = 0;
                string sqlStr = string.Empty;
                string sqlfile = "\\sqlfile.txt";
                string stateFile = "\\sqlfile_state.txt";
                List<int> processedLineNumbers = new List<int>();
                bool iWrote = false;
                DirectoryInfo di = new DirectoryInfo(Directory.GetCurrentDirectory());

                // first try to load state from last processing
                if (File.Exists(di.ToString() + stateFile))
                {
                    string[] stateLines = File.ReadAllLines(di.ToString() + stateFile);
                    if (stateLines != null && stateLines.Length > 0)
                    {
                        stateLines = stateLines[0].Split(',');
                        if (stateLines != null && stateLines.Length > 0)
                        {
                            foreach (string lineIndex in stateLines)
                            {
                                processedLineNumbers.Add(Convert.ToInt32(lineIndex));
                            }
                        }
                    }
                }

                using (var file = new System.IO.StreamReader(di.ToString() + sqlfile))
                {
                    while ((sqlStr = file.ReadLine()) != null)
                    {
                        if (!processedLineNumbers.Contains(lineCounter))
                        {
                            try
                            {
                                iWrote = ss.UploadBackLog(sqlStr);
                            }
                            catch (Exception ex)
                            {
                                iWrote = false;
                            }
                                
                            if (iWrote)
                            {
                                processedLineNumbers.Add(lineCounter);
                            }
                        } // end if lineCounter check

                        // next line
                        lineCounter++;
                    } // end while
                } // end using StreamReader

                // write processed Line Numbers to stateFile
                File.WriteAllLines(di.ToString() + stateFile, new string[] { string.Join(",", processedLineNumbers) });
            } // end if
        } // end void

Open in new window

0

Experts Exchange Solution brought to you by

Your issues matter to us.

Facing a tech roadblock? Get the help and guidance you need from experienced professionals who care. Ask your question anytime, anywhere, with no hassle.

Start your 7-day free trial
r3nderAuthor Commented:
awesome Duy Thank you for the help
0
It's more than this solution.Get answers and train to solve all your tech problems - anytime, anywhere.Try it for free Edge Out The Competitionfor your dream job with proven skills and certifications.Get started today Stand Outas the employee with proven skills.Start learning today for free Move Your Career Forwardwith certification training in the latest technologies.Start your trial today
C#

From novice to tech pro — start learning today.