I have a very large (1gb+) text file containing 3 columns. In the 2nd column I have a date in the format dd/mm/yy.
I need to split the file up based on the date field i.e. create a txt file for each group of dates.
I thought one way to do this would be using the streamreader object to read each line and compare it with the one before to see if it is the same. Having a look at my attempt below.
I'm not sure how to read the line before. Not even sure if this is the best way to do.
My C# skill is very basic as you can tell. I would appreciate any direction.
static void Main(string args)
using (StreamReader sr = new StreamReader("C:\\TEMP\\split\\bigfile.txt"))
// Read and display lines from the file until the end of
// the file is reached.
while ((line = sr.ReadLine()) != null)
string words = line.Split('|');
dat_first = DateTime.Parse(words);
// Need dat_second from the next line and check if the the date is the same as dat_first then
// create a new text file containing this row and subsequent rows whilst the dates are the same.
// only when the dates differ should we create a new file. files titled with the dat_first date.
catch (Exception e)
// Let the user know what went wrong.
Console.WriteLine("The file could not be read:");