C# Read Large ASCII-based Data Files one row at a time.

Posted on 2012-08-15
Last Modified: 2012-08-19
I have a sequential file, where a series of numbers, 1 per row, are about 250 digits long.

Each file has over 60,000 rows.  Some has over 120,000 rows.

Each row will look like this:
01234567889901397819837490187341908734190837419083741902837491087583458934  (this is a string).

What I need to do is read the file one row at a time, without overloading memory and processing power.  I will be processing the whole file at once.

I have the script to do with what it needs to do once I have it, but I need the best way on reading the file one row at a time without overloading memory.

I plan on splitting the row up into parts, then saving the parts into various tables and spreadsheets.

Any help would be greatly appreciated, thanks.
Question by:Evan Cutler
    LVL 85

    Accepted Solution

    Standard StreamReader will do it, allowing you to read only one line at a time:
                string line;
                string FileName = @"c:\some path\folder\file.txt";
                StreamReader SR = new StreamReader(FileName);
                while (!SR.EndOfStream)
                    line = SR.ReadLine();
                    // ... do something with "line" in here...

    Open in new window

    LVL 9

    Author Closing Comment

    by:Evan Cutler
    Thank you very much...
    this worked great.

    Write Comment

    Please enter a first name

    Please enter a last name

    We will never share this with anyone.

    Featured Post

    Looking for New Ways to Advertise?

    Engage with tech pros in our community with native advertising, as a Vendor Expert, and more.

    A long time ago (May 2011), I have written an article showing you how to create a DLL using Visual Studio 2005 to be hosted in SQL Server 2005. That was valid at that time and it is still valid if you are still using these versions. You can still re…
    Exception Handling is in the core of any application that is able to dignify its name. In this article, I'll guide you through the process of writing a DRY (Don't Repeat Yourself) Exception Handling mechanism, using Aspect Oriented Programming.
    An introduction to basic programming syntax in Java by creating a simple program. Viewers can follow the tutorial as they create their first class in Java. Definitions and explanations about each element are given to help prepare viewers for future …
    In this seventh video of the Xpdf series, we discuss and demonstrate the PDFfonts utility, which lists all the fonts used in a PDF file. It does this via a command line interface, making it suitable for use in programs, scripts, batch files — any pl…

    761 members asked questions and received personalized solutions in the past 7 days.

    Join the community of 500,000 technology professionals and ask your questions.

    Join & Ask a Question

    Need Help in Real-Time?

    Connect with top rated Experts

    8 Experts available now in Live!

    Get 1:1 Help Now