Is there a size limit for .csv files?

We have an application that is exporting 1,254,236 records to a .csv file.  The file size is 295,788.  Excel brings up an error msg that it was unable to load the full file.  I have a batch job that successfully splits the file into smaller ones which works ok.  However, when I try to reconcile the counts, it doesn't match the number exported by the application.

Is there a size limit for .csv files?

Thanks for your help.
Who is Participating?

[Product update] Infrastructure Analysis Tool is now available with Business Accounts.Learn More

I wear a lot of hats...

"The solutions and answers provided on Experts Exchange have been extremely helpful to me over the last few years. I wear a lot of hats - Developer, Database Administrator, Help Desk, etc., so I know a lot of things but not a lot about one thing. Experts Exchange gives me answers from people who do know a lot about one thing, in a easy to use platform." -Todd S.

JohnBusiness Consultant (Owner)Commented:
I do not think you are hitting a size limit, but a record limit. For Excel 2010, the number of records is limited to 1048576 rows and you exceed that.
There's no limit for the .csv file itself (other than the max file size of the OS), it's Excel who has to read the data. And therefore the limits of Excel come into play

Excel 2003 and before: max 65,536 rows
Excel 2007 and above: max 1,048,576 rows

I therefore think you might be using 2003 or earlier and the split files are over 65,536 rows, and you lose the amount over that, times the number of files you imported.
costanosNetwork EngineerCommented:
No, there is no hard size limit on .csv files.. The limit in any scenario would be the file system / hdd size.  However, the application used to open these files MIGHT give you a problem.  ie opening a 10gb .csv file in excel.
Amazon Web Services

Are you thinking about creating an Amazon Web Services account for your business? Not sure where to start? In this course you’ll get an overview of the history of AWS and take a tour of their user interface.

Jerry MillerCommented:
If you want to open csv files of a large size you can use a different program to open them. I use Notepad++. It is a free open source program that is very easy to use.
Tony GiangrecoCommented:
The CSV file has no limitation other than free disk space on your drive or an O/S file size limitation.

Excel has a record limit which you are hitting. If you split those files and try importing them into one Excel spreadsheet, you will still have the same problem. I suggest splitting the CSV file into multiple files that are under the Excel record limit providing you room to insert or add records to that spreadsheet without being close to the limit again.

I ran into that a few years ago. Hope this helps!

Experts Exchange Solution brought to you by

Your issues matter to us.

Facing a tech roadblock? Get the help and guidance you need from experienced professionals who care. Ask your question anytime, anywhere, with no hassle.

Start your 7-day free trial
JohnBusiness Consultant (Owner)Commented:
Using an arbitrary number of characters per record of 256, the file would be 321Mb in size. This is large for most systems, but if you need a text editor that will handle this size of file, I use UltraEdit. It has no size limit.

Otherwise, with Excel, you are limited by the Excel limit (1048576) as I noted much further back.
Tony GiangrecoCommented:
FYI, I've been using Ultra Edit for 15 years. It's the best text editor I've ever seen.
Tony GiangrecoCommented:
Glad I could help.
It's more than this solution.Get answers and train to solve all your tech problems - anytime, anywhere.Try it for free Edge Out The Competitionfor your dream job with proven skills and certifications.Get started today Stand Outas the employee with proven skills.Start learning today for free Move Your Career Forwardwith certification training in the latest technologies.Start your trial today
Operating Systems

From novice to tech pro — start learning today.