Want to win a PS4? Go Premium and enter to win our High-Tech Treats giveaway. Enter to Win

x
?
Solved

Maximum Size for PHP Array

Posted on 2004-10-22
12
Medium Priority
?
1,895 Views
Last Modified: 2008-02-01
Is there a maximum size that a PHP array can be?  I purchased a product to parse Excel files recently.  This product reads the whole excel file into a PHP array.  However, the excel file I am trying to parse has 18,000 rows and about 20 columns.  For some reason it just crashes in the middle of it.

I have increase the amount of memory for PHP and also the execution time so those shouldn't be a problem.

So I just want to know if there is a limit on the amount of indexes an array in PHP can have.

Thanks
0
Comment
Question by:leobaz2
[X]
Welcome to Experts Exchange

Add your voice to the tech community where 5M+ people just like you are talking about what matters.

  • Help others & share knowledge
  • Earn cash & points
  • Learn & ask questions
12 Comments
 
LVL 4

Expert Comment

by:KarlPurkhardt
ID: 12381304
I dont know how much you can store in a php array but why would you need to store that much data anyway?  Its not very practical to store large amounts of data in an array (not that large anyway), if the data is being stored to be processed, then i'd rethink the way in which you process the data, maybe prcoess it as its being retrieved from the spreadsheet, rather than storing it in an array and then processing it.
0
 
LVL 33

Expert Comment

by:snoyes_jw
ID: 12381579
I've used arrays with 60000 rows, so that's not the problem.  Odds are it's a memory issue.  Can you figure out the largest file you can use without a crash?

As Karl says, you may want to find some way to read and parse the file a few lines at a time.
0
 

Author Comment

by:leobaz2
ID: 12382193
Well, I what I did was I kept deleting rows until I found the limit as to when the parsing worked.  I was able to find that actual character that was making it crash.  If I remove the last character, it works, if I put a character in it fails.  Therefore, I was assuming it was a memory problem.  But then I doubled the memory with the character in there and it still failed.

Is there anything special I have to do to have the memory update to be used?  I restarted apache after changing php.ini.  I also made sure that the php was configured with the "enable-memory-limit" option.
0
Technology Partners: We Want Your Opinion!

We value your feedback.

Take our survey and automatically be enter to win anyone of the following:
Yeti Cooler, Amazon eGift Card, and Movie eGift Card!

 
LVL 33

Expert Comment

by:snoyes_jw
ID: 12382220
Interesting.  What's the maximum file size that works?

You also might contact whoever sold the product to you and see what they say.
0
 

Author Comment

by:leobaz2
ID: 12382274
I am not sure what the max file size is now since I am at work and my other stuff is at home.  I believe it breaks with around a 6-8 MB file and works with a 5MB and lower.

The company I bought this product from was of no help at all.  I opened 2 issues with them and they have no idea and gave up.  Now they are ignoring me.  They mentioned that Microsoft "might" change the way they save the Excel sheet for files larger than a certain size.

The problem is that this Excel parser doesn't use COM since it is supported on Unix platforms also.  So this is pure binary processing which makes it much harder to debug and fix.

0
 
LVL 33

Accepted Solution

by:
snoyes_jw earned 200 total points
ID: 12382872
I create an 18000 x 20 Excel sheet of random 15-digit numbers.  File size was 6,955,520 bytes.  I deleted a row and file size jumped to 6,958,592 - an increase of 3072 bytes.  I deleted 8 more rows and the size went down by either 0 or 512 bytes for each line.  So it's possible that at either 18000 rows or somewhere around 6.6MB, they change something.

I couldn't find any documentation from MS regarding such a change, but a Google search did reveal several people having similar problems with files of a similar size.
0
 

Author Comment

by:leobaz2
ID: 12383228
Thanks a lot for the invenstigation.  I will look more at this tonight to see what is going on.
0
 
LVL 18

Expert Comment

by:arantius
ID: 12396346
@snoyes

That's probably "fast save" working against you.  MS has the bright idea of not changing the file when you change it, but rather keeping it and appending the changes on.  Which means it reconstructs it all when the file is loaded.

@leobaz
You said when you removed a single character that it could process it, even when the file was much larger but without that character?  Which character?
And is there any off chance you could convince your users to just "Save As..." a CSV file?  That would probably solve your headaches.
0
 
LVL 33

Expert Comment

by:snoyes_jw
ID: 12397346
arantius:  I don't doubt your superior knowledge of Excel, but if the "fast save" is the case, I wonder why it gets smaller as I delete more lines?  It's only the first deletion where it gets bigger, then progressively smaller as I delete more lines from the same file.
0

Featured Post

Free Tool: Port Scanner

Check which ports are open to the outside world. Helps make sure that your firewall rules are working as intended.

One of a set of tools we are providing to everyone as a way of saying thank you for being a part of the community.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Foreword (July, 2015) Since I first wrote this article, years ago, a great many more people have begun using the internet.  They are coming online from every part of the globe, learning, reading, shopping and spending money at an ever-increasing ra…
There are times when I have encountered the need to decompress a response from a PHP request. This is how it's done, but you must have control of the request and you can set the Accept-Encoding header.
The viewer will learn how to dynamically set the form action using jQuery.
The viewer will learn how to look for a specific file type in a local or remote server directory using PHP.
Suggested Courses

636 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question