Solved

Maximum Size for PHP Array

Posted on 2004-10-22
1,856 Views
Last Modified: 2008-02-01
Is there a maximum size that a PHP array can be?  I purchased a product to parse Excel files recently.  This product reads the whole excel file into a PHP array.  However, the excel file I am trying to parse has 18,000 rows and about 20 columns.  For some reason it just crashes in the middle of it.

I have increase the amount of memory for PHP and also the execution time so those shouldn't be a problem.

So I just want to know if there is a limit on the amount of indexes an array in PHP can have.

Thanks
0
Question by:leobaz2
    9 Comments
     
    LVL 4

    Expert Comment

    by:KarlPurkhardt
    I dont know how much you can store in a php array but why would you need to store that much data anyway?  Its not very practical to store large amounts of data in an array (not that large anyway), if the data is being stored to be processed, then i'd rethink the way in which you process the data, maybe prcoess it as its being retrieved from the spreadsheet, rather than storing it in an array and then processing it.
    0
     
    LVL 33

    Expert Comment

    by:snoyes_jw
    I've used arrays with 60000 rows, so that's not the problem.  Odds are it's a memory issue.  Can you figure out the largest file you can use without a crash?

    As Karl says, you may want to find some way to read and parse the file a few lines at a time.
    0
     

    Author Comment

    by:leobaz2
    Well, I what I did was I kept deleting rows until I found the limit as to when the parsing worked.  I was able to find that actual character that was making it crash.  If I remove the last character, it works, if I put a character in it fails.  Therefore, I was assuming it was a memory problem.  But then I doubled the memory with the character in there and it still failed.

    Is there anything special I have to do to have the memory update to be used?  I restarted apache after changing php.ini.  I also made sure that the php was configured with the "enable-memory-limit" option.
    0
     
    LVL 33

    Expert Comment

    by:snoyes_jw
    Interesting.  What's the maximum file size that works?

    You also might contact whoever sold the product to you and see what they say.
    0
     

    Author Comment

    by:leobaz2
    I am not sure what the max file size is now since I am at work and my other stuff is at home.  I believe it breaks with around a 6-8 MB file and works with a 5MB and lower.

    The company I bought this product from was of no help at all.  I opened 2 issues with them and they have no idea and gave up.  Now they are ignoring me.  They mentioned that Microsoft "might" change the way they save the Excel sheet for files larger than a certain size.

    The problem is that this Excel parser doesn't use COM since it is supported on Unix platforms also.  So this is pure binary processing which makes it much harder to debug and fix.

    0
     
    LVL 33

    Accepted Solution

    by:
    I create an 18000 x 20 Excel sheet of random 15-digit numbers.  File size was 6,955,520 bytes.  I deleted a row and file size jumped to 6,958,592 - an increase of 3072 bytes.  I deleted 8 more rows and the size went down by either 0 or 512 bytes for each line.  So it's possible that at either 18000 rows or somewhere around 6.6MB, they change something.

    I couldn't find any documentation from MS regarding such a change, but a Google search did reveal several people having similar problems with files of a similar size.
    0
     

    Author Comment

    by:leobaz2
    Thanks a lot for the invenstigation.  I will look more at this tonight to see what is going on.
    0
     
    LVL 18

    Expert Comment

    by:arantius
    @snoyes

    That's probably "fast save" working against you.  MS has the bright idea of not changing the file when you change it, but rather keeping it and appending the changes on.  Which means it reconstructs it all when the file is loaded.

    @leobaz
    You said when you removed a single character that it could process it, even when the file was much larger but without that character?  Which character?
    And is there any off chance you could convince your users to just "Save As..." a CSV file?  That would probably solve your headaches.
    0
     
    LVL 33

    Expert Comment

    by:snoyes_jw
    arantius:  I don't doubt your superior knowledge of Excel, but if the "fast save" is the case, I wonder why it gets smaller as I delete more lines?  It's only the first deletion where it gets bigger, then progressively smaller as I delete more lines from the same file.
    0

    Write Comment

    Please enter a first name

    Please enter a last name

    We will never share this with anyone.

    Featured Post

    Find Ransomware Secrets With All-Source Analysis

    Ransomware has become a major concern for organizations; its prevalence has grown due to past successes achieved by threat actors. While each ransomware variant is different, we’ve seen some common tactics and trends used among the authors of the malware.

    As you all know PHP got one problem, before using other files classes or functions you need to include or use require function. But now in PHP 5 there's solution for this, it's called __autoload() function. You can read more about function in PHP ma…
    Generating table dynamically is the most common issue faced by php developers.... So it seems there is a need of an article that explains the basic concept of generating tables dynamically. It just requires a basic knowledge of html and little maths…
    The viewer will learn how to count occurrences of each item in an array.
    The viewer will learn how to create a basic form using some HTML5 and PHP for later processing. Set up your basic HTML file. Open your form tag and set the method and action attributes.: (CODE) Set up your first few inputs one for the name and …

    856 members asked questions and received personalized solutions in the past 7 days.

    Join the community of 500,000 technology professionals and ask your questions.

    Join & Ask a Question

    Need Help in Real-Time?

    Connect with top rated Experts

    18 Experts available now in Live!

    Get 1:1 Help Now