• Status: Solved
  • Priority: Medium
  • Security: Public
  • Views: 2346
  • Last Modified:

processing very big json file with php - json_decode()

I'm getting a memory exhausted error on json_decode(), how to resolve this ?
1 Solution
Christian VivasprogrammerCommented:
have you tried with memory_limit? upgrade the memory_max_limit in your php.ini to more than 128 lets say 512
snigerAuthor Commented:
other solutions ?
You've got the solution above - increase PHP's memory
Keep up with what's happening at Experts Exchange!

Sign up to receive Decoded, a new monthly digest with product updates, feature release info, continuing education opportunities, and more.

Ray PaseurCommented:
Please post a URL that gives us a link to the JSON string.   Then we can experiment with it and try to offer some assistance.  It may be as simple as increasing the memory limit, but it's probably going to require a surprisingly large limit.  PHP arrays and objects can run 20X the size of their string equivalents.  So if the JSON string is 4MB, you may be looking at 80MB just to hold the object of the decoded string.  Add to that any arrays or objects that are extracted in the course of processing the JSON object and you can see how this can domino into a huge memory requirement.

FWIW, JSON is a more economical transport mechanism than XML, and has mostly replaced XML in modern applications.

Once we can see the JSON string, if there is anything that can be done with memory limit, we can show you the requirement.  And if there is not any way to get an adequate memory limit, we may be able to suggest some workarounds.
snigerAuthor Commented:
what if 512 or 1024 is still too small ? How can I know the maximum size of a file which I'm going to get from external sources ?
snigerAuthor Commented:

It is some simple key: value, array:
 {key1: val1, key2:val3, key3:val3,........},
  {key1: val1a, key2:val3a, key3:val3a,........},
  {key1: val1b, key2:val3b, key3:val3b,........}
Ray PaseurCommented:
Hi, @sniger.  The key to this issue is the size of the JSON string, the number of nodes, etc.  That's why we need to see the test data set -- one that replicates the likely size of the real JSON strings.  We can't really answer the "what if" questions without seeing some test data.  It's rare that a JSON string is too large for processing with PHP, and since we're dealing with a rare thing here, there may not be a lot of preexisting solutions.
How can I know the maximum size of a file...
In a nutshell, you can't but with some test data we may be able to give you some useful strategies (I hope).
Question has a verified solution.

Are you are experiencing a similar issue? Get a personalized answer when you ask a related question.

Have a better answer? Share it in a comment.

Join & Write a Comment

Featured Post

Get expert help—faster!

Need expert help—fast? Use the Help Bell for personalized assistance getting answers to your important questions.

Tackle projects and never again get stuck behind a technical roadblock.
Join Now