Parsing nested json

Martin Miller
Martin Miller used Ask the Experts™
on
Parsing nested json. I am looking for help to parsing the nested json elements.

I attached a json file I need to output all of the data fields, however some are nested and I don't know the HOW-TO output the nested values. Can you help ?

I also include the start a python script
parser.py
file.json
Comment
Watch Question

Do more with

Expert Office
EXPERT OFFICE® is a registered trademark of EXPERTS EXCHANGE®
David FavorFractional CTO
Distinguished Expert 2018

Commented:
Must be valid JSON to start.

Nested JSON works, so long as the JSON validates.

https://jsonformatter.org/ is a great validator tool which also formats JSON into human readable form.

Reformat your JSON to pass validation + likely all will work well.

Tip: Be sure to check the return result of json.loads() + bail out with an error in your code, to know if the JSON parse has succeeded.

Author

Commented:
@David thank's for the pointer to the validator, I am able to output the elements shown in the script attached, it's the nested elements, that I am not sure how-to do this in python, and ideas ?
Mark BradyPrincipal Data Engineer

Commented:
The main issue as has been stated is the JSON is NOT a JSON. It is a good attempt to be one but not quite. This particular Json has a lot of issues starting on line 3.
"nonPersonalizedCardLine3" :
Error: Parse error on line 3:
...nalizedCardLine3" : ValuedCustomer,"pro
-----------------------^
Expecting 'STRING', 'NUMBER', 'NULL', 'TRUE', 'FALSE', '{', '['
ValuedCustomer,

What that means is the text ValuedCustomer needs to be double quoted:  "ValuedCustomer",

To be valid all values should be one of the following:

STRING, NUMBER, NULL, true, false, { or [

Once you go ahead and fix up the example file to be a proper Json then parsing it will be super easy!
OWASP: Threats Fundamentals

Learn the top ten threats that are present in modern web-application development and how to protect your business from them.

David FavorFractional CTO
Distinguished Expert 2018

Commented:
Mark... restated my point better than I did... "The main issue as has been stated is the JSON is NOT a JSON."

Your JSON must be JSON (validates clean) before your JSON will parse.

You said, "it's the nested elements, that I am not sure how-to do this in python, and ideas?"

Yes. Simple.

1) First you can't fix this... ever... any way... in your parser.py script... well... unless you write some hand rolled (custom) JSON parser which attempts to fix broken JSON.

You'll take this option if you're independently wealthy - unlimited time + money.

2) Better to fix this in the code generating the JSON.

3) To fix your JSON code generator, first output the output the outer block of your JSON (no nested elements) + ensure your JSON validates, then do the same for every individual JSON element, then merge the both outer block + nested element generation + you should be good.

Author

Commented:
I have uploaded a valid json, my question is still the same... how to parse the nested elements. Thanks!!!
Mark BradyPrincipal Data Engineer

Commented:
Can you make the new valid JSON available to us? In order to give you an example of how to parse it we need to see what we are working with.
Looking at the original JSON posted in this question the key of "productLocale" has a value which is a list of objects - or a list of dicts when it is json.loads(). To iter through each key => value in the nested objects:

for key, val in obj['productLocale'].iter():
    print('{KEY} => {VAL}'.format(KEY = key, VAL = val)

If you are not sure if the values are objects themselves you could do an isinstance check on each value before parsing it. If it is an instance of a dict or list then you know to iter the values. If not then that the values like a string (or whatever type of data is in there).

If you post the fixed json and tell us exactly what you expect to get out of it or your purpose for reading the data then we can help further.
David FavorFractional CTO
Distinguished Expert 2018

Commented:
You said, "I have uploaded a valid json, my question is still the same... how to parse the nested elements. Thanks!!!"

Once your JSON is valid, then your parser.py file will work as it's written, so valid JSON will parse correctly.

Maybe what you're asking is how you access these elements in your parsed data.

If this is what you're asking, then refer to Mark's comment.

Sometimes the easiest way to figure this out is to run your data through JSFormatter above. Once you have pretty (human readable) output, you can work out your exact parsing based on Mark's comment.

Author

Commented:
@Mark, Using this version of the script, I pasted output.
========================
import json
obj = json.loads(open('./file.json').read())
print("stopApprovalInd: " + str(obj['stopApprovalInd']))
print("creatorId: " + str(obj['creatorId']))
print("productBarcodes: " + str(obj['productBarcodes']))
print("\n")

Open in new window


stopApprovalInd: True
creatorId: PDM4PDZ0PDNQBGWMVGMRVTK9DM
productBarcodes: [{u'barcodeType': u'3D-CA128'}]

If I just want to output barcodeType, how do I resolve to just that ?

thanks again!!!

Martin
David FavorFractional CTO
Distinguished Expert 2018

Commented:
Best to post a copy of your valid JSON + likely someone can give you the exact syntax required to access data you're after.

Author

Commented:
I updated the attached json file  2 days ago. it is valid, looking for an expert on the extract of elements that are embedded. These don't seem to fit as an item in an array to iter() on.  

Repeat the simple case here.
productBarcodes: [{u'barcodeType': u'3D-CA128'}]

Open in new window


If I just want to output barcodeType, how do I resolve to just that ?

Thanks!
Principal Data Engineer
Commented:
productBarcodes is a list of dicts. Depending on if you just want the value of barcodeType or you want to loop through multiple values in that array will be how to do it. To get the individual value:
print(obj['productBarcodes'][0]['barcodeType'])

To loop multiple dicts:

for this_obj in obj['productBarcodes']:
    print(this_obj['barcodeType']) + '\n'

Author

Commented:
Thanks Mark, Very much appreciated!
Mark BradyPrincipal Data Engineer

Commented:
Anytime!

Do more with

Expert Office
Submit tech questions to Ask the Experts™ at any time to receive solutions, advice, and new ideas from leading industry professionals.

Start 7-Day Free Trial