Solved

MS SQL json query/where clause nested array items

Posted on 2016-11-17
2
164 Views
Last Modified: 2017-01-05
I have json data that i can query on using "CROSS APPLY OPENJSON(" which gets slow once you start adding multiple cross applies or once your json document get too large. So i wanted to add an index on the data im trying to filter on, but i cant get the syntax on nested array items to work with out using a cross apply. As such i cant create an index as you cant use a cross apply when making an index. According to the MS docs i should just be able to do

 JSON_query(my_column, $.parentItem.nestedItemsArray1.nestedItemsArray2)

Open in new window

 

I should be able to get all the values of the nested, array items to then query on and improve performance by adding an index, something like this

  ALTER TABLE mytable 
    ADD vdata AS JSON_query(my_column, 
      $.parentItem.nestedItemsArray1.nestedItemsArray2')  
      
    CREATE INDEX idx_json_my_column ON mytable(vdata)

Open in new window

 

but the above  $.array.arrayitems syntax doesn't work ?
On a side note, I cant help but think in relational terms where normally in Sql you would index a column of data like so

 col
    ---
     1| 
     2|
     3|

Open in new window


 But json data seem to get flattened so when i use JSON_QUERY as per MS example i get "1,2,3" " I assume i want to incdex an array of values rather than a flattened version unless the index will return the inner data of the fattened data ?

my plug and play working example

  declare  @mydata table (    
      ID int NOT NULL,  
      jsondata varchar(max) NOT NULL  
    )
    
    INSERT INTO @mydata (id, jsondata)
    	VALUES (789, '{ "Id": "12345", "FinanceProductResults": [ { "Term": 12, "AnnualMileage": 5000, "Deposits": 0, "ProductResults": [] }, { "Term": 18, "AnnualMileage": 30000, "Deposits": 15000, "ProductResults": [] }, { "Term": 24, "AnnualMileage": 5000, "Deposits": 0, "ProductResults": [ { "Key": "HP", "Payment": 460.28 } ] }, { "Term": 24, "AnnualMileage": 10000, "Deposits": 0, "ProductResults": [ { "Key": "HP", "Payment": 500.32 } ] }]}')
    
    SELECT 
    	j_Id
    	,JSON_query (c.value, '$.Term') as Term
    	,JSON_Value (c.value, '$.AnnualMileage') as AnnualMileage
    	,JSON_Value (c.value, '$.Deposits') as Deposits 	
    	,JSON_Value (p.value, '$.Key') as [Key]
    	,JSON_Value (p.value, '$.Payment') as Payment
    	--,c.value
    FROM @mydata  f 
    	CROSS APPLY OPENJSON(f.jsondata)
    		WITH (j_Id nvarchar(100) '$.Id')
    	CROSS APPLY OPENJSON(f.jsondata, '$.FinanceProductResults') AS c
    	CROSS APPLY OPENJSON(c.value, '$."ProductResults"') AS p
    where 
    	ID = 789 
    	AND JSON_Value (p.value, '$.Payment') = '460.28'

Open in new window


Im using these MS docs to guide me :

How to create an index
How to get data

Update

I was able to improve performance slightly using the "with" method

    
SELECT 
    	j_Id,
		FinanceDetails.Term,
		FinanceDetails.AnnualMileage,
		FinanceDetails.Deposits,
		Payments.Payment
    FROM @mydata  f 
    	CROSS APPLY OPENJSON(f.jsondata)
    		WITH (j_Id nvarchar(100) '$.Id')
		OUTER APPLY OPENJSON (f.jsondata, '$.FinanceProductResults' )  
				WITH (   
					Term			INT				'$.Term',  
					AnnualMileage	INT				'$.AnnualMileage',  
					Deposits		INT				'$.Deposits',  
					ProductResults  NVARCHAR(MAX)	'$.ProductResults'  AS JSON  
				) AS FinanceDetails
			OUTER APPLY OPENJSON(ProductResults,  '$') 
				WITH (   
					Payment DECIMAL(19, 4)			'$.Payment' 
				) AS Payments
	WHERE	
		Payments.Payment = 460.28

Open in new window


but i still like to add an index on the sub array data to aid in improving performance ?
0
Comment
Question by:Rawdon Hume
[X]
Welcome to Experts Exchange

Add your voice to the tech community where 5M+ people just like you are talking about what matters.

  • Help others & share knowledge
  • Earn cash & points
  • Learn & ask questions
  • 2
2 Comments
 
LVL 2

Accepted Solution

by:
Rawdon Hume earned 0 total points
ID: 41903747
So it seems currently, you cannot index nested properties. so ill use Full-text search. I have to create FTS on JSON column and add predicate:

WHERE ....
AND CONTAINS( jsondata, 'NEAR(('Payments,460),1)')

Open in new window


Since JSON is text, this predicate will filter out all records that don't have something like "Payment" and 460 near to each other (this will identify key:value pairs), I can apply CROSS APPLY on the reduced set of rows.
0
 
LVL 2

Author Closing Comment

by:Rawdon Hume
ID: 41903748
why doesn't ms sql 2026 have better support json
0

Featured Post

Free Tool: Path Explorer

An intuitive utility to help find the CSS path to UI elements on a webpage. These paths are used frequently in a variety of front-end development and QA automation tasks.

One of a set of tools we're offering as a way of saying thank you for being a part of the community.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

We will take a look at the d3.js library for visualizations.  I will provide a walkthrough of a short bar graph example as well as introduce you to the d3.js API.  We also will explore links of other examples and further information regarding SVG's.
I have a large data set and a SSIS package. How can I load this file in multi threading?
This video shows, step by step, how to configure Oracle Heterogeneous Services via the Generic Gateway Agent in order to make a connection from an Oracle session and access a remote SQL Server database table.
Via a live example, show how to shrink a transaction log file down to a reasonable size.

707 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question