Solved

MS SQL json query/where clause nested array items

Posted on 2016-11-17
2
89 Views
Last Modified: 2017-01-05
I have json data that i can query on using "CROSS APPLY OPENJSON(" which gets slow once you start adding multiple cross applies or once your json document get too large. So i wanted to add an index on the data im trying to filter on, but i cant get the syntax on nested array items to work with out using a cross apply. As such i cant create an index as you cant use a cross apply when making an index. According to the MS docs i should just be able to do

 JSON_query(my_column, $.parentItem.nestedItemsArray1.nestedItemsArray2)

Open in new window

 

I should be able to get all the values of the nested, array items to then query on and improve performance by adding an index, something like this

  ALTER TABLE mytable 
    ADD vdata AS JSON_query(my_column, 
      $.parentItem.nestedItemsArray1.nestedItemsArray2')  
      
    CREATE INDEX idx_json_my_column ON mytable(vdata)

Open in new window

 

but the above  $.array.arrayitems syntax doesn't work ?
On a side note, I cant help but think in relational terms where normally in Sql you would index a column of data like so

 col
    ---
     1| 
     2|
     3|

Open in new window


 But json data seem to get flattened so when i use JSON_QUERY as per MS example i get "1,2,3" " I assume i want to incdex an array of values rather than a flattened version unless the index will return the inner data of the fattened data ?

my plug and play working example

  declare  @mydata table (    
      ID int NOT NULL,  
      jsondata varchar(max) NOT NULL  
    )
    
    INSERT INTO @mydata (id, jsondata)
    	VALUES (789, '{ "Id": "12345", "FinanceProductResults": [ { "Term": 12, "AnnualMileage": 5000, "Deposits": 0, "ProductResults": [] }, { "Term": 18, "AnnualMileage": 30000, "Deposits": 15000, "ProductResults": [] }, { "Term": 24, "AnnualMileage": 5000, "Deposits": 0, "ProductResults": [ { "Key": "HP", "Payment": 460.28 } ] }, { "Term": 24, "AnnualMileage": 10000, "Deposits": 0, "ProductResults": [ { "Key": "HP", "Payment": 500.32 } ] }]}')
    
    SELECT 
    	j_Id
    	,JSON_query (c.value, '$.Term') as Term
    	,JSON_Value (c.value, '$.AnnualMileage') as AnnualMileage
    	,JSON_Value (c.value, '$.Deposits') as Deposits 	
    	,JSON_Value (p.value, '$.Key') as [Key]
    	,JSON_Value (p.value, '$.Payment') as Payment
    	--,c.value
    FROM @mydata  f 
    	CROSS APPLY OPENJSON(f.jsondata)
    		WITH (j_Id nvarchar(100) '$.Id')
    	CROSS APPLY OPENJSON(f.jsondata, '$.FinanceProductResults') AS c
    	CROSS APPLY OPENJSON(c.value, '$."ProductResults"') AS p
    where 
    	ID = 789 
    	AND JSON_Value (p.value, '$.Payment') = '460.28'

Open in new window


Im using these MS docs to guide me :

How to create an index
How to get data

Update

I was able to improve performance slightly using the "with" method

    
SELECT 
    	j_Id,
		FinanceDetails.Term,
		FinanceDetails.AnnualMileage,
		FinanceDetails.Deposits,
		Payments.Payment
    FROM @mydata  f 
    	CROSS APPLY OPENJSON(f.jsondata)
    		WITH (j_Id nvarchar(100) '$.Id')
		OUTER APPLY OPENJSON (f.jsondata, '$.FinanceProductResults' )  
				WITH (   
					Term			INT				'$.Term',  
					AnnualMileage	INT				'$.AnnualMileage',  
					Deposits		INT				'$.Deposits',  
					ProductResults  NVARCHAR(MAX)	'$.ProductResults'  AS JSON  
				) AS FinanceDetails
			OUTER APPLY OPENJSON(ProductResults,  '$') 
				WITH (   
					Payment DECIMAL(19, 4)			'$.Payment' 
				) AS Payments
	WHERE	
		Payments.Payment = 460.28

Open in new window


but i still like to add an index on the sub array data to aid in improving performance ?
0
Comment
Question by:Rawdon Hume
  • 2
2 Comments
 
LVL 2

Accepted Solution

by:
Rawdon Hume earned 0 total points
ID: 41903747
So it seems currently, you cannot index nested properties. so ill use Full-text search. I have to create FTS on JSON column and add predicate:

WHERE ....
AND CONTAINS( jsondata, 'NEAR(('Payments,460),1)')

Open in new window


Since JSON is text, this predicate will filter out all records that don't have something like "Payment" and 460 near to each other (this will identify key:value pairs), I can apply CROSS APPLY on the reduced set of rows.
0
 
LVL 2

Author Closing Comment

by:Rawdon Hume
ID: 41903748
why doesn't ms sql 2026 have better support json
0

Featured Post

The Eight Noble Truths of Backup and Recovery

How can IT departments tackle the challenges of a Big Data world? This white paper provides a roadmap to success and helps companies ensure that all their data is safe and secure, no matter if it resides on-premise with physical or virtual machines or in the cloud.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Let's review the features of new SQL Server 2012 (Denali CTP3). It listed as below: PERCENT_RANK(): PERCENT_RANK() function will returns the percentage value of rank of the values among its group. PERCENT_RANK() function value always in be…
Nothing in an HTTP request can be trusted, including HTTP headers and form data.  A form token is a tool that can be used to guard against request forgeries (CSRF).  This article shows an improved approach to form tokens, making it more difficult to…
Using examples as well as descriptions, and references to Books Online, show the documentation available for datatypes, explain the available data types and show how data can be passed into and out of variables.
Viewers will learn how the fundamental information of how to create a table.

813 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

16 Experts available now in Live!

Get 1:1 Help Now