Load Large XML doc into Oracle using Bfilename

talahi
talahi used Ask the Experts™
on
My overall goal is to be able to search for specific data in this xlm document probably using XLMQUERY.  Using Oracle 12c, 64bit, windows 2012r2 server, trying to load a 50meg XLM document and verify data correctly loaded.

Using,

CREATE XML_TABLE
(
  XML_DATA  XMLTYPE
);

INSERT INTO xml_table
  VALUES (XMLType(bfilename('XML_DIR', '50Meg_doc.xml'),
                  nls_charset_id('AL32UTF8')));
commit;

to check data,

select XMLQuery('//Navaid/NavaidID' PASSING XML_DATA  RETURNING CONTENT)
from xml_table;

nothing returns.

Sample of data.

<?xml version="1.0" encoding="UTF-8"?>
<TfmsData xmlns="http://tfm.faa.gov/tfms/TfmsData" xmlns:nxce="http://tfm.faa.gov/tfms/NasXCoreElements">
 <GeospatialData>
<Navaid><NavaidID>ZVR</NavaidID><ArtccID>CZV</ArtccID><NavaidType>NDB</NavaidType><GeoPosition><nxce:latitude><nxce:latitudeRadians>.8585565537</nxce:latitudeRadians></nxce:latitude><nxce:longitude><nxce:longitudeRadians>-2.1505947044</nxce:longitudeRadians></nxce:longitude></GeoPosition><State>BC</State>...
Comment
Watch Question

Do more with

Expert Office
EXPERT OFFICE® is a registered trademark of EXPERTS EXCHANGE®
Most Valuable Expert 2012
Distinguished Expert 2018

Commented:
I think the issue is in the select not the load.

What does this return:

select dbms_lob.getlength(x.XML_DATA.getclobval()) from xml_table x;

Author

Commented:
returns 74694362
Most Valuable Expert 2012
Distinguished Expert 2018

Commented:
Then the XML loaded.

I believe the issue is with the namespaces in the XML.

I've never used XMLQuery so I'm trying to figure out how to use namespaces in it.

I'll post when I figure it out.

While we both look, what types of data will you be extracting?

I typically use XMLTABLE not XMLQUERY.

I have examples with XMLTABLE and multiple namespaces.
Ensure you’re charging the right price for your IT

Do you wonder if your IT business is truly profitable or if you should raise your prices? Learn how to calculate your overhead burden using our free interactive tool and use it to determine the right price for your IT services. Start calculating Now!

Author

Commented:
Whatever works, XMLTABLE or XMLQUERY.  I'm trying to find something with an equivalent WHERE clause. Here are some of the unsuccessful attempts I've made following examples.  To start with I'm just trying to do a simple query to understand how the syntax works.

select NavaidID
from xml_table,
      xmltable(
             xmlnamespaces(default 'http://xxx.xxx.xxx/xxx/xxx',
                  'http://xxx.xxx.xxx/xxx/xxx' as "nxce"
            ),
      '/TfmsData'
      passing xml_data
      columns
            Navaid xmltype path '/TfmsData/GeospatialData/Navaid'
      ) xml_data
      ,xmltable(
             xmlnamespaces(default 'http://xxx.xxx.xxx/xxx/xxx',
                  'http://xxx.xxx.xxx/xxx/xxx' as "nxce"
            ),
            '/Navaid'
            passing xml_data --xmltab.Navaid
            columns NavaidID varchar2(10) path '//Navaid/NavaidID'
      );
 
SELECT xml_data, XMLQuery (
'for $i IN /Navaid
 where $i /NavaidID = "ZVR"
 return $i /State'
PASSING BY VALUE xml_data
RETURNING CONTENT) As output --xmldata
FROM xml_table;

ORA-19114: XPST0003 - error during parsing the XQuery expression:
LPX-00801: XQuery syntax error at 'IN'
1   for $i IN /Navaid
-         ^
19114. 00000 -  "error during parsing the XQuery expression: %s"
*Cause:    An error occurred during the parsing of the XQuery expression.
*Action:   Check the detailed error message for the possible causes.
Error at Line: 32 Column: 6

SELECT xml_data, XMLQuery (
'for $i IN /Navaid
 where $i /NavaidID = "ZVR"
 return $i /State'
PASSING BY VALUE xml_data
RETURNING CONTENT) As output --xmldata
FROM xml_table;

ORA-19114: XPST0003 - error during parsing the XQuery expression:
LPX-00801: XQuery syntax error at 'IN'
1   for $i IN /Navaid
-         ^
19114. 00000 -  "error during parsing the XQuery expression: %s"

select EXTRACTVALUE(VALUE(1),'//Navaid/NavaidID')
from xml_table,
   XMLTABLE('///Navaid/NavaidID'
                PASSING xml_data
               );
Most Valuable Expert 2012
Distinguished Expert 2018

Commented:
extract and extractvalue have been deprecated.  Forget it even exists.


Here is an XMLTABLE example with only one NavaidID.

I'm still working on the XMLQuery example.

From your last post, looks like you might have repeating nodes in NavaidID.

If you can update my test case with more XML and expected results, I can modify my example.


drop table tab1 purge;
create table tab1(xml_data xmltype);


insert into tab1 values('
<TfmsData xmlns="http://tfm.faa.gov/tfms/TfmsData" xmlns:nxce="http://tfm.faa.gov/tfms/NasXCoreElements">
  <GeospatialData>
 	<Navaid>
 		<NavaidID>ZVR</NavaidID>
 	</Navaid>
  </GeospatialData>
</TfmsData>
');

commit;

select NavaidID
from tab1,
	xmltable(
		 xmlnamespaces(default 'http://tfm.faa.gov/tfms/TfmsData',
			'http://tfm.faa.gov/tfms/NasXCoreElements' as "nxce"
		),
	'/TfmsData/GeospatialData'
	passing xml_data
	columns
		NavaidID varchar2(50) path '/GeospatialData/Navaid/NavaidID'
	) xmltab
/

Open in new window

Most Valuable Expert 2012
Distinguished Expert 2018

Commented:
XMLQuery that works for navaidid:

select XMLQuery(
	'declare default element namespace "http://tfm.faa.gov/tfms/TfmsData"; (: :)
	//Navaid/NavaidID/text()'
	PASSING XML_DATA  RETURNING CONTENT) 
from tab1;

Open in new window


Still not sure how to provide multiple namespaces but I'll figure you'll need them eventually.

If I figure it out, I'll post back.
Most Valuable Expert 2012
Distinguished Expert 2018
Commented:
XMLQuery for both namespaces:
select XMLQuery(
	'declare default element namespace "http://tfm.faa.gov/tfms/TfmsData"; (: :) declare namespace nxce="http://tfm.faa.gov/tfms/NasXCoreElements"; (: :)
	//Navaid/NavaidID/text()'
	PASSING XML_DATA  RETURNING CONTENT) 
from tab1;

Open in new window

Author

Commented:
Ok thanks, as before you've helped Hugely.
Most Valuable Expert 2012
Distinguished Expert 2018

Commented:
Based on what you posted, I would probably go with XMLTable.

Sounds like you want to treat the data as 'normal' relational data.

From the few minutes I was reading about XMLQuery, it returns XML.

If you wanted two values from your XML, not sure XMLQuery would be the path to go down.

Author

Commented:
The following works to return all values of NavaidID.  The xml data structure is organized below, so if I wanted the ARTCCID value at the same 'level' as NAVAIDID where NAVAIDID = 'ZVR' how would I modify your query below?

select XMLQuery(
      'declare default element namespace "http://tfm.faa.gov/tfms/TfmsData"; (: :) declare namespace nxce="http://tfm.faa.gov/tfms/NasXCoreElements"; (: :)
      //Navaid/NavaidID/text()'
      PASSING XML_DATA  RETURNING CONTENT)
from tab1;

data structure,
<TfmsData xmlns="http://tfm.faa.gov/tfms/TfmsData" xmlns:nxce="http://tfm.faa.gov/tfms/NasXCoreElements">
 <GeospatialData><Navaid><NavaidID>ZVR</NavaidID>
 <ArtccID>CZV</ArtccID>
Most Valuable Expert 2012
Distinguished Expert 2018

Commented:
This is why I would use XMLTable over XMLQuery.

drop table tab1 purge;
create table tab1(xml_data xmltype);

insert into tab1 values('
<TfmsData xmlns="http://tfm.faa.gov/tfms/TfmsData" xmlns:nxce="http://tfm.faa.gov/tfms/NasXCoreElements">
  <GeospatialData>
 	<Navaid>
 		<NavaidID>ZVR</NavaidID>
  		<ArtccID>CZV</ArtccID>
 	</Navaid>
  </GeospatialData>
</TfmsData>
');

commit;

select NavaidID,ArtccID
from tab1,
	xmltable(
		 xmlnamespaces(default 'http://tfm.faa.gov/tfms/TfmsData',
			'http://tfm.faa.gov/tfms/NasXCoreElements' as "nxce"
		),
	'/TfmsData/GeospatialData'
	passing xml_data
	columns
		NavaidID	varchar2(50) path '/GeospatialData/Navaid/NavaidID',
		ArtccID		varchar2(50) path '/GeospatialData/Navaid/ArtccID'
	) xmltab
/

Open in new window

Author

Commented:
Ok lesson learned and thanks again.

Author

Commented:
Playing around with the xmltable query you provided I was able to determine the 'where clause' I was looking for as follows,

select ArtccID
from xml_table,
      xmltable(
             xmlnamespaces(default 'http://tfm.faa.gov/tfms/TfmsData',
                  'http://tfm.faa.gov/tfms/NasXCoreElements' as "nxce"
            ),
      '/TfmsData/GeospatialData/Navaid[NavaidID="ZWG"]'
      passing xml_data
      columns
            artccID varchar2(50) path '//Navaid/ArtccID'
      ) xml_data --xmltab
/
Most Valuable Expert 2012
Distinguished Expert 2018

Commented:
Or just add a where clause.  Since XMLTable returns relational data, it is subject to the same constraints.

select ArtccID
...
where NavaidID = 'ZWG'

Author

Commented:
No the 'where clause' above did not work after I tried numerous variations.  Finally got some time to get back to this and will probably be posting additional questions about expanding this 'where clause' with an equivalent AND to extract specific data.

Do more with

Expert Office
Submit tech questions to Ask the Experts™ at any time to receive solutions, advice, and new ideas from leading industry professionals.

Start 7-Day Free Trial