Solved

SQL: Loading XML into a table::

Posted on 2013-06-25
3
253 Views
Last Modified: 2013-07-03
hello team,

A third party of ours offered us a download method for an XML file.
Here is the issue; they are giving us the table in pagination. Meaning, page 1, 2, etc

I suppose, I can loop through x number of pages but don't know how to put them into ONE table and of course ONE xml in sql server 2008
Any assistance is appreciated
0
Comment
Question by:John Esraelo
  • 2
3 Comments
 
LVL 51

Accepted Solution

by:
Mark Wills earned 500 total points
ID: 39277620
Ummm... Not sure what you mean, or asking for, so will need a bit more explanation.

Are you saying that there are multiple XML files that you want to combine into a single table, or, a single XML file ?

Is it possible to give us an example (changing some labels and obfuscating sensitive data) and what you hope to achieve as a result ?

Sometimes, you might have to first load into a temporary staging area as a transition stage before putting away into your final destination.
0
 

Author Comment

by:John Esraelo
ID: 39278303
It is supposed to be a single table with 10,000 appx records (see URL, softwares.xml).  The keeper of the data does not want to allow users to download the whole table to avoid overload. Therefore, the so called API of theirs is allowing the users to specify a page number.  
http://www.samanage.com/api/index.html
I suppose I can loop thru the X number of pages, insert them as an xml record in a sql table.
At that point I am not sure how to combine all pages together.
0
 
LVL 51

Expert Comment

by:Mark Wills
ID: 39298606
Sorry about that... Didn't get back to you in a timely manner...

The hard bit is knowing how many pages. By the looks of things you can specify how many entries per page, not sure if there is a maximum, so would be interesting to first manually explore the maximum.

It does say that every request will give you the number of entries, the number of pages, etc. as part of the header block. While the example says "tickets" the example results seems to be "hardwares".

So, as a strategy I would write an SQL script to get the first 1 entry, examine that for total number of entries, then loop through grabbing a page at a time (saving as a disk file, use the -o param if using curl), specifying the number of entries per page, import that file into a staging table (a temp table would work), then once all pages have been loaded (or no more results), go through the staging table and then put away.  

Not sure why or when you need to combine the pages, assuming what you really want is the resulting nodes.

I would be breaking up the XML into discrete fields / columns based on the nodes of interest to you. You can always recreate an XML document based on those columns depending on what you ultimately want to do (and what format you need).
0

Featured Post

PRTG Network Monitor: Intuitive Network Monitoring

Network Monitoring is essential to ensure that computer systems and network devices are running. Use PRTG to monitor LANs, servers, websites, applications and devices, bandwidth, virtual environments, remote systems, IoT, and many more. PRTG is easy to set up & use.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Suggested Solutions

Title # Comments Views Activity
tempdb log keep growing 7 33
TSQL Challenge... 7 36
PHP delete contents of file- before writing to it 6 23
I'm using XML(xslt) - How to fix the BOM issue 5 15
Naughty Me. While I was changing the database name from DB1 to DB_PROD1 (yep it's not real database name ^v^), I changed the database name and notified my application fellows that I did it. They turn on the application, and everything is working. A …
Browsing the questions asked to the Experts of this forum, you will be amazed to see how many times people are headaching about monster regular expressions (regex) to select that specific part of some HTML or XML file they want to extract. The examp…
Microsoft Active Directory, the widely used IT infrastructure, is known for its high risk of credential theft. The best way to test your Active Directory’s vulnerabilities to pass-the-ticket, pass-the-hash, privilege escalation, and malware attacks …
The Email Laundry PDF encryption service allows companies to send confidential encrypted  emails to anybody. The PDF document can also contain attachments that are embedded in the encrypted PDF. The password is randomly generated by The Email Laundr…

860 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question