Solved

how to import xml from a website, parse it and save the results into mssql 2k8 r2

Posted on 2013-01-25
8
359 Views
Last Modified: 2013-01-27
OK...

I have an external XML web site (http://rss.timegenie.com/forex2.xml) that I need to import into MSSQL Server 2008 R2.

1. is this even possible?

2. how would this be done?

Thanks,
John
0
Comment
Question by:jschultz0614
8 Comments
 

Author Comment

by:jschultz0614
ID: 38821408
I tried the example found at http://www.experts-exchange.com/Programming/Languages/C_Sharp/Q_26631018.html with no luck, it only returns an empty rowset.
0
 
LVL 39

Expert Comment

by:Aaron Tomosky
ID: 38821453
Here are some good examples
http://stackoverflow.com/questions/3628846/insert-xml-into-sql-server-2008-database

Get the insert working outside of your site, them get a simple insert working, and only then put it all together. Too many things can be a problem when doing it all at once.
0
 
LVL 51

Accepted Solution

by:
Mark Wills earned 500 total points
ID: 38821942
Well, can do it all within a single SQL Script, but, need to use xp_cmdshell

You can turn it on at the start and turn it off at the end if needed. I have included turning on the option which allows xp_cmdshell.

Anyway, check out below, I have tried it and it works...

You just need to change the physical file path and server names (etc)

-- To allow changes to XP_CMDSHELL advanced options to be available.
EXEC sp_configure 'show advanced options', 1
GO
-- To update the currently configured value for advanced options.
RECONFIGURE
GO
-- To enable the feature.
EXEC sp_configure 'xp_cmdshell', 1
GO
-- To update the currently configured value for this feature.
RECONFIGURE
GO
-- if needed, turn off the advanced options again


-- Now, lets create a script within the database then we can export that and use that script to grab the page and then load the XML

if object_id('js_mydownload_script','U') is not null drop table js_mydownload_script
create table js_mydownload_script (line varchar(255))

insert js_mydownload_script values ('var fso = new ActiveXObject(''Scripting.FileSystemObject'');')
insert js_mydownload_script values ('var xmlhttp=new ActiveXObject("microsoft.xmlhttp");')
insert js_mydownload_script values ('xmlhttp.open("GET", "http://rss.timegenie.com/forex2.xml", false);')
insert js_mydownload_script values ('xmlhttp.send();')
insert js_mydownload_script values ('var data=xmlhttp.responsetext;')
insert js_mydownload_script values ('var file = fso.CreateTextFile("c:\\ee\\forex2.xml",true);')  -------- change file path
insert js_mydownload_script values ('file.writeLine(data);')
insert js_mydownload_script values ('file.close();')

-- right, now export that script -- remember to check and change that path.
exec xp_cmdshell 'bcp "select * from ee_test_db..js_mydownload_script" queryout "c:\ee\js_my_download_script.js" -SMY-SERVER\SQL2012 -c -T -CACP'

-- and then execute the script
exec xp_cmdshell 'cscript.exe c:\ee\js_my_download_script.js'

-- or create a generic script file externally - just a text file edited in notepad (between the /* ... */ below) - then use that passing in the params
/*
var fso = new ActiveXObject('Scripting.FileSystemObject');
var args = WScript.Arguments;
var url = args(0);
var fileName = args(1);
var xmlhttp=new ActiveXObject("microsoft.xmlhttp");
xmlhttp.open("GET", url, false);
xmlhttp.send();
var data=xmlhttp.responsetext;
var file = fso.CreateTextFile(fileName, 2);
file.writeLine(data);
file.close();
*/

-- then your execute sql becomes even easier :

exec xp_cmdshell 'cscript c:\ee\js_download_script.js http://rss.timegenie.com/forex2.xml c:\ee\forex2.xml'


-- now we have the XML file locally, we can then set about importing it...

-- first we will need a staging table
if object_id('tempdb..#my_xml_staging','U') is not null drop table #my_xml_staging
create table #my_xml_staging(id int identity, xml_col xml)

-- now bulk load the file into a staging table
INSERT INTO #my_xml_staging(Xml_Col)
SELECT * FROM OPENROWSET(BULK 'c:\ee\forex2.xml',SINGLE_BLOB) AS x

-- now lets see what we have
select R.C.value('@code','char(3)') as code,
       R.C.value('@description','varchar(100)') as description,
       R.C.value('@rate','decimal(12,4)') as rate
from #my_xml_staging X
cross apply X.xml_col.nodes('/forex/data/currency') as R(C)

-- and we can add the "insert my_real_table (code, description, rate)" as the first line above the select to insert into the real home.

-- If needed, turn off xp_cmdshell by using the sp_configure and setting to 0 instead of 1 as per above

Open in new window

0
Best Practices: Disaster Recovery Testing

Besides backup, any IT division should have a disaster recovery plan. You will find a few tips below relating to the development of such a plan and to what issues one should pay special attention in the course of backup planning.

 

Author Comment

by:jschultz0614
ID: 38822187
getting the following error:
Msg 4860, Level 16, State 1, Line 51
Cannot bulk load. The file "c:\ee\forex2.xml" does not exist.

I created the ee directory. The script works only if I manually download the xml file myself.
0
 

Author Comment

by:jschultz0614
ID: 38822195
Disregard :), forgot to create the js script :D.

AWESOME SCRIPT btw.
0
 

Author Closing Comment

by:jschultz0614
ID: 38822196
Awesome Script
0
 
LVL 51

Expert Comment

by:Mark Wills
ID: 38822327
Thanks, I have been using it for a while now, and (so far) it stands the test of time...

Cheers,
Mark
0
 
LVL 1

Expert Comment

by:marrowyung
ID: 38824199
0

Featured Post

PRTG Network Monitor: Intuitive Network Monitoring

Network Monitoring is essential to ensure that computer systems and network devices are running. Use PRTG to monitor LANs, servers, websites, applications and devices, bandwidth, virtual environments, remote systems, IoT, and many more. PRTG is easy to set up & use.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Long way back, we had to take help from third party tools in order to encrypt and decrypt data.  Gradually Microsoft understood the need for this feature and started to implement it by building functionality into SQL Server. Finally, with SQL 2008, …
In this article I will describe the Copy Database Wizard method as one possible migration process and I will add the extra tasks needed for an upgrade when and where is applied so it will cover all.
Nobody understands Phishing better than an anti-spam company. That’s why we are providing Phishing Awareness Training to our customers. According to a report by Verizon, only 3% of targeted users report malicious emails to management. With compan…
In an interesting question (https://www.experts-exchange.com/questions/29008360/) here at Experts Exchange, a member asked how to split a single image into multiple images. The primary usage for this is to place many photographs on a flatbed scanner…

856 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question