?
Solved

how to import xml from a website, parse it and save the results into mssql 2k8 r2

Posted on 2013-01-25
8
Medium Priority
?
373 Views
Last Modified: 2013-01-27
OK...

I have an external XML web site (http://rss.timegenie.com/forex2.xml) that I need to import into MSSQL Server 2008 R2.

1. is this even possible?

2. how would this be done?

Thanks,
John
0
Comment
Question by:jschultz0614
[X]
Welcome to Experts Exchange

Add your voice to the tech community where 5M+ people just like you are talking about what matters.

  • Help others & share knowledge
  • Earn cash & points
  • Learn & ask questions
8 Comments
 

Author Comment

by:jschultz0614
ID: 38821408
I tried the example found at http://www.experts-exchange.com/Programming/Languages/C_Sharp/Q_26631018.html with no luck, it only returns an empty rowset.
0
 
LVL 39

Expert Comment

by:Aaron Tomosky
ID: 38821453
Here are some good examples
http://stackoverflow.com/questions/3628846/insert-xml-into-sql-server-2008-database

Get the insert working outside of your site, them get a simple insert working, and only then put it all together. Too many things can be a problem when doing it all at once.
0
 
LVL 51

Accepted Solution

by:
Mark Wills earned 2000 total points
ID: 38821942
Well, can do it all within a single SQL Script, but, need to use xp_cmdshell

You can turn it on at the start and turn it off at the end if needed. I have included turning on the option which allows xp_cmdshell.

Anyway, check out below, I have tried it and it works...

You just need to change the physical file path and server names (etc)

-- To allow changes to XP_CMDSHELL advanced options to be available.
EXEC sp_configure 'show advanced options', 1
GO
-- To update the currently configured value for advanced options.
RECONFIGURE
GO
-- To enable the feature.
EXEC sp_configure 'xp_cmdshell', 1
GO
-- To update the currently configured value for this feature.
RECONFIGURE
GO
-- if needed, turn off the advanced options again


-- Now, lets create a script within the database then we can export that and use that script to grab the page and then load the XML

if object_id('js_mydownload_script','U') is not null drop table js_mydownload_script
create table js_mydownload_script (line varchar(255))

insert js_mydownload_script values ('var fso = new ActiveXObject(''Scripting.FileSystemObject'');')
insert js_mydownload_script values ('var xmlhttp=new ActiveXObject("microsoft.xmlhttp");')
insert js_mydownload_script values ('xmlhttp.open("GET", "http://rss.timegenie.com/forex2.xml", false);')
insert js_mydownload_script values ('xmlhttp.send();')
insert js_mydownload_script values ('var data=xmlhttp.responsetext;')
insert js_mydownload_script values ('var file = fso.CreateTextFile("c:\\ee\\forex2.xml",true);')  -------- change file path
insert js_mydownload_script values ('file.writeLine(data);')
insert js_mydownload_script values ('file.close();')

-- right, now export that script -- remember to check and change that path.
exec xp_cmdshell 'bcp "select * from ee_test_db..js_mydownload_script" queryout "c:\ee\js_my_download_script.js" -SMY-SERVER\SQL2012 -c -T -CACP'

-- and then execute the script
exec xp_cmdshell 'cscript.exe c:\ee\js_my_download_script.js'

-- or create a generic script file externally - just a text file edited in notepad (between the /* ... */ below) - then use that passing in the params
/*
var fso = new ActiveXObject('Scripting.FileSystemObject');
var args = WScript.Arguments;
var url = args(0);
var fileName = args(1);
var xmlhttp=new ActiveXObject("microsoft.xmlhttp");
xmlhttp.open("GET", url, false);
xmlhttp.send();
var data=xmlhttp.responsetext;
var file = fso.CreateTextFile(fileName, 2);
file.writeLine(data);
file.close();
*/

-- then your execute sql becomes even easier :

exec xp_cmdshell 'cscript c:\ee\js_download_script.js http://rss.timegenie.com/forex2.xml c:\ee\forex2.xml'


-- now we have the XML file locally, we can then set about importing it...

-- first we will need a staging table
if object_id('tempdb..#my_xml_staging','U') is not null drop table #my_xml_staging
create table #my_xml_staging(id int identity, xml_col xml)

-- now bulk load the file into a staging table
INSERT INTO #my_xml_staging(Xml_Col)
SELECT * FROM OPENROWSET(BULK 'c:\ee\forex2.xml',SINGLE_BLOB) AS x

-- now lets see what we have
select R.C.value('@code','char(3)') as code,
       R.C.value('@description','varchar(100)') as description,
       R.C.value('@rate','decimal(12,4)') as rate
from #my_xml_staging X
cross apply X.xml_col.nodes('/forex/data/currency') as R(C)

-- and we can add the "insert my_real_table (code, description, rate)" as the first line above the select to insert into the real home.

-- If needed, turn off xp_cmdshell by using the sp_configure and setting to 0 instead of 1 as per above

Open in new window

0
RHCE - Red Hat OpenStack Prep Course

This course will provide in-depth training so that students who currently hold the EX200 & EX210 certifications can sit for the EX310 exam. Students will learn how to deploy & manage a full Red Hat environment with Ceph block storage, & integrate Ceph into other OpenStack service

 

Author Comment

by:jschultz0614
ID: 38822187
getting the following error:
Msg 4860, Level 16, State 1, Line 51
Cannot bulk load. The file "c:\ee\forex2.xml" does not exist.

I created the ee directory. The script works only if I manually download the xml file myself.
0
 

Author Comment

by:jschultz0614
ID: 38822195
Disregard :), forgot to create the js script :D.

AWESOME SCRIPT btw.
0
 

Author Closing Comment

by:jschultz0614
ID: 38822196
Awesome Script
0
 
LVL 51

Expert Comment

by:Mark Wills
ID: 38822327
Thanks, I have been using it for a while now, and (so far) it stands the test of time...

Cheers,
Mark
0
 
LVL 1

Expert Comment

by:marrowyung
ID: 38824199
0

Featured Post

Free learning courses: Active Directory Deep Dive

Get a firm grasp on your IT environment when you learn Active Directory best practices with Veeam! Watch all, or choose any amount, of this three-part webinar series to improve your skills. From the basics to virtualization and backup, we got you covered.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

In this article I will describe the Detach & Attach method as one possible migration process and I will add the extra tasks needed for an upgrade when and where is applied so it will cover all.
This article explains how to reset the password of the sa account on a Microsoft SQL Server.  The steps in this article work in SQL 2005, 2008, 2008 R2, 2012, 2014 and 2016.
Michael from AdRem Software explains how to view the most utilized and worst performing nodes in your network, by accessing the Top Charts view in NetCrunch network monitor (https://www.adremsoft.com/). Top Charts is a view in which you can set seve…
Monitoring a network: how to monitor network services and why? Michael Kulchisky, MCSE, MCSA, MCP, VTSP, VSP, CCSP outlines the philosophy behind service monitoring and why a handshake validation is critical in network monitoring. Software utilized …

770 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question