Solved

how to import xml from a website, parse it and save the results into mssql 2k8 r2

Posted on 2013-01-25
8
369 Views
Last Modified: 2013-01-27
OK...

I have an external XML web site (http://rss.timegenie.com/forex2.xml) that I need to import into MSSQL Server 2008 R2.

1. is this even possible?

2. how would this be done?

Thanks,
John
0
Comment
Question by:jschultz0614
[X]
Welcome to Experts Exchange

Add your voice to the tech community where 5M+ people just like you are talking about what matters.

  • Help others & share knowledge
  • Earn cash & points
  • Learn & ask questions
8 Comments
 

Author Comment

by:jschultz0614
ID: 38821408
I tried the example found at http://www.experts-exchange.com/Programming/Languages/C_Sharp/Q_26631018.html with no luck, it only returns an empty rowset.
0
 
LVL 39

Expert Comment

by:Aaron Tomosky
ID: 38821453
Here are some good examples
http://stackoverflow.com/questions/3628846/insert-xml-into-sql-server-2008-database

Get the insert working outside of your site, them get a simple insert working, and only then put it all together. Too many things can be a problem when doing it all at once.
0
 
LVL 51

Accepted Solution

by:
Mark Wills earned 500 total points
ID: 38821942
Well, can do it all within a single SQL Script, but, need to use xp_cmdshell

You can turn it on at the start and turn it off at the end if needed. I have included turning on the option which allows xp_cmdshell.

Anyway, check out below, I have tried it and it works...

You just need to change the physical file path and server names (etc)

-- To allow changes to XP_CMDSHELL advanced options to be available.
EXEC sp_configure 'show advanced options', 1
GO
-- To update the currently configured value for advanced options.
RECONFIGURE
GO
-- To enable the feature.
EXEC sp_configure 'xp_cmdshell', 1
GO
-- To update the currently configured value for this feature.
RECONFIGURE
GO
-- if needed, turn off the advanced options again


-- Now, lets create a script within the database then we can export that and use that script to grab the page and then load the XML

if object_id('js_mydownload_script','U') is not null drop table js_mydownload_script
create table js_mydownload_script (line varchar(255))

insert js_mydownload_script values ('var fso = new ActiveXObject(''Scripting.FileSystemObject'');')
insert js_mydownload_script values ('var xmlhttp=new ActiveXObject("microsoft.xmlhttp");')
insert js_mydownload_script values ('xmlhttp.open("GET", "http://rss.timegenie.com/forex2.xml", false);')
insert js_mydownload_script values ('xmlhttp.send();')
insert js_mydownload_script values ('var data=xmlhttp.responsetext;')
insert js_mydownload_script values ('var file = fso.CreateTextFile("c:\\ee\\forex2.xml",true);')  -------- change file path
insert js_mydownload_script values ('file.writeLine(data);')
insert js_mydownload_script values ('file.close();')

-- right, now export that script -- remember to check and change that path.
exec xp_cmdshell 'bcp "select * from ee_test_db..js_mydownload_script" queryout "c:\ee\js_my_download_script.js" -SMY-SERVER\SQL2012 -c -T -CACP'

-- and then execute the script
exec xp_cmdshell 'cscript.exe c:\ee\js_my_download_script.js'

-- or create a generic script file externally - just a text file edited in notepad (between the /* ... */ below) - then use that passing in the params
/*
var fso = new ActiveXObject('Scripting.FileSystemObject');
var args = WScript.Arguments;
var url = args(0);
var fileName = args(1);
var xmlhttp=new ActiveXObject("microsoft.xmlhttp");
xmlhttp.open("GET", url, false);
xmlhttp.send();
var data=xmlhttp.responsetext;
var file = fso.CreateTextFile(fileName, 2);
file.writeLine(data);
file.close();
*/

-- then your execute sql becomes even easier :

exec xp_cmdshell 'cscript c:\ee\js_download_script.js http://rss.timegenie.com/forex2.xml c:\ee\forex2.xml'


-- now we have the XML file locally, we can then set about importing it...

-- first we will need a staging table
if object_id('tempdb..#my_xml_staging','U') is not null drop table #my_xml_staging
create table #my_xml_staging(id int identity, xml_col xml)

-- now bulk load the file into a staging table
INSERT INTO #my_xml_staging(Xml_Col)
SELECT * FROM OPENROWSET(BULK 'c:\ee\forex2.xml',SINGLE_BLOB) AS x

-- now lets see what we have
select R.C.value('@code','char(3)') as code,
       R.C.value('@description','varchar(100)') as description,
       R.C.value('@rate','decimal(12,4)') as rate
from #my_xml_staging X
cross apply X.xml_col.nodes('/forex/data/currency') as R(C)

-- and we can add the "insert my_real_table (code, description, rate)" as the first line above the select to insert into the real home.

-- If needed, turn off xp_cmdshell by using the sp_configure and setting to 0 instead of 1 as per above

Open in new window

0
Optimizing Cloud Backup for Low Bandwidth

With cloud storage prices going down a growing number of SMBs start to use it for backup storage. Unfortunately, business data volume rarely fits the average Internet speed. This article provides an overview of main Internet speed challenges and reveals backup best practices.

 

Author Comment

by:jschultz0614
ID: 38822187
getting the following error:
Msg 4860, Level 16, State 1, Line 51
Cannot bulk load. The file "c:\ee\forex2.xml" does not exist.

I created the ee directory. The script works only if I manually download the xml file myself.
0
 

Author Comment

by:jschultz0614
ID: 38822195
Disregard :), forgot to create the js script :D.

AWESOME SCRIPT btw.
0
 

Author Closing Comment

by:jschultz0614
ID: 38822196
Awesome Script
0
 
LVL 51

Expert Comment

by:Mark Wills
ID: 38822327
Thanks, I have been using it for a while now, and (so far) it stands the test of time...

Cheers,
Mark
0
 
LVL 1

Expert Comment

by:marrowyung
ID: 38824199
0

Featured Post

Free Tool: IP Lookup

Get more info about an IP address or domain name, such as organization, abuse contacts and geolocation.

One of a set of tools we are providing to everyone as a way of saying thank you for being a part of the community.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

In this article I will describe the Detach & Attach method as one possible migration process and I will add the extra tasks needed for an upgrade when and where is applied so it will cover all.
Many times as a report developer I've been asked to display normalized data such as three rows with values Jack, Joe, and Bob as a single comma-separated string such as 'Jack, Joe, Bob', and vice versa.  Here's how to do it. 
If you're a developer or IT admin, you’re probably tasked with managing multiple websites, servers, applications, and levels of security on a daily basis. While this can be extremely time consuming, it can also be frustrating when systems aren't wor…
In this brief tutorial Pawel from AdRem Software explains how you can quickly find out which services are running on your network, or what are the IP addresses of servers responsible for each service. Software used is freeware NetCrunch Tools (https…

707 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question