Solved

MS SQL SSIS xml source task

Posted on 2014-07-23
9
298 Views
Last Modified: 2016-02-11
I am importing xml multiple times a day from a vendor. However when SSIS created the ID's for nested XML data it is not unique.   So importing the first time and I get 3-4 records it looks fine. However subsequent imports all use the same ID's so it isn't unique, how do I go about changing this as I cant find anything about it.
0
Comment
Question by:CaptainGiblets
  • 4
  • 2
9 Comments
 
LVL 4

Expert Comment

by:Nazermohideeen
ID: 40214749
From what I understand, I think you might be able to add a Derived column and add a GUID unique ID, which will be unique all the time.
0
 
LVL 6

Author Comment

by:CaptainGiblets
ID: 40214758
But that would only work for the rows that go into each table. The way SSIS does it it links all the data together using more than 1 primary key and links them all together. If i added a derived column with a GUID it wouldnt work as some columns that are auto created have the same value 4-5 times to link them to certain records etc.
0
 
LVL 4

Expert Comment

by:Nazermohideeen
ID: 40214812
Need more information about your SSIS. May be a screenshot or something.
0
Zoho SalesIQ

Hassle-free live chat software re-imagined for business growth. 2 users, always free.

 
LVL 6

Author Comment

by:CaptainGiblets
ID: 40214832
Will post back tomorrow when in office but it creates 8 tables which all use some sort of unique id and sometimes a non unique. For example I have a questionanswerid which then l
0
 
LVL 6

Author Comment

by:CaptainGiblets
ID: 40219007
Sorry was side tracked.

here is an example of the XML file
C--New-folder-campaneo-Download.xml
0
 
LVL 6

Author Comment

by:CaptainGiblets
ID: 40226684
Still need help with this if anyone has any ideas.
0
 
LVL 37

Accepted Solution

by:
ValentinoV earned 500 total points
ID: 40238329
I have the impression you're currently adding the data from the files to the same tables that already contain data from the previous files, without any staging phase.  Is that right?

To avoid your issue you should work with staging tables.  For each file import they need to get truncated.  That way you import an XML file into empty destination tables and you won't face that duplicate ID issue. Once the data is in the staging tables you can use a second Data Flow transform to extract it from there and move it into the final tables, which might be modeled a bit different, depending on your requirements.
0

Featured Post

Optimizing Cloud Backup for Low Bandwidth

With cloud storage prices going down a growing number of SMBs start to use it for backup storage. Unfortunately, business data volume rarely fits the average Internet speed. This article provides an overview of main Internet speed challenges and reveals backup best practices.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Using SQL Scripts we can save all the SQL queries as files that we use very frequently on our database later point of time. This is one of the feature present under SQL Workshop in Oracle Application Express.
For both online and offline retail, the cross-channel business is the most recent pattern in the B2C trade space.
This videos aims to give the viewer a basic demonstration of how a user can query current session information by using the SYS_CONTEXT function
Viewers will learn how the fundamental information of how to create a table.

863 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

26 Experts available now in Live!

Get 1:1 Help Now