Solved

MS SQL SSIS xml source task

Posted on 2014-07-23
9
305 Views
Last Modified: 2016-02-11
I am importing xml multiple times a day from a vendor. However when SSIS created the ID's for nested XML data it is not unique.   So importing the first time and I get 3-4 records it looks fine. However subsequent imports all use the same ID's so it isn't unique, how do I go about changing this as I cant find anything about it.
0
Comment
Question by:CaptainGiblets
  • 4
  • 2
9 Comments
 
LVL 4

Expert Comment

by:Nazermohideeen
ID: 40214749
From what I understand, I think you might be able to add a Derived column and add a GUID unique ID, which will be unique all the time.
0
 
LVL 6

Author Comment

by:CaptainGiblets
ID: 40214758
But that would only work for the rows that go into each table. The way SSIS does it it links all the data together using more than 1 primary key and links them all together. If i added a derived column with a GUID it wouldnt work as some columns that are auto created have the same value 4-5 times to link them to certain records etc.
0
 
LVL 4

Expert Comment

by:Nazermohideeen
ID: 40214812
Need more information about your SSIS. May be a screenshot or something.
0
Webinar: Aligning, Automating, Winning

Join Dan Russo, Senior Manager of Operations Intelligence, for an in-depth discussion on how Dealertrack, leading provider of integrated digital solutions for the automotive industry, transformed their DevOps processes to increase collaboration and move with greater velocity.

 
LVL 6

Author Comment

by:CaptainGiblets
ID: 40214832
Will post back tomorrow when in office but it creates 8 tables which all use some sort of unique id and sometimes a non unique. For example I have a questionanswerid which then l
0
 
LVL 6

Author Comment

by:CaptainGiblets
ID: 40219007
Sorry was side tracked.

here is an example of the XML file
C--New-folder-campaneo-Download.xml
0
 
LVL 6

Author Comment

by:CaptainGiblets
ID: 40226684
Still need help with this if anyone has any ideas.
0
 
LVL 37

Accepted Solution

by:
ValentinoV earned 500 total points
ID: 40238329
I have the impression you're currently adding the data from the files to the same tables that already contain data from the previous files, without any staging phase.  Is that right?

To avoid your issue you should work with staging tables.  For each file import they need to get truncated.  That way you import an XML file into empty destination tables and you won't face that duplicate ID issue. Once the data is in the staging tables you can use a second Data Flow transform to extract it from there and move it into the final tables, which might be modeled a bit different, depending on your requirements.
0

Featured Post

What is SQL Server and how does it work?

The purpose of this paper is to provide you background on SQL Server. It’s your self-study guide for learning fundamentals. It includes both the history of SQL and its technical basics. Concepts and definitions will form the solid foundation of your future DBA expertise.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Never store passwords in plain text or just their hash: it seems a no-brainier, but there are still plenty of people doing that. I present the why and how on this subject, offering my own real life solution that you can implement right away, bringin…
These days, all we hear about hacktivists took down so and so websites and retrieved thousands of user’s data. One of the techniques to get unauthorized access to database is by performing SQL injection. This article is quite lengthy which gives bas…
Video by: Steve
Using examples as well as descriptions, step through each of the common simple join types, explaining differences in syntax, differences in expected outputs and showing how the queries run along with the actual outputs based upon a simple set of dem…
Via a live example, show how to backup a database, simulate a failure backup the tail of the database transaction log and perform the restore.

821 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question