Solved

MS SQL SSIS xml source task

Posted on 2014-07-23
9
322 Views
Last Modified: 2016-02-11
I am importing xml multiple times a day from a vendor. However when SSIS created the ID's for nested XML data it is not unique.   So importing the first time and I get 3-4 records it looks fine. However subsequent imports all use the same ID's so it isn't unique, how do I go about changing this as I cant find anything about it.
0
Comment
Question by:CaptainGiblets
[X]
Welcome to Experts Exchange

Add your voice to the tech community where 5M+ people just like you are talking about what matters.

  • Help others & share knowledge
  • Earn cash & points
  • Learn & ask questions
  • 4
  • 2
9 Comments
 
LVL 4

Expert Comment

by:Nazermohideeen
ID: 40214749
From what I understand, I think you might be able to add a Derived column and add a GUID unique ID, which will be unique all the time.
0
 
LVL 6

Author Comment

by:CaptainGiblets
ID: 40214758
But that would only work for the rows that go into each table. The way SSIS does it it links all the data together using more than 1 primary key and links them all together. If i added a derived column with a GUID it wouldnt work as some columns that are auto created have the same value 4-5 times to link them to certain records etc.
0
 
LVL 4

Expert Comment

by:Nazermohideeen
ID: 40214812
Need more information about your SSIS. May be a screenshot or something.
0
Optimize your web performance

What's in the eBook?
- Full list of reasons for poor performance
- Ultimate measures to speed things up
- Primary web monitoring types
- KPIs you should be monitoring in order to increase your ROI

 
LVL 6

Author Comment

by:CaptainGiblets
ID: 40214832
Will post back tomorrow when in office but it creates 8 tables which all use some sort of unique id and sometimes a non unique. For example I have a questionanswerid which then l
0
 
LVL 6

Author Comment

by:CaptainGiblets
ID: 40219007
Sorry was side tracked.

here is an example of the XML file
C--New-folder-campaneo-Download.xml
0
 
LVL 6

Author Comment

by:CaptainGiblets
ID: 40226684
Still need help with this if anyone has any ideas.
0
 
LVL 37

Accepted Solution

by:
ValentinoV earned 500 total points
ID: 40238329
I have the impression you're currently adding the data from the files to the same tables that already contain data from the previous files, without any staging phase.  Is that right?

To avoid your issue you should work with staging tables.  For each file import they need to get truncated.  That way you import an XML file into empty destination tables and you won't face that duplicate ID issue. Once the data is in the staging tables you can use a second Data Flow transform to extract it from there and move it into the final tables, which might be modeled a bit different, depending on your requirements.
0

Featured Post

Free Tool: ZipGrep

ZipGrep is a utility that can list and search zip (.war, .ear, .jar, etc) archives for text patterns, without the need to extract the archive's contents.

One of a set of tools we're offering as a way to say thank you for being a part of the community.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Create a Windows 10 custom Image with custom task bar and custom start menu using XML for deployment.
This post looks at MongoDB and MySQL, and covers high-level MongoDB strengths, weaknesses, features, and uses from the perspective of an SQL user.
Via a live example, show how to setup several different housekeeping processes for a SQL Server.
Viewers will learn how to use the INSERT statement to insert data into their tables. It will also introduce the NULL statement, to show them what happens when no value is giving for any given column.

635 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question