Solved

Bulk import from MDS using SSIS

Posted on 2016-11-02
4
44 Views
Last Modified: 2016-11-07
Hi

Please assist on ImportType (https://msdn.microsoft.com/en-us/library/ee633854) on MDS I can use to insert data to the <mds. stg table> without duplicating existing data?I have used 0,1,2 and all of them are giving me the same results,each time I run the SSIS package it insert new records to the table even the records that are already in the table.
0
Comment
Question by:devguru001
  • 2
  • 2
4 Comments
 
LVL 3

Expert Comment

by:Shailesh Shinde
Comment Utility
Hi,
Each entity in SQL Server 2012 Master Data Services (MDS) will have it’s own staging table (stg.<name>_Leaf). Using this staging table, you can create, update, deactivate and delete left members in bulk.
Have you gone through with this post...


Thanks,
Shail
0
 
LVL 3

Expert Comment

by:Shailesh Shinde
Comment Utility
0
 
LVL 13

Expert Comment

by:Megan Brooks
Comment Utility
Are you saying that the data values for two different members (codes) are the same? Are you automatically generating codes? The import doesn't check for duplicate data for a given member.

The ImportType compares members using the Code attribute. You can specify that only new members (codes) are created (ImportType = 1), or have it also update existing members (identified by code) with new data, either treating NULL attribute data as 'don't change this attribute' (ImportType = 0) or as actual data (ImportType = 2).

(While I have worked extensively with MDS, it was 3+ years ago. Apologies in advance if I am not understanding the problem correctly.)
0
 
LVL 13

Accepted Solution

by:
Megan Brooks earned 500 total points
Comment Utility
You may need to load an SSIS lookup with data from an MDS subscription view, and then check new incoming data against that before sending it to the staging table (so as to exclude the duplicate rows). It sounds, though, like there is an issue with your coding scheme, or that you are using MDS to store something other than attribute data, for which codes might not be meaningful.

If you can provide a little more background on what you are trying to do then I might be able to suggest something more.
0

Featured Post

Threat Intelligence Starter Resources

Integrating threat intelligence can be challenging, and not all companies are ready. These resources can help you build awareness and prepare for defense.

Join & Write a Comment

Sometimes, you need to use PIVOT in SSIS to ensure your data matches the output requirements of your users. So, what is PIVOT and what is SSIS and how can that help ? Firstly a quick explanation of those acronyms as described by Microsoft :   M…
Performance is the key factor for any successful data integration project, knowing the type of transformation that you’re using is the first step on optimizing the SSIS flow performance, by utilizing the correct transformation or the design alternat…
This video discusses moving either the default database or any database to a new volume.
This video explains how to create simple products associated to Magento configurable product and offers fast way of their generation with Store Manager for Magento tool.

762 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

12 Experts available now in Live!

Get 1:1 Help Now