We help IT Professionals succeed at work.

How to link data from different data sources in SSIS

Ali Shah
Ali Shah asked
Hi there,

I have requirement to produce a csv file from SSIS. I have different data sources i think six of them with different common id fields. I suppose i need to bring them in separate queries and join them and then produce csv file. Can you please guide me how to start on this?

Watch Question

Máté FarkasDatabase Developer and Administrator
This sounds like a complete project and not a simple question. You have 2 options:
1. Start it yourself and if you have any problem then we will help you.
2. Create a GIGS project on Experts Exchange, offer some money and somebody will help you to implement it.

Some additions if you want to do it on your own:
 - You have to create separate connection managers for all different servers of your data sources
 - You have to create separate OLE DB or ADO .NET sources for all of your data sources
 - You have to implement the query logic in your SSIS Data Flow using some transformation components (Sort, Derived Column, Merge Join).
- Finally the result can be exported to a .csv file.
This way data will not be stored in intermediate files/tables but used and calculated on the fly like a data stream in your SSIS package.
Another way is to copy all the data sources into a central / local SQL Server instance and merge / process them with classic T-SQL commands and then you can export the result into a file with SSIS.
Máté FarkasDatabase Developer and Administrator

Top Expert 2010
You can use Merge Join in SSIS to join data from different sources.

For example: https://www.red-gate.com/simple-talk/sql/ssis/ssis-basics-using-the-merge-join-transformation/
Ali ShahSQL Developer


Thanks a lot both of you for your help. I have started working creating my own project as i know a little bit about the SSIS and also company wouldn't pay extra for the project.
However i am having some issues with merge join and have raised these issues here too. can you please help me. the thread is https://www.experts-exchange.com/questions/29065465/Help-with-SSIS-merge-join.html 

Kindest regards,