How to link data from different data sources in SSIS

Hi there,

I have requirement to produce a csv file from SSIS. I have different data sources i think six of them with different common id fields. I suppose i need to bring them in separate queries and join them and then produce csv file. Can you please guide me how to start on this?

regards,
Ali ShahSQL DeveloperAsked:
Who is Participating?

[Product update] Infrastructure Analysis Tool is now available with Business Accounts.Learn More

x
I wear a lot of hats...

"The solutions and answers provided on Experts Exchange have been extremely helpful to me over the last few years. I wear a lot of hats - Developer, Database Administrator, Help Desk, etc., so I know a lot of things but not a lot about one thing. Experts Exchange gives me answers from people who do know a lot about one thing, in a easy to use platform." -Todd S.

Máté FarkasDatabase Developer and AdministratorCommented:
This sounds like a complete project and not a simple question. You have 2 options:
1. Start it yourself and if you have any problem then we will help you.
2. Create a GIGS project on Experts Exchange, offer some money and somebody will help you to implement it.

Some additions if you want to do it on your own:
 - You have to create separate connection managers for all different servers of your data sources
 - You have to create separate OLE DB or ADO .NET sources for all of your data sources
 - You have to implement the query logic in your SSIS Data Flow using some transformation components (Sort, Derived Column, Merge Join).
- Finally the result can be exported to a .csv file.
This way data will not be stored in intermediate files/tables but used and calculated on the fly like a data stream in your SSIS package.
Another way is to copy all the data sources into a central / local SQL Server instance and merge / process them with classic T-SQL commands and then you can export the result into a file with SSIS.
Máté FarkasDatabase Developer and AdministratorCommented:
This expert suggested creating a Gigs project.
GIGS:
zephyr_hex (Megan)DeveloperCommented:
You can use Merge Join in SSIS to join data from different sources.

For example: https://www.red-gate.com/simple-talk/sql/ssis/ssis-basics-using-the-merge-join-transformation/

Experts Exchange Solution brought to you by

Your issues matter to us.

Facing a tech roadblock? Get the help and guidance you need from experienced professionals who care. Ask your question anytime, anywhere, with no hassle.

Start your 7-day free trial
Ali ShahSQL DeveloperAuthor Commented:
Thanks a lot both of you for your help. I have started working creating my own project as i know a little bit about the SSIS and also company wouldn't pay extra for the project.
However i am having some issues with merge join and have raised these issues here too. can you please help me. the thread is https://www.experts-exchange.com/questions/29065465/Help-with-SSIS-merge-join.html 

Kindest regards,
It's more than this solution.Get answers and train to solve all your tech problems - anytime, anywhere.Try it for free Edge Out The Competitionfor your dream job with proven skills and certifications.Get started today Stand Outas the employee with proven skills.Start learning today for free Move Your Career Forwardwith certification training in the latest technologies.Start your trial today
Microsoft SQL Server

From novice to tech pro — start learning today.