Solved

SSIS v BCP

Posted on 2013-12-03
4
1,075 Views
Last Modified: 2016-02-10
Hi All,
Just wondering if you had any preference on how to transfer data between data centers.

We are using SQL Server 2005 in the US and want to transfer data to a different data center in Europe, the database in Europe will be an oracle database.

two options we are working with are Batch copy process.
Transferring txt files from our sql server to a location where our counterparts can pick up and ingest into their own systems.

Or using SSIS ETL solution- linking our source data sql server tables to the destination tables in Europe (oracle db destination)

Using BCP before, I have pulled data from the same data center from one database and inserted to a different database, this process was taking possibly 3 hours to run.
When I started using SSIS to preform this process, it shortened the processing time to aprox 15 mins. So I know using SSIS when transferring data between two different SQL server databases is the way to go. But i'm wondering, if i'm transferring from SQL server database on one data center on one continent to another data center to an oracle db on a different continent would this be quicker than bcp?

thank you,
P
0
Comment
Question by:Putoch
[X]
Welcome to Experts Exchange

Add your voice to the tech community where 5M+ people just like you are talking about what matters.

  • Help others & share knowledge
  • Earn cash & points
  • Learn & ask questions
4 Comments
 
LVL 44

Accepted Solution

by:
Rainer Jeschor earned 500 total points
ID: 39694921
Hi,
it depends a lot on your connectivity between the data centers (and of course the amount of data): speed, bandwith, latency, availability

If you estimate that the transfer would not take longer as 15-30 minutes, then I would use SSIS directly to read from SQL and write to Oracle.

If this process might take multiple hours,
Alternative 1:
use SSIS to create well formatted text files, copy that files to your Oracle Data Center and start importing there
Alternative 2:
use SSIS to create RAW files, ZIP them, copy them to the other data center and using SSIS there (so there has to be a SQL Server instance) to import into Oracle.

Just my 2ct
Rainer
0
 
LVL 15

Expert Comment

by:jorge_toriz
ID: 39697575
I think both approaches are valid, the thing is to set up the right network packet size and batch size.
0
 

Author Comment

by:Putoch
ID: 39701868
Yea the amount of detail being transferred and connectivity would have to be considered.
Thank you all for your feedback,
0

Featured Post

Online Training Solution

Drastically shorten your training time with WalkMe's advanced online training solution that Guides your trainees to action. Forget about retraining and skyrocket knowledge retention rates.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

This article explains how to reset the password of the sa account on a Microsoft SQL Server.  The steps in this article work in SQL 2005, 2008, 2008 R2, 2012, 2014 and 2016.
The Delta outage: 650 cancelled flights, more than 1200 delayed flights, thousands of frustrated customers, tens of millions of dollars in damages – plus untold reputational damage to one of the world’s most trusted airlines. All due to a catastroph…
Via a live example, show how to set up a backup for SQL Server using a Maintenance Plan and how to schedule the job into SQL Server Agent.
Via a live example, show how to setup several different housekeeping processes for a SQL Server.

733 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question