Huge data movement from Sybase IQ to SQL server approach

learning_sybase
learning_sybase used Ask the Experts™
on
How to do large set of data movement from SAP ASE/ ASE IQ server to other RDBMS say SQL server.??
The data is need to move is around 2-3 TB , could you please provide a complete step to step approach.
My Sybase IQ tables are too big. A detailed explanation will be highly appreciated.

Thanks
Comment
Watch Question

Do more with

Expert Office
EXPERT OFFICE® is a registered trademark of EXPERTS EXCHANGE®
lcohanDatabase Analyst

Commented:
From the description is not clear if this is a one time data migration or one sync from Sybase to MS SQL database or whavere else it may be so depending on the case scenario you could use the Sybase migration too from Microsoft SQL as per https://docs.microsoft.com/en-us/sql/ssma/sybase/sql-server-migration-assistant-for-sybase-sybasetosql?view=sql-server-2017&viewFallbackFrom=sql-server-2014 OR use SQL Server SSIS Integration Packages to Import your Sybase data into SQL OR you could use Sybase ddlgen utilitly to extract the database structure then use the Sybase bcp utility to extract the data from the Sybase database and bcp into SQL.

Author

Commented:
Thanks Icohon. Joe and Alpmoon, waiting for your inputs too. Please advise.
SQL Server DBA & Architect, EE Solution Guide
Awarded 2009
Distinguished Expert 2018
Commented:
>> The data is need to move is around 2-3 TB

Few pointers to be noted..
1. What is the number of tables in your database.
2. Are all those tables huge in size, if so what would be the size of the individual tables..
3. Calculate only the data size without the space occupied by indexes..

Once you have the above values identified, you can handle the Data Movement or Migration better by following the below guidelines:
1. Drop the Indexes on the Destination table to transfer data faster.
2. Disable any Foreign key constraints so that Data Movement would be faster.
3. Depending upon the individual record size, identify the optimal batch size for the tables.
4. Make sure the Destination SQL Server database is set to Bulk Recovery Mode or Simple Mode to reduce the Log file size increase..
5. Use SSIS or BCP to perform bulk load on the destination tables which would be faster.

As mentioned by lcohan, you can also use Database Migration Assistant for SQL Server which can help validate the data type and few other prerequisites for Migration..

Author

Commented:
Thanks to all

Do more with

Expert Office
Submit tech questions to Ask the Experts™ at any time to receive solutions, advice, and new ideas from leading industry professionals.

Start 7-Day Free Trial