Solved

Copying large records takes too lonk

Posted on 2013-05-15
2
220 Views
Last Modified: 2013-06-13
Hi SQL Server Guru's.

I have a project that is copying the data in a table (this include an image so, these records a large (anywhere from 12 to 30 K per row).   And there are about 1million records.  
SELECT * INTO TEMP_TABLE FROM IMAGE_TABLE
This process is taking about 24 hours.  Another table with a relatively small row size copies in a few minutes.

I'm guessing that this is because it can only grab so much data in the copy and has to "make multiple trips back to the source" per record.  But, that's just a guess.

If that's close to accurate, is there a way to adjust the memory caching in an insert so that the more data  can be copied per packet?

Or, if I'm way off the mark, can someone explain why this takes so long?

Hope someone can help.
0
Comment
Question by:MikeDelaney
2 Comments
 
LVL 39

Expert Comment

by:lcohan
ID: 39169681
I suggest use SSIS Data transfer task to do it and you must see at least some improvement.
You can do just a quick test by using the wizzard from SSMS to do a EXPORT or IMPORT.

Also think that image data(and text/ntext) is stored differently than regular data types in SQL plus I bet if you look at your database T-log/MDF files there must be some (significant) growth during this transfer.
0
 
LVL 69

Accepted Solution

by:
Scott Pletcher earned 500 total points
ID: 39172036
I suspect it's log file growth/expansion that is really killing your performance.

You need to pre-allocate enough log space to handle the entire INSERT.

Also, using BULK INSERT might give you better performance and would definitely give you better control over how the table is loaded: batch size, do triggers fire, etc..
0

Featured Post

Simplifying Server Workload Migrations

This use case outlines the migration challenges that organizations face and how the Acronis AnyData Engine supports physical-to-physical (P2P), physical-to-virtual (P2V), virtual to physical (V2P), and cross-virtual (V2V) migration scenarios to address these challenges.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Suggested Solutions

Title # Comments Views Activity
SQL STANDARD CORE 7 31
Powershell v3 - SQLCMD 3 26
Solar Winds can't see SQL Server Express 17 32
Run an action on recently added records to a table 13 61
Nowadays, some of developer are too much worried about data. Who is using data, who is updating it etc. etc. Because, data is more costlier in term of money and information. So security of data is focusing concern in days. Lets' understand the Au…
The Delta outage: 650 cancelled flights, more than 1200 delayed flights, thousands of frustrated customers, tens of millions of dollars in damages – plus untold reputational damage to one of the world’s most trusted airlines. All due to a catastroph…
This video shows, step by step, how to configure Oracle Heterogeneous Services via the Generic Gateway Agent in order to make a connection from an Oracle session and access a remote SQL Server database table.
Using examples as well as descriptions, and references to Books Online, show the documentation available for date manipulation functions and by using a select few of these functions, show how date based data can be manipulated with these functions.

856 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question