Solved

Copying large records takes too lonk

Posted on 2013-05-15
2
193 Views
Last Modified: 2013-06-13
Hi SQL Server Guru's.

I have a project that is copying the data in a table (this include an image so, these records a large (anywhere from 12 to 30 K per row).   And there are about 1million records.  
SELECT * INTO TEMP_TABLE FROM IMAGE_TABLE
This process is taking about 24 hours.  Another table with a relatively small row size copies in a few minutes.

I'm guessing that this is because it can only grab so much data in the copy and has to "make multiple trips back to the source" per record.  But, that's just a guess.

If that's close to accurate, is there a way to adjust the memory caching in an insert so that the more data  can be copied per packet?

Or, if I'm way off the mark, can someone explain why this takes so long?

Hope someone can help.
0
Comment
Question by:MikeDelaney
2 Comments
 
LVL 39

Expert Comment

by:lcohan
ID: 39169681
I suggest use SSIS Data transfer task to do it and you must see at least some improvement.
You can do just a quick test by using the wizzard from SSMS to do a EXPORT or IMPORT.

Also think that image data(and text/ntext) is stored differently than regular data types in SQL plus I bet if you look at your database T-log/MDF files there must be some (significant) growth during this transfer.
0
 
LVL 69

Accepted Solution

by:
ScottPletcher earned 500 total points
ID: 39172036
I suspect it's log file growth/expansion that is really killing your performance.

You need to pre-allocate enough log space to handle the entire INSERT.

Also, using BULK INSERT might give you better performance and would definitely give you better control over how the table is loaded: batch size, do triggers fire, etc..
0

Featured Post

What Should I Do With This Threat Intelligence?

Are you wondering if you actually need threat intelligence? The answer is yes. We explain the basics for creating useful threat intelligence.

Join & Write a Comment

I wrote this interesting script that really help me find jobs or procedures when working in a huge environment. I could I have written it as a Procedure but then I would have to have it on each machine or have a link to a server-related search that …
Having an SQL database can be a big investment for a small company. Hardware, setup and of course, the price of software all add up to a big bill that some companies may not be able to absorb.  Luckily, there is a free version SQL Express, but does …
Viewers will learn how to use the SELECT statement in SQL to return specific rows and columns, with various degrees of sorting and limits in place.
Viewers will learn how to use the UPDATE and DELETE statements to change or remove existing data from their tables. Make a table: Update a specific column given a specific row using the UPDATE statement: Remove a set of values using the DELETE s…

757 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

23 Experts available now in Live!

Get 1:1 Help Now