Solved

Copying large records takes too lonk

Posted on 2013-05-15
2
232 Views
Last Modified: 2013-06-13
Hi SQL Server Guru's.

I have a project that is copying the data in a table (this include an image so, these records a large (anywhere from 12 to 30 K per row).   And there are about 1million records.  
SELECT * INTO TEMP_TABLE FROM IMAGE_TABLE
This process is taking about 24 hours.  Another table with a relatively small row size copies in a few minutes.

I'm guessing that this is because it can only grab so much data in the copy and has to "make multiple trips back to the source" per record.  But, that's just a guess.

If that's close to accurate, is there a way to adjust the memory caching in an insert so that the more data  can be copied per packet?

Or, if I'm way off the mark, can someone explain why this takes so long?

Hope someone can help.
0
Comment
Question by:MikeDelaney
[X]
Welcome to Experts Exchange

Add your voice to the tech community where 5M+ people just like you are talking about what matters.

  • Help others & share knowledge
  • Earn cash & points
  • Learn & ask questions
2 Comments
 
LVL 40

Expert Comment

by:lcohan
ID: 39169681
I suggest use SSIS Data transfer task to do it and you must see at least some improvement.
You can do just a quick test by using the wizzard from SSMS to do a EXPORT or IMPORT.

Also think that image data(and text/ntext) is stored differently than regular data types in SQL plus I bet if you look at your database T-log/MDF files there must be some (significant) growth during this transfer.
0
 
LVL 69

Accepted Solution

by:
Scott Pletcher earned 500 total points
ID: 39172036
I suspect it's log file growth/expansion that is really killing your performance.

You need to pre-allocate enough log space to handle the entire INSERT.

Also, using BULK INSERT might give you better performance and would definitely give you better control over how the table is loaded: batch size, do triggers fire, etc..
0

Featured Post

Forrester Webinar: xMatters Delivers 261% ROI

Guest speaker Dean Davison, Forrester Principal Consultant, explains how a Fortune 500 communication company using xMatters found these results: Achieved a 261% ROI, Experienced $753,280 in net present value benefits over 3 years and Reduced MTTR by 91% for tier 1 incidents.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Suggested Solutions

Title # Comments Views Activity
VM SQL server license. 1 67
SQL query 45 41
Delete duplicates from SQL Server table 2 27
Group ordersum by orderdate 3 20
Introduction In my previous article (http://www.experts-exchange.com/Microsoft/Development/MS-SQL-Server/SSIS/A_9150-Loading-XML-Using-SSIS.html) I showed you how the XML Source component can be used to load XML files into a SQL Server database, us…
This article explains how to reset the password of the sa account on a Microsoft SQL Server.  The steps in this article work in SQL 2005, 2008, 2008 R2, 2012, 2014 and 2016.
This video shows, step by step, how to configure Oracle Heterogeneous Services via the Generic Gateway Agent in order to make a connection from an Oracle session and access a remote SQL Server database table.
Viewers will learn how to use the UPDATE and DELETE statements to change or remove existing data from their tables. Make a table: Update a specific column given a specific row using the UPDATE statement: Remove a set of values using the DELETE s…

751 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question