Solved

Do a BCP import without growing the T-Log???

Posted on 2003-11-01
3
299 Views
Last Modified: 2008-03-03
We do a BCP import into our database ever morning which causes a 250MB spike in our T-Logs.  The data that is getting imported is static, read-only data that is maintained on a separate server.  

Is there any way to prevent BCP from logging all of that traffic every morning so I won't have to back it up?  There is still other data hitting the database that is worth holding on to the T-logs, so I can't just turn T-logging off, or just delete the logs.

0
Comment
Question by:zclobes
3 Comments
 
LVL 15

Expert Comment

by:namasi_navaretnam
ID: 9663419
Search for "logged bulk copy" in the MS SQL SERVER HELP FILE and you will see some useful info. Also see the link on that page "Optimizing Bulk Copy Performance".

It seems that logged cannot be completely turned off. But I am not sure.


Logged and Minimally Logged Bulk Copy Operations
When using the full recovery model, all row-insert operations performed by bcp are logged in the transaction log. For large data loads, this can cause the transaction log to fill rapidly. To help prevent the transaction log from running out of space, a minimally logged bulk copy can be performed if all of these conditions are met:

The recovery model is simple or bulk-logged.


The target table is not being replicated.


The target table does not have any triggers.


The target table has either 0 rows or no indexes.


The TABLOCK hint is specified. For more information, see Controlling the Locking Behavior.
Any bulk copy into an instance of Microsoft® SQL Server™ that does not meet these conditions is logged.

Before doing bulk copy operations, it is recommended that you set the recovery model to bulk-logged if you usually use full recovery. This will prevent the bulk copy operations from using excessive log space and possibly filling the log. However, even with bulk-logged recovery, some transaction log space will be used. You may want to create transaction log backups during the bulk copy operation to free up transaction log space.

When bulk copying a large number of rows into a table with indexes, it can be faster to drop all the indexes, perform the bulk copy, and re-create the indexes. For more information, see Optimizing Bulk Copy Performance.



Note  Although data insertions are not logged in the transaction log when a minimally logged bulk copy is performed, SQL Server still logs extent allocations each time a new extent is allocated to the table.

0
 
LVL 50

Accepted Solution

by:
Lowfatspread earned 125 total points
ID: 9665008
Hi zclobes,

have you considered importing/loading into a
separate "temporary table database" which youd don't "log"

and then  
running sql to actually determine if any data changes are present
and just performing the necessary updates...

should reduce your logging requirements if the data changes infrequently...

also
transactional replication may be an answer if again only infrequent changes are taking place...
(and you can live with near time updates..)

other scenarios would be possible..

 

Cheers!
0

Featured Post

Migrating Your Company's PCs

To keep pace with competitors, businesses must keep employees productive, and that means providing them with the latest technology. This document provides the tips and tricks you need to help you migrate an outdated PC fleet to new desktops, laptops, and tablets.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Suggested Solutions

Introduction SQL Server Integration Services can read XML files, that’s known by every BI developer.  (If you didn’t, don’t worry, I’m aiming this article at newcomers as well.) But how far can you go?  When does the XML Source component become …
This article explains how to reset the password of the sa account on a Microsoft SQL Server.  The steps in this article work in SQL 2005, 2008, 2008 R2, 2012, 2014 and 2016.
Using examples as well as descriptions, and references to Books Online, show the documentation available for date manipulation functions and by using a select few of these functions, show how date based data can be manipulated with these functions.
Via a live example, show how to backup a database, simulate a failure backup the tail of the database transaction log and perform the restore.

685 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question