Go Premium for a chance to win a PS4. Enter to Win

x
  • Status: Solved
  • Priority: Medium
  • Security: Public
  • Views: 324
  • Last Modified:

Do a BCP import without growing the T-Log???

We do a BCP import into our database ever morning which causes a 250MB spike in our T-Logs.  The data that is getting imported is static, read-only data that is maintained on a separate server.  

Is there any way to prevent BCP from logging all of that traffic every morning so I won't have to back it up?  There is still other data hitting the database that is worth holding on to the T-logs, so I can't just turn T-logging off, or just delete the logs.

0
zclobes
Asked:
zclobes
1 Solution
 
namasi_navaretnamCommented:
Search for "logged bulk copy" in the MS SQL SERVER HELP FILE and you will see some useful info. Also see the link on that page "Optimizing Bulk Copy Performance".

It seems that logged cannot be completely turned off. But I am not sure.


Logged and Minimally Logged Bulk Copy Operations
When using the full recovery model, all row-insert operations performed by bcp are logged in the transaction log. For large data loads, this can cause the transaction log to fill rapidly. To help prevent the transaction log from running out of space, a minimally logged bulk copy can be performed if all of these conditions are met:

The recovery model is simple or bulk-logged.


The target table is not being replicated.


The target table does not have any triggers.


The target table has either 0 rows or no indexes.


The TABLOCK hint is specified. For more information, see Controlling the Locking Behavior.
Any bulk copy into an instance of Microsoft® SQL Server™ that does not meet these conditions is logged.

Before doing bulk copy operations, it is recommended that you set the recovery model to bulk-logged if you usually use full recovery. This will prevent the bulk copy operations from using excessive log space and possibly filling the log. However, even with bulk-logged recovery, some transaction log space will be used. You may want to create transaction log backups during the bulk copy operation to free up transaction log space.

When bulk copying a large number of rows into a table with indexes, it can be faster to drop all the indexes, perform the bulk copy, and re-create the indexes. For more information, see Optimizing Bulk Copy Performance.



Note  Although data insertions are not logged in the transaction log when a minimally logged bulk copy is performed, SQL Server still logs extent allocations each time a new extent is allocated to the table.

0
 
LowfatspreadCommented:
Hi zclobes,

have you considered importing/loading into a
separate "temporary table database" which youd don't "log"

and then  
running sql to actually determine if any data changes are present
and just performing the necessary updates...

should reduce your logging requirements if the data changes infrequently...

also
transactional replication may be an answer if again only infrequent changes are taking place...
(and you can live with near time updates..)

other scenarios would be possible..

 

Cheers!
0

Featured Post

Free Tool: Path Explorer

An intuitive utility to help find the CSS path to UI elements on a webpage. These paths are used frequently in a variety of front-end development and QA automation tasks.

One of a set of tools we're offering as a way of saying thank you for being a part of the community.

Tackle projects and never again get stuck behind a technical roadblock.
Join Now