tonypatton21
asked on
Insert and Index Overhead
I am working on a file import of 43 million records on a table that is truncated and reloaded with this import each week. There is an index on one of the fields. If I drop the index and then reload the table , and re-create the index, the process finishes in 10 mins. If I don't drop the index first and try to insert, the process doesn't finish in an acceptable timeframe. Where is all the extra overhead for the insert when the index is left there? Is this purely a disk bottleneck or more than that?
Last Comment
ASKER CERTIFIED SOLUTION
THIS SOLUTION IS ONLY AVAILABLE TO MEMBERS.
View this solution by signing up for a free trial.
Members can start a 7-Day free trial and enjoy unlimited access to the platform.
dqmq
Also, it it's a clustered index and the import is out-of-sequence, you are continuosly rearranging the data pages during the load.
ASKER
Yes, I changed the recovery model to bulk logged and the bulk insert is functioning pretty well. So should I leave the import as-is where I programmatically drop index, insert data, and rebuild index?
Yes, unless you have some compelling reason not to.
Microsoft SQL Server 2005
Microsoft SQL Server 2005 is a suite of relational database management system (RDBMS) products providing multi-user database access functionality.Component services include integration (SSIS), reporting (SSRS), analysis (SSAS), data quality, master data, T-SQL and performance tuning. It includes support for managing XML data and allows a database server to be exposed over web services using Tabular Data Stream (TDS) packets encapsulated within SOAP (protocol) requests.
72K
Questions
--
Followers
--
Top Experts
Get a personalized solution from industry experts
TRUSTED BY
