Solved

bulk insert keeps blowing the log

Posted on 2008-10-01
11
316 Views
Last Modified: 2008-10-01
sql v2k, i bulk insert into a view in order to process some data, then push it to another table.  that content of that 'data' was changed last night, so i'm trying to bulk it into the table, ensure all formatting is correct, and then approve the same change to production

it keeps blowing the log file --
Server: Msg 9002, Level 17, State 6, Line 1
The log file for database 'databasename' is full. Back up the transaction log for the database to free up some log space.
Note: Bulk Insert through a view may result in base table default values being ignored for NULL columns in the data file.
The statement has been terminated.

this is just a dev box, and just a test file, not close to production capacity, but it's completely idle except for me, i've got 2G data, 2G log (ample size for this effort).   this particular @sql is a part of my production logic which is in place and has been running for years w/out failure.  until now.

i didn't have the TABLOCK or ROWS_PER_BATCH in there before, so I put it in, no difference, it keeps puking.  curious, is there any way to batch through this differently than I am now?

SELECT @sql =
'BULK INSERT dbo.viewname FROM ''' + @inputfile + ''' WITH (firstrow=2, FIELDTERMINATOR = '','', TABLOCK,ROWS_PER_BATCH=50000,ROWTERMINATOR = ''' + NCHAR(10) + ''')'
0
Comment
Question by:dbaSQL
  • 5
  • 4
  • 2
11 Comments
 
LVL 60

Expert Comment

by:chapmandew
ID: 22613991
backup the transaction log then shrink it...OR set your recovery model to SIMPLE

backup log dbname to disk = 'c:\logfile.bak'

dbcc shrinkfile('dbname_log', 0)
0
 
LVL 17

Author Comment

by:dbaSQL
ID: 22614003
sorry, chap, i should have provided more detail.  i have already done both of those, several times.  the db is in simple recovery mode
0
 
LVL 60

Accepted Solution

by:
chapmandew earned 250 total points
ID: 22614019
how much free space do you have in the drive where your trans log is located?  you may have to break your file up into smaller files for insert purposes.
0
 
LVL 17

Author Comment

by:dbaSQL
ID: 22614064
unfortunately, that's what i was thinking, too.  i'm down to 9mb on the data drive (it's dev, no panic :-) )
my logic as is, that's pretty much the only way to control the flow on the bulk insert, yes?
0
 
LVL 60

Expert Comment

by:chapmandew
ID: 22614081
If you're down to 9mb then that is your problem.
yes
0
Ransomware-A Revenue Bonanza for Service Providers

Ransomware – malware that gets on your customers’ computers, encrypts their data, and extorts a hefty ransom for the decryption keys – is a surging new threat.  The purpose of this eBook is to educate the reader about ransomware attacks.

 
LVL 142

Assisted Solution

by:Guy Hengel [angelIII / a3]
Guy Hengel [angelIII / a3] earned 250 total points
ID: 22614094
what recovery mode is your db?
if it's full, change to simple recovery mode.
0
 
LVL 17

Author Comment

by:dbaSQL
ID: 22614097
it is simple
0
 
LVL 142

Expert Comment

by:Guy Hengel [angelIII / a3]
ID: 22614147
2 questions:
* does the view have a INSTEAD OF trigger?
* what is the view code?
0
 
LVL 60

Expert Comment

by:chapmandew
ID: 22614173
This is odd...by default the BULK INSERT statement will not fire triggers unless you use the FIRE_TRIGGERS clause of the statement
0
 
LVL 17

Author Comment

by:dbaSQL
ID: 22614186
The view is pretty simple -- i just use it to load the table via bulk insert, w/columns in the file that don't match the table.

create view viewname as
select [OrderNo],[MsgSource],[MsgType],[MsgIndex],[TimestampDate],[TimestampMS] FROM Database.dbo.tablename
0
 
LVL 17

Author Comment

by:dbaSQL
ID: 22615707
smaller file, it works fine.  i will keep tablock and rows_per_batch in there, going forward
but i think it was just a matter of too much data for too small a resource (dev bed)

thank you both for looking
0

Featured Post

[Webinar] Disaster Recovery and Cloud Management

Learn from Unigma and CloudBerry industry veterans which providers are best for certain use cases and how to lower cloud costs, how to grow your Managed Services practice in IaaS clouds, and how to utilize public cloud for Disaster Recovery

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

JSON is being used more and more, besides XML, and you surely wanted to parse the data out into SQL instead of doing it in some Javascript. The below function in SQL Server can do the job for you, returning a quick table with the parsed data.
Load balancing is the method of dividing the total amount of work performed by one computer between two or more computers. Its aim is to get more work done in the same amount of time, ensuring that all the users get served faster.
Via a live example combined with referencing Books Online, show some of the information that can be extracted from the Catalog Views in SQL Server.
Via a live example, show how to set up a backup for SQL Server using a Maintenance Plan and how to schedule the job into SQL Server Agent.

911 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

22 Experts available now in Live!

Get 1:1 Help Now