[Last Call] Learn how to a build a cloud-first strategyRegister Now

x
?
Solved

ORA 27070 followed by OS 04016

Posted on 2010-01-06
15
Medium Priority
?
2,375 Views
Last Modified: 2012-05-08
Hi,

I have been facing "i/O" errors trying to import data into my Oracle 10g database.
Initially the database files were hosted on a SAN volume, so I assumed it was a SAN issue.
So I switched to working on an eSATA drive. Same error.
So now... I am working on the INTERNAL drive (DELL PERC 6/i SCSI Disk Device) ! Same error!

The consistent error on the command prompt, and in the alert log is:

KCF: write/open error block=0x1289 online=1
     file=2 D:\ORACLEDATABASEFILES\AM791\UNDOTBS01.DBFX
     error=27070 txt: 'OSD-04016: Error queuing an asynchronous I/O request.
O/S-Error: (OS 31) A device attached to the system is not functioning.'

Wed Jan 06 12:30:11 2010
Errors in file d:\oracledatabasefiles\am791\admin\bdump\am791_dbw1_5572.trc:
ORA-01242: data file suffered media failure: database in NOARCHIVELOG mode
ORA-01114: IO error writing block to file 2 (block # 4745)
ORA-01110: data file 2: 'D:\ORACLEDATABASEFILES\AM791\UNDOTBS01.DBFX'
ORA-27070: async read/write failed
OSD-04016: Error queuing an asynchronous I/O request.
O/S-Error: (OS 31) A device attached to the system is not functioning.

I know that the DB in in noarchivelog mode, and the inability to recover doesn't bother me as much as the fact that the undo / system tablespace files just become invisible to my Oracle process. I can the files in windows explorer, they are right there!

What should I be looking at?
0
Comment
Question by:DallasDAF
  • 5
  • 4
  • 4
  • +1
15 Comments
 
LVL 47

Expert Comment

by:David
ID: 26194179
Well, if any other program reported this issue, then the conclusion would be that the write is failing on block #4745, which means the target disk has error on that block, and the disk is out of spares, so it can't remap it.

I think in your situation, there is a bug in the program, and you are having a READ error in the file, which is root cause why it can't write.   physical block #4745 in input file is the problem.

Copy the file and rename it, using command-line, and see if it gives READ error.  This will confirm everything.


0
 
LVL 78

Expert Comment

by:slightwv (䄆 Netminder)
ID: 26194240
Please explain what you mean by:  the undo / system tablespace files just become invisible to my Oracle process

At a high level it looks like you copied over a corrupt datafile.  Just continuing to copy if from disk to disk doesn't help.  It's corrupt to Oracle not the disk itself.
0
 

Author Comment

by:DallasDAF
ID: 26194344
@dlethe:
I create the database using the Oracle DBCA, no other program is using the file.
Also, this is the n-th time I have seen this error, and each time, it was a different block number.

@slightwv:
What I mean by 'undo/system tablespace becoming invisible' is that  I see ORA-01114, ORA-01110 and ORA-27070 errors, and the process is not able to write to / read from the files.

Also, sometimes it is the system01.dbfx in th error message, and sometimes it is undotbs01.dbfx.

As for copying the file, I haven't copied it. Created the DB using DBCA, and was importing data into it when I got this error.
0
What does it mean to be "Always On"?

Is your cloud always on? With an Always On cloud you won't have to worry about downtime for maintenance or software application code updates, ensuring that your bottom line isn't affected.

 
LVL 78

Expert Comment

by:slightwv (䄆 Netminder)
ID: 26194464
Thanks for the clarification.

If you've seen these types of messages across several disk sub-systems, the problem might be with the OS drivers.

I checked with our sys admin and 2008 changed some async IO API's.  MSoft claims the old APIs will work but the preferred is something called overlapping async (I think that's what he said).

Just had a thought:
I'm not sure 10g is certified on Windows 2008.  I know 11g has a specific version for 2008.

This might be a general incompatibility issue.
0
 

Author Comment

by:DallasDAF
ID: 26194513
I have another server with 10g running on 2008 with no issues at all!

In the mean time, what APIs should I be looking to fix/replace?

0
 
LVL 78

Expert Comment

by:slightwv (䄆 Netminder)
ID: 26194619
Don't know that you can 'fix' APIs.  I'm not a windows admin type but since you moved across 3 disk subsystems the problem has to be with Oracle or the OS itself.

With the message "O/S-Error: (OS 31) A device attached to the system is not functioning.", I'm leaning towards a problem with OS.

0
 

Author Comment

by:DallasDAF
ID: 26194682
:) Logical conclusion. But given all these errors, my sys admins re-imaged the machine for me just a while ago.

Is there an Oracle patch I should be looking at? Want to be super-sure that the issue is not with the Oracle installation.
0
 
LVL 78

Expert Comment

by:slightwv (䄆 Netminder)
ID: 26194729
I don't know of a specific patch for this error.  I've used Oracle for 20+ years and I can't ever remember seeing those particular errors.

I would say that you should be at least on the same version of the machine that isn't causing you any problems.

Personally, I like to stay 1 one patchset behind unless the current one fixes a specific problem I'm having.
0
 
LVL 11

Accepted Solution

by:
it-rex earned 2000 total points
ID: 26195981
DallasDAF,
this usually happens when there is an OS DB locking conflict;
the most common I have seen this as;is with Veritas netbackup that has the locking option on.
when we did a weekly backup we used to have this error every week.

NOW the Q is how frequent do you get this these errors like every night every week?
what other apps run on the same server other than the DB?
what backup app do you use on this server?
0
 
LVL 47

Expert Comment

by:David
ID: 26196015
If this was a transport-level bug, you would think it would complain about everything, regardless of the file name.  

Any chance you are dancing on the fence with the 2TB limit, or mixing source/target with MBR / GPT formatting?
0
 
LVL 11

Expert Comment

by:it-rex
ID: 26196253
it could be an antivirus too ;
that caused this locking.
but 75% it is a third party app locking the datafile.
0
 

Author Comment

by:DallasDAF
ID: 26306445
I was using hyperbac for oracle (hence the extension  dbfx of the database files) for online compression. Turned off the hyperbac services and then created a regular oracle database, no I/O errors for a week now.
0
 
LVL 11

Expert Comment

by:it-rex
ID: 26306496
that is what I have told you @ ID: 26196253
0
 

Author Comment

by:DallasDAF
ID: 26308732
it-rex: I do not know for sure if it was a locking issue or the plain fact that Hyperbac for Oracle did not deliver on its online feature. Have you seen prior cases where Hyperbac for Oracle caused the lock?
0
 
LVL 11

Expert Comment

by:it-rex
ID: 26308841
as I have told you refer to metalink;
they explain that lots of third party apps will cause this issue like backup apps especially if they do encryption.
there is not test case for your particular application;same happend with me ;
Oracle was talking about seagate agent for backup;while our issue was with veritas netbackup after enabling encryption.
0

Featured Post

Has Powershell sent you back into the Stone Age?

If managing Active Directory using Windows Powershell® is making you feel like you stepped back in time, you are not alone.  For nearly 20 years, AD admins around the world have used one tool for day-to-day AD management: Hyena. Discover why.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

This post first appeared at Oracleinaction  (http://oracleinaction.com/undo-and-redo-in-oracle/)by Anju Garg (Myself). I  will demonstrate that undo for DML’s is stored both in undo tablespace and online redo logs. Then, we will analyze the reaso…
A procedure for exporting installed hotfix details of remote computers using powershell
This video shows syntax for various backup options while discussing how the different basic backup types work.  It explains how to take full backups, incremental level 0 backups, incremental level 1 backups in both differential and cumulative mode a…
This tutorial will walk an individual through setting the global and backup job media overwrite and protection periods in Backup Exec 2012. Log onto the Backup Exec Central Administration Server. Examine the services. If all or most of them are stop…
Suggested Courses

834 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question