Question

Cannot get Oracle to drop tablespaces and recognize that datafiles have been dropped

Asked by: groucho47

I have a slightly different problem, but along the way I got the same error. A dba-in-training was attempting to do some partition maintenance on one of our largest tables yesterday. It has 90 days worth of partitions on a rolling basis. When he went to drop the old partitions (older than 90 days) and add the new ones for next week, he made a procedural error (he accidentally did things completely out of order). So on my way in this morning, I got a call that the database was inaccessible. When I arrived and looked at the alert log, last evening we were getting
Wed May 28 22:57:38 2008
Errors in file /u01/app/oracle/admin/DWHPRD1/bdump/dwhprd1_ckpt_16904.trc:
ORA-01110: data file 99: '/f11/oracle/DWHPRD1/DAH080605.dbf'
ORA-01116: error in opening database file 99
ORA-01110: data file 99: '/f11/oracle/DWHPRD1/DAH080605.dbf'
ORA-27041: unable to open file
SVR4 Error: 2: No such file or directory
Additional information: 3
and soon afterward we got
Wed May 28 22:57:38 2008
CKPT: terminating instance due to error 1242
Wed May 28 22:57:38 2008
Errors in file /u01/app/oracle/admin/DWHPRD1/bdump/dwhprd1_pmon_16892.trc:
ORA-01242: data file suffered media failure: database in NOARCHIVELOG mode

The datafiles involved in this mess were being created for next weeks data, so they were empty. I started to use the old trick of "alter database datafile 'filename' offline drop" Oracle accepted the command, but my colleague failed to mention that he had already removed the physical files from the server. Oracle accepted the commands, but still shows the datafiles as being there. I was eventually able to restart the database, and it is open for business, but there is a lingering problem I do not know how to resolve. My colleague said he had dropped the partitions (each partition is in its own tablespace) but Oracle still reports that the tablespace, datafile, and partition are out there, as follows:

select tablespace_name, file_name, bytes, autoextensible, increment_by, maxbytes from dba_data_files
where tablespace_name like 'DAH%'
TSPACE                        FILE_NAME                                                            BYTES
DAH080601                     /f11/oracle/DWHPRD1/DAH080601.dbf
****************************  ******************************************************  ------------------
DAH080602                   /f11/oracle/DWHPRD1/DAH080602.dbf
****************************  ******************************************************  ------------------
DAH080603                     /f11/oracle/DWHPRD1/DAH080603.dbf
****************************  ******************************************************  ------------------
DAH080604                     /f11/oracle/DWHPRD1/DAH080604.dbf
****************************  ******************************************************  ------------------
DAH080605                     /f11/oracle/DWHPRD1/DAH080605.dbf
****************************  ******************************************************  ------------------
DAH080606                     /f11/oracle/DWHPRD1/DAH080606.dbf
****************************  ******************************************************  ------------------
DAH080607                     /f11/oracle/DWHPRD1/DAH080607.dbf
****************************  ******************************************************  ------------------
So Oracle still thinks the datafiles are there, but shows no size for them (there are no files on server). If I tell Oracle to do an offline drop of the files, it says operation completed, but still shows the files as existing. If I try to offline or drop the tablespaces for which no files exist, or drop the partitions, here is what I get.

SQLDW1>alter tablespace dah080607 offline;
alter tablespace dah080607 offline
*
ERROR at line 1:
ORA-01191: file 117 is already offline - cannot do a normal offline
ORA-01110: data file 117: '/f11/oracle/DWHPRD1/DAH080607.dbf'

SQLDW1>drop tablespace dah080607 including contents;
drop tablespace dah080607 including contents
*
ERROR at line 1:
ORA-14404: partitioned table contains partitions in a different tablespace

SQLDW1>alter table incentive.daily_account_history drop partition PART080607;
alter table incentive.daily_account_history drop partition PART080607
*
ERROR at line 1:
ORA-00376: file 117 cannot be read at this time
ORA-01110: data file 117: '/f11/oracle/DWHPRD1/DAH080607.dbf'

Well of course! It makes perfect sense! So, how can I work around this conundrum? I need to be able to get Oracle to accept commands that will drop the 7 tablespaces/datafiles/partitions involved. We do NOT want to have to drop and recreate the entire table from exports, it is very large and it would take a long time. It could also result in the premature demise of my colleague.

ORIGINAL RELATED ISSUE:
I have an application that errored out.

On checking out the logs I found the following log contents:

What is the problem and how can I rectify it?

SQL> set time on
23:13:35 SQL> set timing on
23:13:35 SQL> @/home/appmgr/PPGP/sfi/sql/sfi_drop.sql
23:13:35 SQL> set verify off
23:13:35 SQL> select distinct a.tablespace_name ,a.status
23:13:35   2  from dba_tablespaces a, dba_data_files b
23:13:35   3  where a.status <> 'READ ONLY'
23:13:35   4  and a.tablespace_name = b.tablespace_name
23:13:35   5  and b.file_name like '%/u391%'
23:13:35   6  order by tablespace_name
23:13:35   7  /

no rows selected

Elapsed: 00:00:00.04
23:13:35 SQL> drop tablespace SFI_DATA_Q1 including contents ;
drop tablespace SFI_DATA_Q1 including contents
*
ERROR at line 1:
ORA-14404: partitioned table contains partitions in a different tablespace



23:13:43 SQL> drop tablespace SFI_INDEX_Q9 including contents ;
drop tablespace SFI_INDEX_Q9 including contents
*
ERROR at line 1:
ORA-14405: partitioned index contains partitions in a different tablespace


Elapsed: 00:00:00.35


This Question has been solved and asker verified All Experts Exchange premium technology solutions are available to subscription members.

Subscribe now for full access to Experts Exchange and get

Instant Access to this Solution

  • Plus...
  • 30 Day FREE access, no risk, no obligation
  • Collaborate with the world's top tech experts
  • Unlimited access to our exclusive solution database
  • Never be left without tech help again

Subscribe Now

Asked On
2008-05-29 at 07:46:46ID23441575
Tags

ORA-14404

,

ORA-03376

Topics

Oracle Database

,

Oracle 8.x

Participating Experts
2
Points
100
Comments
3

Trusted by hundreds of thousands everyday for fast, accurate and reliable tech support.

  • "The time we save is the biggest benefit of Experts Exchange to Warner Bros. What could take multiple guys 2 hours or more each to find is accessed in around 15 minutes on Experts Exchange." Mike Kapnisakis, Warner Bros.
  • "Our team likes having a resource that is more secure than just using Google and most experts using this service really know their stuff. It's nice to look here first versus using Google." Dayna Sellner, Lockheed Martin
  • "Anytime that I've been stumped with a problem, 9 out of 10 times Experts Exchange has either the accepted solution or an open discussion of the potential solution to the problem." Kenny Red, eBay Inc.

See what Experts Exchange can do for you.

Got a question?

We've got the answer.

Experts Exchange has been collecting answers to technology questions since 1996…3 million and counting! If you have a question, chances are we already have your answer.

Screenshot of Experts Exchange Knowledgebase

Need individual assistance?

Our experts are ready to help.

If you can't find the exact answer you're looking for, ask our exclusive community of 50,000 experts. You’ll get a personalized answer from a trusted professional.

Screenshot of Experts Exchange Knowledgebase

Want to learn from the best?

Read articles from industry experts.

Thousands of free tech tips, tricks, how-to’s and tutorials are available in our peer reviewed articles section. See for yourself how smart our experts are, no login required.

Screenshot of an Article

Working on a long term project?

Store your work and research.

Save solutions to your questions, answers you’ve discovered through searching plus helpful articles in your personal knowledgebase for easy future access.

Screenshot of Experts Exchange Knowledgebase

Access the answers to your technology questions today.

Subscribe Now

30-day free trial. Register in 60 seconds.

What Makes Experts Exchange Unique?

Members of the expert community talk about why the experience at Experts Exchange is different than what you will find anywhere else.

Trusted by the world's most respected brands.

image of each brand's logo

Faithfully serving IT professionals since 1996.

Experts Exchange Logo

Try it out and discover for yourself.

Subscribe Now

30-day free trial. Register in 60 seconds.

Related Solutions

  1. Oracle DBA Responsibilities
    Hi, can anyone give me a list of responsibilities of Oracle DBA(Maintenance)? Can I also get any checklist sort of list that a DBA should monitor/Query about the database on a daily basis or in particular intervels to ensure that the database is running well or some interve...
  2. How to restore dbf file into tablespace in oracle
    Hi I have a dbf file which is to be restored in newly installed oracle server. How can i assign this dbf file to a new tablespace created. Thanks, Vipul Unadkat
  3. Oracle DBA Urgent Help Pls
    Hi all, I am not a oracle DBA but since our DBA has quit, I am trying to recover from this problem and would appreciate your help with this pls.. On one of this server, suddenly we got error as Error: [MERANT][ODBC Oracle 8 driver][Oracle 8]ORA-01034: ORACLE not available ...
  4. Oracle DBA - undo tablespace in RAID5
    Hi, Is it ok to keep UNDO Tablespace in RAID 5 or need to keep on internal disk? oracle 9i/sun solaires. what is difference between undo tablespace and redo logs?
  5. Tablespace import in Oracle
    Hi I have the following scenario..i have a tablespace 'x'... and i have around 4 data files wilth data in this tablespace..now i created 2 additional datafiles..immediately..i thought these additional files were not needed..so i wanted to remove it...I gave alter database &...
  6. Add Oracle Tablespace file
    I have an Oracle database 8.1.7 that I've inherited. The main production dbf is PROD01 There's been tablespace added in the past PROD02 and PROD03 PROD03 now only has 9mb available so I need to add tablespace. I'm using TOAD ...what is the commant to add PROD04.dbf ?

Free Tech Articles

  1. WARNING: 5 Reasons why you should NEVER fix a computer for free.
    It is in our nature to love the puzzle. We are obsessed. The lot of us. We love puzzles. We love the challenge. We thrive on finding the answer. We hate disarray. It bothers us deep in our soul. W...
  2. SCCM OSD Basic troubleshooting
    SCCM 2007 OSD is a fantastic way to deploy operating systems, however, like most things SCCM issues can sometimes be difficult to resolve due to the sheer volume of logs to sift through and the dispe...
  3. Migrate Small Business Server 2003 to Exchange 2010 and Windows 2008 R2
    This guide is intended to provide step by step instructions on how to migrate from Small Business Server 2003 to Windows 2008 R2 with Exchange 2010. For this migration to work you will need the fo...
  4. Create a Win7 Gadget
    This article shows you how to create a simple "Gadget" -- a sort of mini-application supported by Windows 7 and Vista. Gadgets can be dropped anywhere on the desktop to provide instant information, ...
  5. Outlook continually prompting for username and password
    There have been a lot of questions recently regarding Outlook prompting for a username and password whilst using Exchange 2007. There are a few reasons why this would happen and I will try to cover t...
  6. Backup Exchange 2010 Information Store using Windows Backup
    There seems to be quite a lot of confusion around the ability to backup Exchange 2010 using the built in Windows Backup feature. This stems from the omission of this feature prior to Exchange 2007 s...

Cloud Class Webinars

  1. Avoiding Bugs in Microsoft Access
    Alison Balter takes and in-depth look at avoiding bugs in Access. In this webinar you will learn about using the immediate window to debug your applications, invoking the debugger, using breakpoints to troubleshoot, stepping through code, setting the next statement to execute, ...
  2. Top 10 Best New Features in Visio 2010
    Scott Helmers gives live demonstrations of the top 10 new features in Visio 2010. This webinar will teach you how to create compelling diagrams by adding shapes to the page with a single click, linking the shapes in a diagram to data in Excel (or SQL Server, or SharePoint), ...
  3. IT Consultant Business Secrets Revealed
    Michael Munger, Experts Exchange tech pro and IT consultant, pulls back the curtain on his very successful businesses and answers question on every IT consultant and business owner should know about. He shares secrets on what he did to solve the 5 most common problems in IT, ...
  4. Disaster Recovery and Business Continuity
    Quest CTO, Mike Billon, gives an overview of the steps involved in building a dunamic disaster recovery plan. Through case studies and an examination of software/hardware tooles for monitoring and testing, you'll gain a better understandin of where you are, where you want ...
  5. Organize Your Visio Diagrams with Containers and Lists
    Scott Helmers uses cross functional flowcharts, wireframe diagrams, data graphic legends and seating charts to teach you: how to ustilize all three new structured diagram components in Visio 2010, the best practices for organizeing shapes in previous version of Visio, how to organize ...
  6. How to Us Objects, Properties, Events and Methods in Microsoft Access
    Alison Dalter gives an in-depbth look at objects, properties, events and methods in Microsoft Access. In this webinar you will learn about using the object browser, referring to objects, working with properties and methods, working with object variables, understanding the ...

Join the Community

Give a Little. Get a Lot.

Join the community of experts here and help other tech pros by answering question in your area of expertise. You can earn FREE access to all Experts Exchange's premium features and resources.

Join the Community

Answers

 

by: schwertnerPosted on 2008-05-29 at 08:11:07ID: 21669849

What about
drop tablespace SFI_DATA_Q1 including contents and datafile;

Also possibly you can put there empty dummy datafiles with the same name.
I hope that Oracle will not check the contents of that empty datafiles
because they are offline.
I am not sure, but you can give a try.

 

by: chedgeyPosted on 2008-05-29 at 09:59:47ID: 21671027

I had exactly the same problem last week (a rogue application developer who obtainted the root password and did some "tidying up"!!! of files including a datafile) so if Schwertner's solution does not work try the following which worked for me:

1. Shutdown the database;
2. Startup Mount
3. Alter Database Datafile '<File Name causing problem' OffLine Drop;
4. Alter Database Open;
5. Drop Tablespace <Tabspace Name> Including Contents;

I hope that helps

Regards

Chedgey

 

by: groucho47Posted on 2008-05-29 at 11:37:40ID: 21672010

Thanks for feedback, I tried both solutions with no luck. The table that was botched is a large partitioned table. I just sent my boss the details of how I got past this issue. Here goes:

The reason that the offline drop option and then drop the tablespace option did not work is that this is a large partitioned table, and my associate incorrectly did the partition maintenance in such a way that there were no datafiles on the OS to support the tablespaces. When I did an offline drop for each file, Oracle accepted the command but continued to show the datafile if you queried dba_data_files. If I tried to drop the tablespace it would not allow me because of the ora-14404 error. If I tried to drop the partitions contained in the tablespace that had no supporting datafile, I got a file read error. And around and around and around I went.

Finally, I stumbled across a solution. I created a dummy table dahtemp1 as select * from the source table with the problem tablespaces, but with a where clause that would assure an empty table. Then, one at a time, I exchanged the partitions with that dummy table. An example:

create table dahtemp1 as select * from daily_account_history where report_dt ='01-JAN-08'; ------------this would assure an exact dupe of the the table but with no rows, since there would be no data that old in the table.

Then,

alter  table incentive.daily_account_history exchange partition part080607 with table incentive.dahtemp1 without validation;

Having done that, Oracle allowed me to then drop the partition, and then the associated tablespace. I did that repetitively for each of the 7 tablespaces/partitions that had no supporting datafile. The only errors I got during the process were like this:

drop tablespace dah080604 including contents
Thu May 29 13:17:33 2008
Completed: drop tablespace dah080604 including contents
Thu May 29 13:18:33 2008
Some indexes or index [sub]partitions of table INCENTIVE.DAILY_ACCOUNT_HISTORY have been marked unusable

So, I just finished using the old handy-dandy analyze index validate structure command to see if the primary key (local) index is structurally sound. It did not return any errors.

Maybe that will help future visitors who get an ora-14404 on a large partitioned tablespace with tablespaces that have no supporting datafiles but where the partitions in each tablespace were never dropped!!!! Yikes.


20120131-EE-VQP-002

3 Ways to Join

30-Day Free Trial

The Experts

98% positive feedback on 31,087 answers since March 2000. angeliii is a Microsoft Most Valuable Professional for his work with MS SQL Server & Develoment.

He has also proven his knowledge of Visual Basic Programming, PHP Scripting and Oracle Databases.

The Experts

97% positive feedback on 10,752 answers since July 2000. lrmoore has more than 18 years experience in the networking industry.

The six-time Mircosoft MVPs specialties include firewalls, virtual private networking, and network management.

Testimonials

"...and excellent source for support... Kind of like having your very own IT dept." Electriciansnet

Testimonials

"I was apprehensive at signing up at first. However... it has already made my life as an IT administrator much easier." JaCrews

Testimonials

"WOW! You guys have great, active, and knowledgeable people on here." moore50

Business Clients

Business Clients

In the Press

"If you’ve got a question... Experts Exchange can supply an answer.”

In the Press

"...an invaluable aid for both IT professionals and those who require tech support."

In the Press

"where IT professionals provide quick answers on just about any topic"

Business Account Plans

Loading Advertisement...