Solved

Oracle internal error -  SMON was doing temporary segment drop

Posted on 2004-04-16
8
2,368 Views
Last Modified: 2012-08-13
I am in big trouble friends. This is the error.

Errors in file /u01/appl/oracle/admin/UTL/bdump/utl_smon_11837.trc:
ORA-00600: internal error code, arguments: [5120], [256], [1920], [68], [12], [], [],
[]
Non-fatal internal error happenned while SMON was doing temporary segment drop.
SMON encountered 10 out of maximum 100 non-fatal internal errors.

Once the error reach 100, my PRODUCTION instance is stopped.

I am able to open it after a shutdown, but error continues.

Am running Oracle9i Enterprise Edition Release 9.0.1.0.0 - Production on Solaris 8, 64-bit.

Please help. Thanks.
Richard.
0
Comment
Question by:rkayondo
8 Comments
 
LVL 1

Expert Comment

by:ora-dba
ID: 10840566
Hello Richard,
one possibility for this error is block corruption. Please run the dbverify utility to check each datafile.
0
 
LVL 47

Expert Comment

by:schwertner
ID: 10841112
ORA-600 in many cases leads to Oracle bugs.
Is this a fresh installation?
Applying the last patches can help -there are too many bugs.
Or migrating to 9.2.


Oracle is sensitive to the Unix environment.
You have to be very carefull about kernel parameters, swap space, etc.
Also check the settings of the Solaris Kernel Parameters:

SEMMNI 100
SEMMNS 256
SEMMSL 256
SHMMAX 4294967295
SHMMIN 1
SHMMNI 100
SHMSEG 10





OS SPECIFIC COMMANDS
-----------------------------------------
To Find Swap, RAM, and OS Version
OS      SWAP      RAM      OS VERSION
AIX      /usr/sbin/lsps -a      /usr/sbin/lsattr -HE -l sys0 -a realmem      oslevel
HP      swapinfo -q      dmesg | grep -i mem       uname -a
Tru64      swapon -s      vmstat -P       /usr/sbin/sizer -v
Solaris      swap -s      /usr/sbin/prtconf | grep -i memory      uname -r
Linux      free      free      uname -a
To Determine the Unix Kernel Parameters
OS      LOCATION      COMMAND TO SEARCH
AIX      automatically configured      do 'env' for LINK_CNTRL, To determine if AIX is 64 bit enabled do 'genkex | grep 64' or 'genkex | grep call' (see <Note:1019106.102>)
HP      /stand/system      /etc/sysdef or /usr/sbin/kmtune
Tru64      /etc/sysconfigtab      /sbin/sysconfig -q ipc or /sbin/sysconfig -q vm or /sbin/sysconfig -q proc
Solaris       /etc/system      "/etc/sysdef | grep SHM" or "/etc/sysdef | grep SEM"
Linux      /usr/src/linux/include/asm/shmparam.h /usr/src/linux/include/linux/sem.h /proc/sys/kernel/sem /proc/sys/kernel/shmall /proc/sys/kernel/shmmax /proc/sys/kernel/shmmni       ipcs -lms
O/S Software Installed Packages and Patches
OS      COMMAND FOR PACKAGES      COMMAND FOR PATCHES
AIX      lslpp -w | grep -i "software title" (applies to APARs and PTFs)      /usr/sbin/instfix -ia -ivk patch number
HP      /usr/sbin/swlist -l fileset | grep -i "software title"       /usr/sbin/swlist -l fileset | grep -i
Solaris      /bin/pkginfo -l | grep -i "software title"      /bin/showrev -p
Tru64      /usr/sbin/setld -i | grep -i "software title"      /usr/sbin/setld -i | moreFor patchkits:/usr/sbin/dupatch -track -type kit
Linux      rpm -qa      
To Check if an OS is 64 bit Capable or not
OS      COMMAND      RESULTS
Aix      lslpp -L | grep 64bit      It should return "bos.64bit"
HP      getconf KERNEL_BITS      It should return "64"
Solaris      /bin/isainfo -kv      If the isainfo command does not exist it is not 64-bit. It should return "64-bit sparcv9 kernel modules"
To Check version of Compiler
COMPILER      COMMAND
C/C++      do a "which cc", then cd to the directory it is in, then do a "what cc" except for Linux, on Linux do a "rpm -qa | grep egcs
COBOL      cd $COBDIR and do "more cobver"
Fortran      do a "which f77", then cd to directory it is in, then do a "what f77"


Sun Patches

32bit/64bit
SUNWarc,
SUNWbtool,
SUNWhea,
SUNWlibm,
SUNWlibms,
SUNWsprot,
SUNWtoo
Additional
64bit
SUNWarcx,
SUNW2x (believe this is a typo and should be SUNWtoox)

108528-01 (2.8)
105181-15 (2.6)

0
 

Author Comment

by:rkayondo
ID: 10841408
Thanks for all the help, however I haven't tried working on the problem yet because am doing a cold backup of the database.
I realised from the backup of my control file that tempfiles entries could not be found. Which makes me to think that the problem is with temporary tablespace.
I will give you update after restore of temporary tablespace.

rgds,
Richard
0
PRTG Network Monitor: Intuitive Network Monitoring

Network Monitoring is essential to ensure that computer systems and network devices are running. Use PRTG to monitor LANs, servers, websites, applications and devices, bandwidth, virtual environments, remote systems, IoT, and many more. PRTG is easy to set up & use.

 
LVL 3

Expert Comment

by:dbms_chu
ID: 10841998
rkayondo,

Have you tried using the ORA-600 Tool on metalink?  Search for Note:153788.1 on metalink.

0
 

Author Comment

by:rkayondo
ID: 10876676
Thanks all for your help. I managed to solve this problem and here is the solution.
The key hint to the problem was the message from the SMON ---  Non-fatal internal error happenned while SMON was doing temporary segment drop---
 I though that the TEMP tablespace had corrupt segments, so I dropped it and constructed a new temp tablespace.  This didnot work! I had to check on all tablespaces to look for one which could have had a corrupt segment.
So, I put all the tablespaces in read only mode and the SMON error stopped---this gave me lots of hope!
I shutdown the database ( with tablespaces in read only mode) and restarted it normally. Error were not coming any more, but my tablespaces where in read only mode. So I put them in read write one by one, and everything looked fine with nor SMON errors.
So I shutdown the database normally and started it normally...uuuuchhh, errors where thrown again!!
This was a hint to a currupt segment in one of my tablespaces  that is was being written to at startup and causing errors.
I needed to find this tablespaces and probably try a repair.
This is how I found the corrupt tablespace.
1) . I put one  of tablespaces in readonly mode and did a restart of the database and then monitored the SMON error.
2). I put back the tablespace in readwrite mode if the error continued to appear.

3). Go back to 1). untill error stopped on a particular tablespace!

This tablespace contained the corrupt segment. I tried repairing it using DBMS_SPACE_ADMIN  but failed. I reran dbverify command on all datafiles but didnot help. So I export all objects in the tablespace, dropped the tablespace, created a new one and imported the objects back.
That worked nicely and no data was lost.
Thanks once again. Problem solved.

Richard.




0
 
LVL 1

Expert Comment

by:ora-dba
ID: 10876746
Thank you for the feedback, I'm glad you resolved the problem. So it was block corruption after all.
0
 
LVL 1

Accepted Solution

by:
Computer101 earned 0 total points
ID: 12279755
PAQed, with points refunded (500)

Computer101
E-E Admin
0

Featured Post

PRTG Network Monitor: Intuitive Network Monitoring

Network Monitoring is essential to ensure that computer systems and network devices are running. Use PRTG to monitor LANs, servers, websites, applications and devices, bandwidth, virtual environments, remote systems, IoT, and many more. PRTG is easy to set up & use.

Join & Write a Comment

Have you ever had to make fundamental changes to a table in Oracle, but haven't been able to get any downtime?  I'm talking things like: * Dropping columns * Shrinking allocated space * Removing chained blocks and restoring the PCTFREE * Re-or…
Using SQL Scripts we can save all the SQL queries as files that we use very frequently on our database later point of time. This is one of the feature present under SQL Workshop in Oracle Application Express.
This video shows how to recover a database from a user managed backup
This video explains what a user managed backup is and shows how to take one, providing a couple of simple example scripts.

707 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

14 Experts available now in Live!

Get 1:1 Help Now