• Status: Solved
  • Priority: Medium
  • Security: Public
  • Views: 3297
  • Last Modified:

ORA-15205 when starting RAC database (asm disks shown as dropped)

ORA-15205: requested mirror side unavailable
cause the RAC database to fail at startup.

In ASM, two disks (without any redundancy) are shown as "_DROPPED_". They belong to DATA and FRA main diskgroups.

[root@bidegi01 ~]# fdisk -l | grep sd
Disco /dev/sda: 21.4 GB, 21474836480 bytes
/dev/sda1   *           1          13      104391   83  Linux
/dev/sda2              14        2610    20860402+  8e  Linux LVM
Disco /dev/sdb: 32.2 GB, 32212254720 bytes
/dev/sdb1   *           1        3916    31455238+  8e  Linux LVM
Disco /dev/sdc: 1073 MB, 1073741824 bytes
/dev/sdc1               2        1886     1047552   83  Linux
Disco /dev/sdd: 1073 MB, 1073741824 bytes
/dev/sdd1               2        1886     1047552   83  Linux
Disco /dev/sde: 1073 MB, 1073741824 bytes
/dev/sde1               2        1886     1047552   83  Linux
Disco /dev/sdf: 21.4 GB, 21474836480 bytes
/dev/sdf1               1        2611    20970496   83  Linux
Disco /dev/sdg: 21.4 GB, 21474836480 bytes
/dev/sdg1               1        2611    20970496   83  Linux
Disco /dev/sdh: 21.4 GB, 21474836480 bytes
/dev/sdh1               1        2611    20970496   83  Linux
Disco /dev/sdi: 21.4 GB, 21474836480 bytes
/dev/sdi1               1        2611    20970496   83  Linux


SQL> select path, name from v$asm_disk;

PATH                           NAME
------------------------------ ------------------------------
ORCL:ASM_DATA02
ORCL:ASM_DATA03
ORCL:OCR_VOTE01
                               _DROPPED_0001_DATOS
                               _DROPPED_0000_FRA
ORCL:ASM_DATA01                ASM_DATA01
ORCL:ASM_DATA04                ASM_DATA04
ORCL:OCR_VOTE02                OCR_VOTE02
ORCL:OCR_VOTE03                OCR_VOTE03

9 rows selected.

Open in new window




[grid@bidegi02 bin]$ ./crs_stat -t
Name           Type           Target    State     Host
------------------------------------------------------------
ora.DATOS.dg   ora....up.type ONLINE    ONLINE    bidegi01
ora.FRA.dg     ora....up.type ONLINE    ONLINE    bidegi01
ora....ER.lsnr ora....er.type ONLINE    ONLINE    bidegi01
ora....N1.lsnr ora....er.type ONLINE    ONLINE    bidegi02
ora....N2.lsnr ora....er.type ONLINE    OFFLINE
ora....N3.lsnr ora....er.type ONLINE    ONLINE    bidegi01
ora....VOTE.dg ora....up.type ONLINE    ONLINE    bidegi01
ora.asm        ora.asm.type   ONLINE    ONLINE    bidegi01
ora....SM1.asm application    ONLINE    ONLINE    bidegi01
ora....01.lsnr application    ONLINE    ONLINE    bidegi01
ora....i01.gsd application    OFFLINE   OFFLINE
ora....i01.ons application    ONLINE    ONLINE    bidegi01
ora....i01.vip ora....t1.type ONLINE    ONLINE    bidegi01
ora....SM2.asm application    ONLINE    ONLINE    bidegi02
ora....02.lsnr application    ONLINE    ONLINE    bidegi02
ora....i02.gsd application    OFFLINE   OFFLINE
ora....i02.ons application    ONLINE    ONLINE    bidegi02
ora....i02.vip ora....t1.type ONLINE    ONLINE    bidegi02
ora.bret.db    ora....se.type ONLINE    OFFLINE
ora.cvu        ora.cvu.type   ONLINE    ONLINE    bidegi01
ora.gsd        ora.gsd.type   OFFLINE   OFFLINE
ora....network ora....rk.type ONLINE    ONLINE    bidegi01
ora.oc4j       ora.oc4j.type  ONLINE    ONLINE    bidegi02
ora.ons        ora.ons.type   ONLINE    ONLINE    bidegi01
ora....ry.acfs ora....fs.type ONLINE    ONLINE    bidegi01
ora.scan1.vip  ora....ip.type ONLINE    ONLINE    bidegi02
ora.scan2.vip  ora....ip.type ONLINE    OFFLINE
ora.scan3.vip  ora....ip.type ONLINE    ONLINE    bidegi01

Open in new window


Starting database manually causes the following error:

SQL> startup
ORACLE instance started.

Total System Global Area  839282688 bytes
Fixed Size                  2231128 bytes
Variable Size             633341096 bytes
Database Buffers          197132288 bytes
Redo Buffers                6578176 bytes
Database mounted.
ORA-15205: requested mirror side unavailable

Open in new window



I've tried to shutdown all CRS and startup all the services from scratch, but no success.

[root@bidegi02 ~]#  /etc/init.d/oracleasm scandisks
Scanning the system for Oracle ASMLib disks:               [  OK  ]
[root@bidegi02 ~]#  /etc/init.d/oracleasm listdisks
ASM_DATA01
ASM_DATA02
ASM_DATA03
ASM_DATA04
OCR_VOTE01
OCR_VOTE02
OCR_VOTE03

Open in new window


Any solution to startup the database?
0
Javier Morales
Asked:
Javier Morales
  • 4
3 Solutions
 
Javier MoralesOracle DBAAuthor Commented:
Also, when I try to restart the asm, I get the following error:


ORA-15097: cannot SHUTDOWN ASM instance with connected client (process 31682)
CRS-2675: Fallo al parar ''ora.asm'' en ''bidegi02''

La parada de `ora.OCR_VOTE.dg` en el miembro `bidegi01` se ha realizado correctamente.
Intentando parar `ora.asm` en el miembro `bidegi01`
CRS-2673: Intentando parar ''ora.asm'' en ''bidegi01''
CRS-5017: La acción de recurso "ora.asm stop" ha detectado el siguiente error:
ORA-15097: cannot SHUTDOWN ASM instance with connected client (process 11592)
CRS-2675: Fallo al parar ''ora.asm'' en ''bidegi01''

CRS-0216: Could not stop resource 'ora.asm bidegi01 1'.

CRS-0216: Could not stop resource 'ora.asm bidegi02 1'.

CRS-0216: Could not stop resource 'ora.gsd'.

CRS-0216: Could not stop resource 'ora.gsd bidegi01 1'.

CRS-0216: Could not stop resource 'ora.gsd bidegi02 1'.

CRS-0216: Could not stop resource 'ora.ons bidegi02 1'.

Open in new window

0
 
Praveen Kumar ChandrashekatrDatabase Analysist Senior Commented:
did you dropped disk groups?

if it is not droped completly you can try undrop the diskgroup.

ALTER DISKGROUP dgroup1 UNDROP disks;

It clear states that there are active client connected to the ASM instance, plz query V$ASM_CLIENT or use ASMCMD LSOF to list active clients. kill that process and restart the asm instance

you can also use shutdown abort for your ASM instance.
0
 
Javier MoralesOracle DBAAuthor Commented:
Hi Praveencpk, Thanks a lot for your answer!

I didn't dropped the diskgroups, It seems that ASM dropped the disks by itself.
Anyway, the dropping isn't finish yet, and disks can't be dropped.

SQL> alter diskgroup datos drop disk '_DROPPED_0001_DATOS';
alter diskgroup datos drop disk '_DROPPED_0001_DATOS'
*
ERROR at line 1:
ORA-15032: not all alterations performed
ORA-15071: ASM disk "_DROPPED_0001_DATOS" is already being dropped

Open in new window


I followed metalink's note "
ALTER DISKGROUP DROP DISK FAILS WITH ORA-15071 ASM DISK IS ALREADY BEING DROPPED [ID 1066335.1]" that also recommend to undrop disks and retry dropping, and get the same error ORA-15071

lsof shows no active clients and there's now only one ASM instance started (to avoid blocking processes).

I shutted down ASM instance, but when I started it again, the issue remains the same.
0
Ultimate Tool Kit for Technology Solution Provider

Broken down into practical pointers and step-by-step instructions, the IT Service Excellence Tool Kit delivers expert advice for technology solution providers. Get your free copy now.

 
Richard OlutolaConsultantCommented:
You cannot stop ASM with 'ordinary' methods because OCR and voting disks are highly likely held on diskgroups on the ASM. Hence you cannot stop ASM without stopping the clusterware stack.

To stop the clusterware stack use the following:

crsctl stop cluster

R.
0
 
Javier MoralesOracle DBAAuthor Commented:
Hi,

@rolutola, I finally could stop the cluster, but ASM instance couldn't start because corruption on the OCR files. Once recovered from backup, I'm still facing the same issues with dg:

SQL>  select name, path from v$asm_disk;

NAME                           PATH
------------------------------ ------------------------------
                               ORCL:ASM_DATA02
                               ORCL:ASM_DATA03
                               ORCL:OCR_VOTE01
                               ORCL:OCR_VOTE02
                               ORCL:ASM_DATA04
_DROPPED_0001_OCR_VOTE
_DROPPED_0001_DATOS
ASM_DATA01                     ORCL:ASM_DATA01
OCR_VOTE03                     ORCL:OCR_VOTE03

9 rows selected.

Open in new window

The point is that actually, I finally could open the database, but I can't manage the _DROPPED_ disks and can't explain what happened with disk labeling.

My database is now open, but the ASM issue is still there...

@praveencpk, I did shutdown abort the ASM database, and it caused me the OCR corruption.
[root@bidegi02 bin]# ./ocrcheck
Status of Oracle Cluster Registry is as follows :
         Version                  :          3
         Total space (kbytes)     :     262120
         Used space (kbytes)      :       2900
         Available space (kbytes) :     259220
         ID                       : 1164751242
         Device/File Name         :  +OCR_VOTE
                                    Device/File integrity check failed
                                    Device/File not configured
                                    Device/File not configured
                                    Device/File not configured
                                    Device/File not configured
         Cluster registry integrity check failed
         Logical corruption check bypassed due to insufficient quorum

Open in new window


As I previous mention, I could restore the OCR and start the clusterware.

[root@bidegi02 bin]# ./ocrcheck
Status of Oracle Cluster Registry is as follows :
         Version                  :          3
         Total space (kbytes)     :     262120
         Used space (kbytes)      :       2932
         Available space (kbytes) :     259188
         ID                       : 1164751242
         Device/File Name         :  +OCR_VOTE
                                    Device/File integrity check succeeded
                                    Device/File not configured
                                    Device/File not configured
                                    Device/File not configured
                                    Device/File not configured
         Cluster registry integrity check succeeded
         Logical corruption check succeeded

Open in new window


I guess shutdown abort ASM instance when OCR or other clients are locking dg's is quite dangerous...

The point is ... how may I solve the ASM disks labeling and offline state?
0
 
Javier MoralesOracle DBAAuthor Commented:
Shutdown abort of the ASM instance in 11.2.0.2 causes the OCR may get corrupted. I had to restore OCR and reboot nodes.

With old OCR, rebooting the nodes, seemed to solve the issue with no additional tasks.

I split the points just to thank you the effort to answer my question. Thanks a lot.
0
Question has a verified solution.

Are you are experiencing a similar issue? Get a personalized answer when you ask a related question.

Have a better answer? Share it in a comment.

Join & Write a Comment

Featured Post

Cloud Class® Course: MCSA MCSE Windows Server 2012

This course teaches how to install and configure Windows Server 2012 R2.  It is the first step on your path to becoming a Microsoft Certified Solutions Expert (MCSE).

  • 4
Tackle projects and never again get stuck behind a technical roadblock.
Join Now