Link to home
Start Free TrialLog in
Avatar of themeeper1
themeeper1

asked on

Oracle sqlplus error "ORA-12547: TNS:lost contact" after applying opatch on RAC environment

Hello,

I'm currently running Oracle 11.2.0 Oracle RAC on AIX 6.1TL7. This is a 2 node cluster on a shared file system. I recently applied the patch 14275621 to the database and when I try to bring up the database after the patch on sqlplus / as sysdba. I get the following error:


oracle/db11gR2/bin/oracle: 0403-006 Execute permission denied.
ERROR:
ORA-12547: TNS:lost contact


Prior to starting the patch. I shutdown the database and ASM service but didn't stop the cluster. This patch has been applied with no issues on my single instance developement databases.

Have any of you experts seen or encountered this issue? It might be because when I kicked off the opatch apply on server #1 and finished it asked me if I wanted to apply to server #2 even though it was a shared file system? I'm not sure either. Any help is appreciated!!
Avatar of johnsone
johnsone
Flag of United States of America image

Based on the error message, it would appear to me that the file permissions on $ORACLE_HOME/bin/oracle are incorrect.  Are you sure you ran all the steps outlined in the patch?
Avatar of themeeper1
themeeper1

ASKER

Yes, I followed the same steps as I did for my developement single instance(working) environments. Someone suggested doing a relink all under oracle home. Does that sound like it would resolve this?
After the patch apply, did you check whether it changed the sticky bit for oracle binary?
In the sense, if you do ls -l oracle/db11gR2/bin/oracle, what do you see there?
is it set to 6751?  
want to ruleout any possibility of that.
Based on the error, I would guess it is the permission on the oracle binary that is the issue.  A relink would not correct that.  You can set the permission the same as the other environments that are working, but I would be concerned that something was missed or got an error that you are ignoring.
I did a ls -l of the oracle/db11gR2/bin/oracle and i see the following. -rwsr-s--x

It doesn't look like the sticky bit is there (denoted by a small t?) But I checked my other environment that was patched/working and it shows the same.

Is there a way to rollback a patch if I don't have access to sqlplus?
to rollback a patch, we use $ORACLE_HOME/OPatch/bin/opatch rollback -id <patch id>.
But, please wait, still trying to understand what caused this?
is this a production system and did you raise an SR asking the same Q?
As what OS user you are logging in?
And when you do ls -l $ORACLE_HOME/bin/sqlplus, who owns that and the group?
If you are not logging as the owner, do you belong to the group?
And do that group have execute permissions?

I'm asking in a more elaborated way for the comment ID: 38600371 posted by johnsone.
I'm logged in as the owner of oracle. The permissioning is correct.
What is the size of the file "$ORACLE_HOME/rdbms/lib/config.o" ?
And I hope the instance is currently down.  If yes, just issue a "relink all".
Also do you have your $ORACLE_HOME/dbs directory intact ?
ASKER CERTIFIED SOLUTION
Avatar of Chakravarthi Ayyala
Chakravarthi Ayyala
Flag of United States of America image

Link to home
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
Start Free Trial
Thanks for the udpate.
Could you please share with us, what was the step/item that you performed that worked for you?