JPNeron
asked on
Why won't Oracle start?
We use Oracle intermittently on a development platform, and today when I went to connect, I saw it wasn't running. I'm far from a expert in managing Oracle, I know just barely enough to get it going, and today the commands I usually use don't work. My notes say to log in as user oracle, then:
lsnrctl start
dbstart
The listener seems to start, but the dbstart command fails with a useless error message. Here's what happens:
$ lsnrctl start
LSNRCTL for IBM/AIX RISC System/6000: Version 10.2.0.1.0 - Production on 28-MAY-
2009 10:22:15
Copyright (c) 1991, 2005, Oracle. All rights reserved.
Starting /oracle/app/oracle/oracle/ product/10 .2.0/db_1/ bin/tnslsn r: please wait.
..
TNSLSNR for IBM/AIX RISC System/6000: Version 10.2.0.1.0 - Production
System parameter file is /oracle/app/oracle/oracle/ product/10 .2.0/db_1/ network/a
dmin/listener.ora
Log messages written to /oracle/app/oracle/oracle/ product/10 .2.0/db_1/ network/lo
g/listener.log
Listening on: (DESCRIPTION=(ADDRESS=(PRO TOCOL=ipc) (KEY=EXTPR OC1)))
Listening on: (DESCRIPTION=(ADDRESS=(PRO TOCOL=tcp) (HOST=bigg erblue.can sys.mb.ca)
(PORT=1521)))
Connecting to (DESCRIPTION=(ADDRESS=(PRO TOCOL=IPC) (KEY=EXTPR OC1)))
STATUS of the LISTENER
------------------------
Alias LISTENER
Version TNSLSNR for IBM/AIX RISC System/6000: Version 10.2.0.1
.0 - Production
Start Date 28-MAY-2009 10:22:15
Uptime 0 days 0 hr. 0 min. 50 sec
Trace Level off
Security ON: Local OS Authentication
SNMP ON
Listener Parameter File /oracle/app/oracle/oracle/ product/10 .2.0/db_1/ network/
admin/listener.ora
Listener Log File /oracle/app/oracle/oracle/ product/10 .2.0/db_1/ network/
log/listener.log
Listening Endpoints Summary...
(DESCRIPTION=(ADDRESS=(PRO TOCOL=ipc) (KEY=EXTPR OC1)))
(DESCRIPTION=(ADDRESS=(PRO TOCOL=tcp) (HOST=bigg erblue.can sys.mb.ca) (PORT=1521 ))
)
Services Summary...
Service "PLSExtProc" has 1 instance(s).
Instance "PLSExtProc", status UNKNOWN, has 1 handler(s) for this service...
Service "orcl" has 1 instance(s).
Instance "orcl", status UNKNOWN, has 1 handler(s) for this service...
The command completed successfully
$ dbstart
Failed to auto-start Oracle Net Listene using /ade/vikrkuma_new/oracle/b in/tnslsnr
The path name in this error message does not exist on our server, and has never existed on our server.
Help!
Jean
lsnrctl start
dbstart
The listener seems to start, but the dbstart command fails with a useless error message. Here's what happens:
$ lsnrctl start
LSNRCTL for IBM/AIX RISC System/6000: Version 10.2.0.1.0 - Production on 28-MAY-
2009 10:22:15
Copyright (c) 1991, 2005, Oracle. All rights reserved.
Starting /oracle/app/oracle/oracle/
..
TNSLSNR for IBM/AIX RISC System/6000: Version 10.2.0.1.0 - Production
System parameter file is /oracle/app/oracle/oracle/
dmin/listener.ora
Log messages written to /oracle/app/oracle/oracle/
g/listener.log
Listening on: (DESCRIPTION=(ADDRESS=(PRO
Listening on: (DESCRIPTION=(ADDRESS=(PRO
(PORT=1521)))
Connecting to (DESCRIPTION=(ADDRESS=(PRO
STATUS of the LISTENER
------------------------
Alias LISTENER
Version TNSLSNR for IBM/AIX RISC System/6000: Version 10.2.0.1
.0 - Production
Start Date 28-MAY-2009 10:22:15
Uptime 0 days 0 hr. 0 min. 50 sec
Trace Level off
Security ON: Local OS Authentication
SNMP ON
Listener Parameter File /oracle/app/oracle/oracle/
admin/listener.ora
Listener Log File /oracle/app/oracle/oracle/
log/listener.log
Listening Endpoints Summary...
(DESCRIPTION=(ADDRESS=(PRO
(DESCRIPTION=(ADDRESS=(PRO
)
Services Summary...
Service "PLSExtProc" has 1 instance(s).
Instance "PLSExtProc", status UNKNOWN, has 1 handler(s) for this service...
Service "orcl" has 1 instance(s).
Instance "orcl", status UNKNOWN, has 1 handler(s) for this service...
The command completed successfully
$ dbstart
Failed to auto-start Oracle Net Listene using /ade/vikrkuma_new/oracle/b
The path name in this error message does not exist on our server, and has never existed on our server.
Help!
Jean
ASKER CERTIFIED SOLUTION
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
Hi mark, dbstart is provided by Oracle to auto-start instances using the UNIX init script and /etc/oratab
It is included in the bin directory, and is the standard way to start Oracle automatically on UNIX systems.
I have been changing that path above for it seems like years, and you are correct, it is one of those internal leaks.
It is called by /etc/init.d/dbora
You can then use it and the whole sysv run levels to stop/start Oracle instances in /etc/oratab.
/etc/init.d/dbora stop
/etc/init.d/dbora start
At bootup, it will usually start at run-level 5, but again, you have to install dbora in /etc/init.d and make a link to the appropriate /etc/rc.d/* directory for runlevel.
It is included in the bin directory, and is the standard way to start Oracle automatically on UNIX systems.
I have been changing that path above for it seems like years, and you are correct, it is one of those internal leaks.
It is called by /etc/init.d/dbora
You can then use it and the whole sysv run levels to stop/start Oracle instances in /etc/oratab.
/etc/init.d/dbora stop
/etc/init.d/dbora start
At bootup, it will usually start at run-level 5, but again, you have to install dbora in /etc/init.d and make a link to the appropriate /etc/rc.d/* directory for runlevel.
Its one of those things us UNIX guys have to do manually that Windows Oracle DBCA does out of the box by installing the service or using oradim.exe
ASKER
mrjoltcola,
Sure enough, that path was in the dbstart script. What I don't understand is that Oracle used to run on this machine, and I never changed that path before. Why does it suddenly matter now?
I changed the path and reran dbstart. No error messges but I still can't connect to Oracle. I'm rebooting the machine just to make sure I'm at a clean start.
Jean
Sure enough, that path was in the dbstart script. What I don't understand is that Oracle used to run on this machine, and I never changed that path before. Why does it suddenly matter now?
I changed the path and reran dbstart. No error messges but I still can't connect to Oracle. I'm rebooting the machine just to make sure I'm at a clean start.
Jean
Likely Oracle has been started manually until now?
If you cannot connect to Oracle it does not mean Oracle is not up.
Once you restart, verify that the processes are running.
# Check for instances
ps -ef | grep smon
# Check the listener
lsnrctl status
The listener should start on its own with dbstart, however, it may be that you have don't have a static listener config, so if the listener was started _after_ the database, the instance will not be registered. This is why I almost always manually configure listener.ora so the order of startup does not matter.
Please verify the above, then show us how you are connecting before restarting again. No need to reboot the kernel.
If you cannot connect to Oracle it does not mean Oracle is not up.
Once you restart, verify that the processes are running.
# Check for instances
ps -ef | grep smon
# Check the listener
lsnrctl status
The listener should start on its own with dbstart, however, it may be that you have don't have a static listener config, so if the listener was started _after_ the database, the instance will not be registered. This is why I almost always manually configure listener.ora so the order of startup does not matter.
Please verify the above, then show us how you are connecting before restarting again. No need to reboot the kernel.
ASKER
The ps -ef|grep smon shows nothing is running. lsnrctl status shows:
LSNRCTL for IBM/AIX RISC System/6000: Version 10.2.0.1.0 - Production on 28-MAY-2009 11:28:00
Copyright (c) 1991, 2005, Oracle. All rights reserved.
Connecting to (DESCRIPTION=(ADDRESS=(PRO TOCOL=IPC) (KEY=EXTPR OC1)))
STATUS of the LISTENER
------------------------
Alias LISTENER
Version TNSLSNR for IBM/AIX RISC System/6000: Version 10.2.0.1.0 - Production
Start Date 28-MAY-2009 11:26:27
Uptime 0 days 0 hr. 1 min. 33 sec
Trace Level off
Security ON: Local OS Authentication
SNMP ON
Listener Parameter File /oracle/app/oracle/oracle/ product/10 .2.0/db_1/ network/ad min/listen er.ora
Listener Log File /oracle/app/oracle/oracle/ product/10 .2.0/db_1/ network/lo g/listener .log
Listening Endpoints Summary...
(DESCRIPTION=(ADDRESS=(PRO TOCOL=ipc) (KEY=EXTPR OC1)))
(DESCRIPTION=(ADDRESS=(PRO TOCOL=tcp) (HOST=bigg erblue.can sys.mb.ca) (PORT=1521 )))
Services Summary...
Service "PLSExtProc" has 1 instance(s).
Instance "PLSExtProc", status UNKNOWN, has 1 handler(s) for this service...
Service "orcl" has 1 instance(s).
Instance "orcl", status UNKNOWN, has 1 handler(s) for this service...
The command completed successfully
To check Oracle, I've been using the web interface, either the isqlplus interface or the Oracle Enterprise Manager. I usually start the OEM via:
emctl start dbconsole
This shows:
Oracle Enterprise Manager 10g Database Control Release 10.2.0.1.0
Copyright (c) 1996, 2005 Oracle Corporation. All rights reserved.
http://thsnmb01dc1-192-150.dynamic.mts.net:1158/em/console/aboutApplication
Starting Oracle Enterprise Manager 10g Database Control ....................
And it's been sitting there like that for 10+ minutes. Trying to connect to OEM via Firefox just hangs. Trying to connect via http://biggerblue:5560/isqlplus/ just gives a "Firefox can't establish a connection to the server at biggerblue:5560"
Jean
LSNRCTL for IBM/AIX RISC System/6000: Version 10.2.0.1.0 - Production on 28-MAY-2009 11:28:00
Copyright (c) 1991, 2005, Oracle. All rights reserved.
Connecting to (DESCRIPTION=(ADDRESS=(PRO
STATUS of the LISTENER
------------------------
Alias LISTENER
Version TNSLSNR for IBM/AIX RISC System/6000: Version 10.2.0.1.0 - Production
Start Date 28-MAY-2009 11:26:27
Uptime 0 days 0 hr. 1 min. 33 sec
Trace Level off
Security ON: Local OS Authentication
SNMP ON
Listener Parameter File /oracle/app/oracle/oracle/
Listener Log File /oracle/app/oracle/oracle/
Listening Endpoints Summary...
(DESCRIPTION=(ADDRESS=(PRO
(DESCRIPTION=(ADDRESS=(PRO
Services Summary...
Service "PLSExtProc" has 1 instance(s).
Instance "PLSExtProc", status UNKNOWN, has 1 handler(s) for this service...
Service "orcl" has 1 instance(s).
Instance "orcl", status UNKNOWN, has 1 handler(s) for this service...
The command completed successfully
To check Oracle, I've been using the web interface, either the isqlplus interface or the Oracle Enterprise Manager. I usually start the OEM via:
emctl start dbconsole
This shows:
Oracle Enterprise Manager 10g Database Control Release 10.2.0.1.0
Copyright (c) 1996, 2005 Oracle Corporation. All rights reserved.
http://thsnmb01dc1-192-150.dynamic.mts.net:1158/em/console/aboutApplication
Starting Oracle Enterprise Manager 10g Database Control ....................
And it's been sitting there like that for 10+ minutes. Trying to connect to OEM via Firefox just hangs. Trying to connect via http://biggerblue:5560/isqlplus/ just gives a "Firefox can't establish a connection to the server at biggerblue:5560"
Jean
SOLUTION
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
SOLUTION
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
SOLUTION
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
ASKER
The auto start setting in oratab was "N", so I changed it to "Y". I connected via isqlplus and started the database manually. That all seemed to go well. Tried to start OEM via 'emctl start dbconsole', but that hung. Not sure if previous attempt at starting buggered something up. Now trying a 'emctl stop dbconsole' to shut it down & then will try restarting it.
>>The auto start setting in oratab was "N", so I changed it to "Y"
So that will take affect next time you use the auto-start/stop (/etc/init.d/dbora and dbstart), but to be clear for beginners, it has no effect on manual start/stop via sqlplus. Please see mark's post above. I am not too familiar with EM troubleshooting, I am a command line DBA, sorry. Maybe mark can help you with EM.
You should at least take this opportunity to confirm and test the auto-start script again while the works are all fuzzed up.
So that will take affect next time you use the auto-start/stop (/etc/init.d/dbora and dbstart), but to be clear for beginners, it has no effect on manual start/stop via sqlplus. Please see mark's post above. I am not too familiar with EM troubleshooting, I am a command line DBA, sorry. Maybe mark can help you with EM.
You should at least take this opportunity to confirm and test the auto-start script again while the works are all fuzzed up.
I'm not an expert on OEM either. Even though much of my Oracle experience has been with Oracle on Windows, I'm mainly a command-line DBA as well. I like the visiblity and the control that the command-line interface offers.
Good, then I'm in good company. :)
@JPNeron: I have a startup script for emctl start dbconsole _after_ both my database and TNS listener are started. That is how it is on all my boxes. You might want to open up a new question specifically related to OEM, schwertner is pretty good with it if I recall, better than us apparently :)
@JPNeron: I have a startup script for emctl start dbconsole _after_ both my database and TNS listener are started. That is how it is on all my boxes. You might want to open up a new question specifically related to OEM, schwertner is pretty good with it if I recall, better than us apparently :)
ASKER
Thanks for all the help!. I'll open a new question on the EM, if/when I have the time to follow up on it. Oracle is running now, and that's what I need today.
Jean
Jean
I recognize "lsnrctl start" as a standard Oracle command to start Oracle's TNS listener. The other command: "dbstart" does not look like a standard Oracle command to me. This may be a shell script that someone at your organization wrote. (Or, maybe it is standard for Oracle on AIX, since I have no experience with that O/S.)
What is "dbstart"? Is that a shell script file? If yes, try running the individual commands in the file one at a time to find the line with the actual error. Can you post the contents of the dbstart file here?