DB2 LUW HADR primary did not establish connection with standby within timeout and will shut down. BY FORCE option required to start primary without standby.

Hi!


Check for HADR errors in the db2diag.log on both standby and primary.
Also check if you have configured the HADR parameters correctly.

In some cases when HADR communications between servers are in strange state I usually
stop HADR on both primary and standby servers. Then restart HADR in the correct order that is first the standby then the primary.
http://www-01.ibm.com/support/docview.wss?uid=swg21410648
https://www.ibm.com/developerworks/community/wikis/home?lang=en#!/wiki/DB2HADR/page/HADR%20Tutorial

Regards,
    Tomas Helgi

i stopped and re-started in correct order. but still having the same issue.

I am also seeing below in the DIAG log:

RETCODE : ZRC=0x8280001A=-2105540582=HDR_ZRC_NO_STANDBY
          "Comm time-out in unforced HADR primary start, to avoid split-brain"

I am also seeing below in the DIAG log:

 MESSAGE : HADR EDU sqlcode:
DATA #1 : Hexdump, 4 bytes
0x0780000003CD5C1C : FFFF F918

Hi!

Is the standby in rollforward pending mode or standard ?
issue db2 get db cfg for yourdbname | grep "Rollforward pending" it should say NO if HADR standby is running correctly.

Check if there is any firewall blocking your HADR ports.
Which OS are you running DB2 on ?

Regards,
     Tomas Helgi

I am using AIX

Yes,  Rollforward pending                                     = NO

on stand BY DB

Hi!

Have you checked if firewall is blocking the HADR ports between primary and standby ?
Also what is your HADR_TIMEOUT value ? db2 get db cfg for yourdbname | grep "HADR_TIMEOUT" ?

Regards,
    Tomas Helgi

HADR timeout value                       (HADR_TIMEOUT) = 120
 HADR log write synchronization mode     (HADR_SYNCMODE) = NEARSYNC
 HADR peer window duration (seconds)  (HADR_PEER_WINDOW) = 300

I could not check the port availability

netstat -an | grep 50051

$ netstat -an | grep db2c_db2inst1_hadr

I am not retrieving any output on primary or standby

does this mean it is port issue?

Hi!

You could also try to stop the HADR on both servers, then issue db2stop/db2start on the standby and restart the HADR.
If your HADR configuration is correct and this doesn't work then the the firewall is the most likely cause.
What version is your DB2 ?
Regards,
    Tomas Helgi

Version 9.7

I did the re-start of HADR and checked, it is not starting.

SQL1768N  Unable to start HADR. Reason code = "7".

facing the above error while starting in primary

can you help me how to check the firewall issue and
How to check whether a port if free or not.

$netstat -an | grep 50051

$ netstat -an | grep db2c_db2inst1_hadr

These commands are not retrieving any output.  

I am using above commands to check whether a port is free or not.

can you pls correct me if i am wrong.

Hi!

Please check this out https://www-304.ibm.com/support/docview.wss?uid=swg21460503 
Post both the HADR config for primary and standby ( db2 get db cfg for yourdbname | grep HADR )
Use ping and telnet commands on both servers to find if the ports are open and they can talk to each other on the HADR ports.

Regards,
    Tomas helgi

Configuration parameters are good.
It was working previously. To-day it is not working.

Is this command correct : telnet Ipaddress 50050

is this command correct ?

Hi!

Yes this command is correct.

Regards,
     Tomas Helgi

Ports are good, How to check the last or latest transaction log file processed or replayed in stand by DB?

DO I need to check in archlogs path in stand by server?

Thanks Tomas for your help.