Solved

How can i fix following HACMP error ?

Posted on 2009-04-10
3
2,898 Views
Last Modified: 2013-11-17
Dear all !
I have created the two node cluster on AIX (configuration file is attached ). Cluster has one resource group including 1 Service IP label and 1 volume group.
The status of the cluster as follow

bash-3.00# /usr/es/sbin/cluster/utilities/cldump

Obtaining information via SNMP from Node: node1...

_____________________________________________________________________________
Cluster Name: crmcluster
Cluster State: UP
Cluster Substate: STABLE
_____________________________________________________________________________

Node Name: node1                State: UP

  Network Name: net_diskhb_01      State: UP

    Address:                 Label: hdisk6             State: UP

  Network Name: net_ether_01       State: UP

    Address: 10.149.34.38    Label: vnpcrm05           State: UP

  Network Name: net_ether_02       State: UP

    Address: 1.1.1.1         Label: node1-boot         State: UP
    Address: 10.149.34.44    Label: vnpcluscrm         State: UP

Node Name: node2                State: UP

  Network Name: net_diskhb_01      State: DOWN


  Network Name: net_ether_01       State: UP

    Address: 10.149.34.39    Label: vnpcrm06           State: UP

  Network Name: net_ether_02       State: UP

    Address: 1.1.1.2         Label: node2-boot         State: UP


Cluster Name: crmcluster

Resource Group Name: test
Startup Policy: Online On First Available Node
Fallover Policy: Fallover To Next Priority Node In The List
Fallback Policy: Fallback To Higher Priority Node In The List
Site Policy: ignore
Primary instance(s):
The following node temporarily has the highest priority for this instance:
node1, user-requested rg_move performed on Wed Apr  8 16:42:18 2009

Node                         Group State      
---------------------------- ---------------
node1                        ONLINE          
node2                        OFFLINE


When I excute fallover test by using Shutdown Fr on one node, or using Moving resource group function on HACMP, it has done. But if I unplug the ethernet cable or the fibre cable, nothing happens.

Using Cluster Test Tool, I encouter some errors (some tests failed)


|| Test 1 Complete - NODE_UP: Start cluster services on all available nodes
||
08/04/2009_14:43:37: || Test Completion Status: NOT RATIONAL
         

|| Test 2 Complete - NETWORK_UP_GLOBAL: Bring up network1 globally
||
08/04/2009_14:52:23: || Test Completion Status: FAILED
||


|| Test 1 Complete - NETWORK_DOWN_GLOBAL: Bring down non-ip network
||
08/04/2009_14:54:01: || Test Completion Status: FATAL

How can i fix these errors?

cluster040909haw.doc
0
Comment
Question by:vinhbt
  • 2
3 Comments
 
LVL 68

Expert Comment

by:woolmilkporc
ID: 24121323
Hi,

there seems to be a problem with your disk heartbeat network net_diskhb_01.

Consider deleting and redefining it. When selecting hdisks, carefully compare the PVids (not the hdiskn names) to be sure to have the same disk defined at both ends. Best let HACMP discover the resources, then select the appropriate devices.

Additionally, please post the output of

lssrc -ls topsvcs

... and please explain the purpose of the IP interface 10.149.34.44   (vnpcluscrm). Seems a bit strange that it is a member of the same network as 1.1.1.1,  What are your netmask settings?

wmp


0
 

Author Comment

by:vinhbt
ID: 24213104
Hi woolmilkporc:
I have reconfigured HACMP with following change:
/etc/hosts
################HACMP IP ADDRESS##########################
192.168.1.1     vnpcrm05_boot
10.0.0.1        vnpcrm05_standby
10.149.34.38    vnpcrm05

192.168.1.2     vnpcrm06_boot
10.0.0.2        vnpcrm06_standby
10.149.34.39    vnpcrm06

10.149.34.44    vnpcrm_virtual

# ifconfig -a
en0: flags=5e080863,c0<UP,BROADCAST,NOTRAILERS,RUNNING,SIMPLEX,MULTICAST,GROUPRT,64BIT,CHECKSUM_OFFLOAD(ACTIVE),PSEG,LARGESEND,CHAIN>
        inet 192.168.1.1 netmask 0xffffffe0 broadcast 192.168.1.31
         tcp_sendspace 131072 tcp_recvspace 65536 rfc1323 0
en1: flags=5e080863,c0<UP,BROADCAST,NOTRAILERS,RUNNING,SIMPLEX,MULTICAST,GROUPRT,64BIT,CHECKSUM_OFFLOAD(ACTIVE),PSEG,LARGESEND,CHAIN>
        inet 10.0.0.1 netmask 0xffffffe0 broadcast 10.0.0.31
        inet 10.149.34.38 netmask 0xffffff00 broadcast 10.149.34.255
         tcp_sendspace 131072 tcp_recvspace 65536 rfc1323 0
lo0: flags=e08084b<UP,BROADCAST,LOOPBACK,RUNNING,SIMPLEX,MULTICAST,GROUPRT,64BIT>
        inet 127.0.0.1 netmask 0xff000000 broadcast 127.255.255.255
        inet6 ::1/0
         tcp_sendspace 131072 tcp_recvspace 131072 rfc1323 1
I also exported the configuration to the attach file (crmcluster.haw)
I still have the same problem. HACMP can not fail-over when I unplug the ethernet cable, unplug the fibre cable&
Running the cluster testtool, I get the log file (cl_testtool)
I excuted the command  lssrc ls  topsvcs and the output is
below:
lssrc -ls topsvcs
0513-036 The request could not be passed to the topsvcs subsystem.
Start the subsystem and try your command again.
Do  you have any ideal ?

cl-testtool.log
crmcluster.haw.txt
0
 
LVL 68

Accepted Solution

by:
woolmilkporc earned 500 total points
ID: 24213341
Hi,
lssrc -ls topsvcs works only when the cluster is up. But no problem, the test tool log contains that output.
Did you pull both ethernet cables? From your output I see that there is a standby configuration.
Did you unplug ethernet and fibre (SAN, I assume) at the same time? At least in that case a failover should occur!
Did you wait long enough? Check actual grace periods and failure detection rate with
/usr/es/sbin/cluster/utilities/cllsnim -g -n 'ether'  
or
/usr/es/sbin/cluster/utilities/cllsnim -g -n 'diskhb'
wmp
0

Featured Post

What Should I Do With This Threat Intelligence?

Are you wondering if you actually need threat intelligence? The answer is yes. We explain the basics for creating useful threat intelligence.

Join & Write a Comment

Suggested Solutions

Installing FreeBSD… FreeBSD is a darling of an operating system. The stability and usability make it a clear choice for servers and desktops (for the cunning). Savvy?  The Ports collection makes available every popular FOSS application and packag…
Using libpcap/Jpcap to capture and send packets on Solaris version (10/11) Library used: 1.      Libpcap (http://www.tcpdump.org) Version 1.2 2.      Jpcap(http://netresearch.ics.uci.edu/kfujii/Jpcap/doc/index.html) Version 0.6 Prerequisite: 1.      GCC …
Learn how to find files with the shell using the find and locate commands. Use locate to find a needle in a haystack.: With locate, check if the file still exists.: Use find to get the actual location of the file.:
In a previous video, we went over how to export a DynamoDB table into Amazon S3.  In this video, we show how to load the export from S3 into a DynamoDB table.

743 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

13 Experts available now in Live!

Get 1:1 Help Now