Solaris 10 IPMP failover behaviour and RAC interconnect restrictions

Posted on 2009-04-21
Last Modified: 2013-12-21
We are trying to do link state IPMP for Solaris 10 on 3 database servers running oracle RAC. we have 3 networks on 2 subnets 10..x.139.0/24 and 10.x.138.0/24
10.x.138.0 = net2
10.x.139.0 = net3, and net4
net4 is oracle interconnect, only oracle interconnect traffic should go over this(these) interface(s).

-no active probing of the IPMP interfaces is happening (nor can we use it)
-Currently all traffic is going out net3 (nxge4)
-also on failover testing, traffic will default to lowest interface (nxge2) which is bad.
-Tried subnetting in /etc/netmasks file, no luck.
-we've reviewed a number of docs, just missing the last bit of magic

Q) How do we separate traffic to only take net4 traffic to net4 interfaces

Q) Is there a way to control the failover behaviour of the interfaces to only failover to the appropriate IPMP interface?


lo0: flags=2001000849<UP,LOOPBACK,RUNNING,MULTICAST,IPv4,VIRTUAL> mtu 8232 index 1

        inet netmask ff000000

nxge1: flags=201000843<UP,BROADCAST,RUNNING,MULTICAST,IPv4,CoS> mtu 1500 index 2

        inet netmask ff000000 broadcast

        groupname IPMPnet2

        ether 0:21:28:9:17:ef

nxge2: flags=201000843<UP,BROADCAST,RUNNING,MULTICAST,IPv4,CoS> mtu 1500 index 3

        inet netmask ff000000 broadcast

        groupname IPMPnet4

        ether 0:21:28:9:17:f0

nxge4: flags=201000843<UP,BROADCAST,RUNNING,MULTICAST,IPv4,CoS> mtu 1500 index 4

        inet 10.x.139.50 netmask ffffff00 broadcast 10.x.139.255

        ether 0:21:28:9:29:8e

nxge5: flags=201000843<UP,BROADCAST,RUNNING,MULTICAST,IPv4,CoS> mtu 1500 index 5

        inet 10.x.138.73 netmask ffffff00 broadcast 10.x.138.255

        groupname IPMPnet2

        ether 0:21:28:9:29:8f

nxge5:1: flags=201040843<UP,BROADCAST,RUNNING,MULTICAST,DEPRECATED,IPv4,CoS> mtu 1500 index 5

        inet 10.x.138.145 netmask ffffff00 broadcast 10.x.138.255

nxge6: flags=201000843<UP,BROADCAST,RUNNING,MULTICAST,IPv4,CoS> mtu 1500 index 6

        inet 10.x.139.195 netmask ffffff00 broadcast 10.x.139.255

        groupname IPMPnet4

        ether 0:21:28:9:29:90


IRE Table: IPv4

  Destination             Mask           Gateway          Device Mxfrg Rtt   Ref Flg  Out  In/Fwd

-------------------- --------------- -------------------- ------ ----- ----- --- --- ----- ------

default             10.x.138.1                  1500*    0   1 UG    1715      0

10.x.138.0   10.x.138.73         nxge5   1500*    0   1 U     1390      0

10.x.138.0   10.x.138.73         nxge5:1  1500*    0   1 U        0      0

10.x.139.0   10.x.139.50         nxge4   1500*    0   1 U     1148      0

10.x.139.0   10.x.139.195        nxge6   1500*    0   1 U        3      0         10.x.138.73         nxge5   1500*    0   1 U        0      0              lo0     8232*    0  30 UH   69238      0

-- hostname.* files --

bash-3.00# more /etc/hostname.*





group IPMPnet2 up





group IPMPnet4 up





10.x.139.50 netmask broadcast + up









10.x.138.73 netmask broadcast + group IPMPnet2 up





10.x.139.195 netmask broadcast + group IPMPnet4 up





10.x.139.195 netmask broadcast + group IPMPnet4 up

--netmasks inf --




-- default router info --

cat /etc/defaultrouter


Open in new window

Question by:ruggerrick
    LVL 22

    Assisted Solution

    Please post the output of "netstat -rn". What is the IP address of the other end of the net4 link?

    Accepted Solution

    we had to break down the config, I cannot get an applicable output of netstat -nr

    Write Comment

    Please enter a first name

    Please enter a last name

    We will never share this with anyone.

    Featured Post

    Looking for New Ways to Advertise?

    Engage with tech pros in our community with native advertising, as a Vendor Expert, and more.

    Suggested Solutions

    This post first appeared at Oracleinaction  ( Anju Garg (Myself). I  will demonstrate that undo for DML’s is stored both in undo tablespace and online redo logs. Then, we will analyze the reaso…
    PRTG Network Monitor lets you monitor your bandwidth usage, so you know who is using up your bandwidth, and what they're using it for.
    Via a live example, show how to take different types of Oracle backups using RMAN.
    This video shows how to copy an entire tablespace from one database to another database using Transportable Tablespace functionality.

    737 members asked questions and received personalized solutions in the past 7 days.

    Join the community of 500,000 technology professionals and ask your questions.

    Join & Ask a Question

    Need Help in Real-Time?

    Connect with top rated Experts

    18 Experts available now in Live!

    Get 1:1 Help Now