IPMP on Solaris

I have a V100 with 2 NICs.  I am trying to set up IPMP for failover using the fewest amount of IP addresses.
root@jm091# ifconfig -a                                                        
lo0: flags=1000849<UP,LOOPBACK,RUNNING,MULTICAST,IPv4> mtu 8232 index 1        
        inet 127.0.0.1 netmask ff000000                                        
dmfe0: flags=9040843<UP,BROADCAST,RUNNING,MULTICAST,DEPRECATED,IPv4,NOFAILOVER>
mtu 1500 index 2                                                                
        inet 10.76.88.94 netmask ffffff00 broadcast 10.76.88.255                
        groupname ipmp0                                                        
        ether 0:3:ba:1f:94:f1                                                  
dmfe0:1: flags=1000843<UP,BROADCAST,RUNNING,MULTICAST,IPv4> mtu 1500 index 2    
        inet 192.168.5.10 netmask ffffff00 broadcast 192.168.5.255              
dmfe1: flags=9040843<UP,BROADCAST,RUNNING,MULTICAST,DEPRECATED,IPv4,NOFAILOVER>
mtu 1500 index 3                                                                
        inet 10.76.88.59 netmask ffffff00 broadcast 10.76.88.255                
        groupname ipmp0                                                        
        ether 0:3:ba:1f:94:f2                                                  
dmfe1:1: flags=1000843<UP,BROADCAST,RUNNING,MULTICAST,IPv4> mtu 1500 index 3    
        inet 192.168.5.20 netmask ffffff00 broadcast 192.168.5.255

root@jm091# more hosts                                                          
#                                                                              
# Internet host table                                                          
#                                                                              
127.0.0.1       localhost                                                      
10.76.88.94     jm091-int0                                                      
10.76.88.59     jm091-int1                                                      
192.168.5.10    jm091-ext0                                                      
192.168.5.20    jm091-ext1 jm091 jm091.volvocf.com      loghost                

root@jm091# more hostname.dmfe1                                                
jm091-int1 netmask + broadcast + group ipmp0 deprecated -failover up \          
addif jm091-ext1 netmask + broadcast + failover up                              
root@jm091# more hostname.dmfe0                                                
jm091-int0 netmask + broadcast + group ipmp0 deprecated -failover up \          
addif jm091-ext0 netmask + broadcast + failover up

root@jm091# more /etc/defaultrouter                                            
10.76.88.1

Obviously I am doing something wrong here.  The interfaces do come up (see above ifconfig output) but performance is terrible.  It also does not failover correctly.  Sun has sent me about 400 pages of documentation but I seem to just not get it.

Thanks,
kkatulaAsked:
Who is Participating?

[Product update] Infrastructure Analysis Tool is now available with Business Accounts.Learn More

x
I wear a lot of hats...

"The solutions and answers provided on Experts Exchange have been extremely helpful to me over the last few years. I wear a lot of hats - Developer, Database Administrator, Help Desk, etc., so I know a lot of things but not a lot about one thing. Experts Exchange gives me answers from people who do know a lot about one thing, in a easy to use platform." -Todd S.

peter991Commented:
Maybe you are running in half duplex?


nickstatus.txt
0
kkatulaAuthor Commented:
    Link:  Auto-Neg:   Status:   Speed:    Mode:  Ethernet Address:          
---------------------------------------------------------------------          
     dmfe0        OFF        UP    100MB      FDX    0:3:ba:1f:94:f1            
     dmfe1        OFF        UP    100MB      FDX    0:3:ba:1f:94:f2

No it is 100mb FDX.
0
arnoldCommented:
post netstat -rn.

Are you looking at the failover for dfme0:1 and dfme1:1? It also looks as though you have both failover directives to up versus having one up and one on standby.

The information you provided as well as the issues you have are very vague.
i.e. dealing with performance as well as with failover not working did you compare these when one of the nics was down.

Ref:
http://www.eng.auburn.edu/~doug/howtos/multipathing.html
0
Powerful Yet Easy-to-Use Network Monitoring

Identify excessive bandwidth utilization or unexpected application traffic with SolarWinds Bandwidth Analyzer Pack.

Brian UtterbackPrinciple Software EngineerCommented:
I don't understand your setup. You say you want to have the fewest number of IP addresses. That would be 3, but you use 4. And it doesn't seem that you have a redundant address at all.

The idea with IPMP is to have one physical address per interface and once virtual address that is highly available. You don't seem to have that.


Which address do you want HA?
0
kkatulaAuthor Commented:
root@jm091# netstat -rn                                                        
                                                                               
Routing Table: IPv4                                                            
  Destination           Gateway           Flags  Ref   Use   Interface          
-------------------- -------------------- ----- ----- ------ ---------          
10.76.88.0           192.168.5.20         U         1     53  dmfe0            
10.76.88.0           192.168.5.20         U         1      8  dmfe1            
192.168.5.0          192.168.5.10         U         1      0  dmfe0:1          
192.168.5.0          192.168.5.20         U         1      0  dmfe1:1          
224.0.0.0            192.168.5.20         U         1      0  dmfe1:1          
default              10.76.88.1           UG        1     36                    
127.0.0.1            127.0.0.1            UH        4    330  lo0              
root@jm091#

I want dmfe0 to be my primary interface and dmfe1 to be my failover address.  Or have them both failover for each other.  I have never done this before so I am sure I am nowhere close to being right.  
I have limited addresses on the 10.76.88.x subnet so I was trying to use 192.168.x.x somehow to alleviate the limitation.
Thanks,
0
peter991Commented:
As blu says.. You need at least 3 addresses.
0
peter991Commented:
I think that one of your failover-interfaces should have "-failover standby up" instead of "-failover up" only.

0
kkatulaAuthor Commented:
Tried changing to standby and failover still does not work.
0
arnoldCommented:
When you changed to standby, did you also remove the dmfe1:1?
Have you looked at  http://www.eng.auburn.edu/~doug/howtos/multipathing.html?

Also, you are using cross segment IPs which could impact the performance:
10.76.88.x and 192.168.5.x.

The other issue is that you are adding on each interface a different "real' host
On the "primary" you have jm091-ext1 and on the "standby" you have jm091-ext0.
Do you need both 192.168.5.20 and 192.168.5.10 to shift between the two network interfaces?
0
kkatulaAuthor Commented:
Ok, I got it to work. One last question, is this an active-active configuration where you have 2 nics?  How can I tell that traffic is going out both interfaces?
0
arnoldCommented:
I think this is a Failover setup since only one interface will have the 192.168.5.20 IP address.
traffic going out over either interface would need that you set their index to the same value.  From the original post the index on the primary has 2 and the secondary has 3 meaning dmfe0 is preferred over dmfe1.
0
kkatulaAuthor Commented:
Given this output:
root@jm091# ifconfig -a                                                        
lo0: flags=1000849<UP,LOOPBACK,RUNNING,MULTICAST,IPv4> mtu 8232 index 1        
        inet 127.0.0.1 netmask ff000000                                        
dmfe0: flags=19040803<UP,BROADCAST,MULTICAST,DEPRECATED,IPv4,NOFAILOVER,FAILED>
mtu 1500 index 2                                                                
        inet 10.76.88.95 netmask ffffff00 broadcast 10.76.88.255                
        groupname ipmp0                                                        
        ether 0:3:ba:1f:94:f1                                                  
dmfe1: flags=9040843<UP,BROADCAST,RUNNING,MULTICAST,DEPRECATED,IPv4,NOFAILOVER>
mtu 1500 index 3                                                                
        inet 10.76.88.59 netmask ffffff00 broadcast 10.76.88.255                
        groupname ipmp0                                                        
        ether 0:3:ba:1f:94:f2                                                  
dmfe1:1: flags=1000842<BROADCAST,RUNNING,MULTICAST,IPv4> mtu 1500 index 3      
        inet 10.76.88.94 netmask ffffff00 broadcast 10.76.88.255                
dmfe1:2: flags=1000843<UP,BROADCAST,RUNNING,MULTICAST,IPv4> mtu 1500 index 3    
        inet 10.76.88.94 netmask ffffff00 broadcast 10.76.88.255

root@jm091# more /etc/hosts                                                    
#                                                                              
# Internet host table                                                          
#                                                                              
127.0.0.1       localhost                                                      
10.76.88.95     jm091-int0                                                      
10.76.88.59     jm091-int1                                                      
10.76.88.94     jm091-ext0 jm091 jm091.xxxxxcf.com      loghost                
root@jm091# more /etc/hostname.dmfe0                                            
jm091-int0 netmask + broadcast + group ipmp0 deprecated -failover up \          
addif jm091-ext0 netmask + broadcast + failover up                              
root@jm091# more /etc/hostname.dmfe1                                            
jm091-int1 netmask + broadcast + group ipmp0 deprecated -failover up \          
addif jm091-ext0 netmask + broadcast + failover up

The above output was after I pulled the cable from dmfe0 and it failed over to dmfe1 and never missed a ping.  I am wanting to have active active traffic down both interfaces.  I don't understand how it is possible to only use one ip address which is what dns points to.  Can someone clarify if it is possible and how to do this?

0
arnoldCommented:
Not sure what you are asking.  The whole idea is to reference a single IP and have the system accessible at all times without regard to which Network Interface is up.
usually each network inteface is connected to a different switch such that a switch failure does not make the system inaccessible.
0
kkatulaAuthor Commented:
True,  I want failover and load balancing.  The load balancing is the part I don't seem to have at this point.
0
arnoldCommented:
When both interfaces are up, do you not have the same IP on index 1 of dmfe0 and dmfe1?  Check the switch, it may record a conflict. note when dmfe0 is failed, you have dmfe1:1 and dmfe1:2 reflecting the same IP address.  If your switch is managed, look at the traffic on the ports to which dmfe0 and dmfe1 are plugged in.
0
kkatulaAuthor Commented:
This is what I have when the server first boots with no failures.  I would like the traffic to load balance across the two network interfaces.

root@jm091# ifconfig -a
lo0: flags=1000849<UP,LOOPBACK,RUNNING,MULTICAST,IPv4> mtu 8232 index 1
        inet 127.0.0.1 netmask ff000000
dmfe0: flags=9040843<UP,BROADCAST,RUNNING,MULTICAST,DEPRECATED,IPv4,NOFAILOVER> mtu 1500 index 2
        inet 10.76.88.95 netmask ffffff00 broadcast 10.76.88.255
        groupname ipmp0
        ether 0:3:ba:1f:94:f1
dmfe0:1: flags=1000843<UP,BROADCAST,RUNNING,MULTICAST,IPv4> mtu 1500 index 2
        inet 10.76.88.94 netmask ffffff00 broadcast 10.76.88.255
dmfe1: flags=9040843<UP,BROADCAST,RUNNING,MULTICAST,DEPRECATED,IPv4,NOFAILOVER> mtu 1500 index 3
        inet 10.76.88.59 netmask ffffff00 broadcast 10.76.88.255
        groupname ipmp0
        ether 0:3:ba:1f:94:f2
dmfe1:1: flags=1000842<BROADCAST,RUNNING,MULTICAST,IPv4> mtu 1500 index 3
        inet 10.76.88.94 netmask ffffff00 broadcast 10.76.88.255
0
arnoldCommented:
How are the two network interfaces connected? Two separate switches?
Are you using managed switches?  Do you monitor port traffic? Are the switch ports bonded?

 
0
kkatulaAuthor Commented:
Today, they are on the same switch.  We would like to get this working and move one nic to a second switch.
Are you using managed switches?
Yes
Do you monitor port traffic?
No but we can sniff a port if needed.
Are the switch ports bonded?
Not the (2) switch ports this server is plugged into.

Thanks,
0
arnoldCommented:
Look at the port usage which will tell you whether the setup is "load balanced" or not.  I.e. if the traffic on both ports is close, or if the traffic on one is higher than the other, you will know which mode.  Don't you have errors on the switch dealing with the IP? How about on the firewall?  Is this a level 2 switch?
0
kkatulaAuthor Commented:
This 2 switches are Cisco 6509's, which are  Layer 3 switches.  We looked at the port usage and only one of the interfaces is being used with the configuration now.  My ideal would be for both nics to be used in a load balancing configuration with failover as well.
0
arnoldCommented:
What is the value in /etc/default/mpathd?
try ping 192.168.5.20 from two different systems and then get the arp table (arp -a) to see whether the arp entry reflects the same interface.

The IPMP is not a distributive load balancer it simply makes the IPMP IP always available and accessible.  Could you check the firewall if any to see whether you are locking the IP to a specific MAC address as well as whether there are errors dealing with the same IP with different MACs conflict?
0

Experts Exchange Solution brought to you by

Your issues matter to us.

Facing a tech roadblock? Get the help and guidance you need from experienced professionals who care. Ask your question anytime, anywhere, with no hassle.

Start your 7-day free trial
It's more than this solution.Get answers and train to solve all your tech problems - anytime, anywhere.Try it for free Edge Out The Competitionfor your dream job with proven skills and certifications.Get started today Stand Outas the employee with proven skills.Start learning today for free Move Your Career Forwardwith certification training in the latest technologies.Start your trial today
Unix OS

From novice to tech pro — start learning today.