[Okta Webinar] Learn how to a build a cloud-first strategyRegister Now

x
?
Solved

IPMP on Solaris

Posted on 2008-11-10
21
Medium Priority
?
1,656 Views
Last Modified: 2013-12-27
I have a V100 with 2 NICs.  I am trying to set up IPMP for failover using the fewest amount of IP addresses.
root@jm091# ifconfig -a                                                        
lo0: flags=1000849<UP,LOOPBACK,RUNNING,MULTICAST,IPv4> mtu 8232 index 1        
        inet 127.0.0.1 netmask ff000000                                        
dmfe0: flags=9040843<UP,BROADCAST,RUNNING,MULTICAST,DEPRECATED,IPv4,NOFAILOVER>
mtu 1500 index 2                                                                
        inet 10.76.88.94 netmask ffffff00 broadcast 10.76.88.255                
        groupname ipmp0                                                        
        ether 0:3:ba:1f:94:f1                                                  
dmfe0:1: flags=1000843<UP,BROADCAST,RUNNING,MULTICAST,IPv4> mtu 1500 index 2    
        inet 192.168.5.10 netmask ffffff00 broadcast 192.168.5.255              
dmfe1: flags=9040843<UP,BROADCAST,RUNNING,MULTICAST,DEPRECATED,IPv4,NOFAILOVER>
mtu 1500 index 3                                                                
        inet 10.76.88.59 netmask ffffff00 broadcast 10.76.88.255                
        groupname ipmp0                                                        
        ether 0:3:ba:1f:94:f2                                                  
dmfe1:1: flags=1000843<UP,BROADCAST,RUNNING,MULTICAST,IPv4> mtu 1500 index 3    
        inet 192.168.5.20 netmask ffffff00 broadcast 192.168.5.255

root@jm091# more hosts                                                          
#                                                                              
# Internet host table                                                          
#                                                                              
127.0.0.1       localhost                                                      
10.76.88.94     jm091-int0                                                      
10.76.88.59     jm091-int1                                                      
192.168.5.10    jm091-ext0                                                      
192.168.5.20    jm091-ext1 jm091 jm091.volvocf.com      loghost                

root@jm091# more hostname.dmfe1                                                
jm091-int1 netmask + broadcast + group ipmp0 deprecated -failover up \          
addif jm091-ext1 netmask + broadcast + failover up                              
root@jm091# more hostname.dmfe0                                                
jm091-int0 netmask + broadcast + group ipmp0 deprecated -failover up \          
addif jm091-ext0 netmask + broadcast + failover up

root@jm091# more /etc/defaultrouter                                            
10.76.88.1

Obviously I am doing something wrong here.  The interfaces do come up (see above ifconfig output) but performance is terrible.  It also does not failover correctly.  Sun has sent me about 400 pages of documentation but I seem to just not get it.

Thanks,
0
Comment
Question by:kkatula
  • 9
  • 8
  • 3
  • +1
21 Comments
 
LVL 6

Expert Comment

by:peter991
ID: 22921748
Maybe you are running in half duplex?


nickstatus.txt
0
 

Author Comment

by:kkatula
ID: 22923409
    Link:  Auto-Neg:   Status:   Speed:    Mode:  Ethernet Address:          
---------------------------------------------------------------------          
     dmfe0        OFF        UP    100MB      FDX    0:3:ba:1f:94:f1            
     dmfe1        OFF        UP    100MB      FDX    0:3:ba:1f:94:f2

No it is 100mb FDX.
0
 
LVL 81

Expert Comment

by:arnold
ID: 22923415
post netstat -rn.

Are you looking at the failover for dfme0:1 and dfme1:1? It also looks as though you have both failover directives to up versus having one up and one on standby.

The information you provided as well as the issues you have are very vague.
i.e. dealing with performance as well as with failover not working did you compare these when one of the nics was down.

Ref:
http://www.eng.auburn.edu/~doug/howtos/multipathing.html
0
A Cyber Security RX to Protect Your Organization

Join us on December 13th for a webinar to learn how medical providers can defend against malware with a cyber security "Rx" that supports a healthy technology adoption plan for every healthcare organization.

 
LVL 22

Expert Comment

by:Brian Utterback
ID: 22923534
I don't understand your setup. You say you want to have the fewest number of IP addresses. That would be 3, but you use 4. And it doesn't seem that you have a redundant address at all.

The idea with IPMP is to have one physical address per interface and once virtual address that is highly available. You don't seem to have that.


Which address do you want HA?
0
 

Author Comment

by:kkatula
ID: 22923796
root@jm091# netstat -rn                                                        
                                                                               
Routing Table: IPv4                                                            
  Destination           Gateway           Flags  Ref   Use   Interface          
-------------------- -------------------- ----- ----- ------ ---------          
10.76.88.0           192.168.5.20         U         1     53  dmfe0            
10.76.88.0           192.168.5.20         U         1      8  dmfe1            
192.168.5.0          192.168.5.10         U         1      0  dmfe0:1          
192.168.5.0          192.168.5.20         U         1      0  dmfe1:1          
224.0.0.0            192.168.5.20         U         1      0  dmfe1:1          
default              10.76.88.1           UG        1     36                    
127.0.0.1            127.0.0.1            UH        4    330  lo0              
root@jm091#

I want dmfe0 to be my primary interface and dmfe1 to be my failover address.  Or have them both failover for each other.  I have never done this before so I am sure I am nowhere close to being right.  
I have limited addresses on the 10.76.88.x subnet so I was trying to use 192.168.x.x somehow to alleviate the limitation.
Thanks,
0
 
LVL 6

Expert Comment

by:peter991
ID: 22928730
As blu says.. You need at least 3 addresses.
0
 
LVL 6

Expert Comment

by:peter991
ID: 22929386
I think that one of your failover-interfaces should have "-failover standby up" instead of "-failover up" only.

0
 

Author Comment

by:kkatula
ID: 22932342
Tried changing to standby and failover still does not work.
0
 
LVL 81

Expert Comment

by:arnold
ID: 22933084
When you changed to standby, did you also remove the dmfe1:1?
Have you looked at  http://www.eng.auburn.edu/~doug/howtos/multipathing.html?

Also, you are using cross segment IPs which could impact the performance:
10.76.88.x and 192.168.5.x.

The other issue is that you are adding on each interface a different "real' host
On the "primary" you have jm091-ext1 and on the "standby" you have jm091-ext0.
Do you need both 192.168.5.20 and 192.168.5.10 to shift between the two network interfaces?
0
 

Author Comment

by:kkatula
ID: 22961450
Ok, I got it to work. One last question, is this an active-active configuration where you have 2 nics?  How can I tell that traffic is going out both interfaces?
0
 
LVL 81

Expert Comment

by:arnold
ID: 22961698
I think this is a Failover setup since only one interface will have the 192.168.5.20 IP address.
traffic going out over either interface would need that you set their index to the same value.  From the original post the index on the primary has 2 and the secondary has 3 meaning dmfe0 is preferred over dmfe1.
0
 

Author Comment

by:kkatula
ID: 22962117
Given this output:
root@jm091# ifconfig -a                                                        
lo0: flags=1000849<UP,LOOPBACK,RUNNING,MULTICAST,IPv4> mtu 8232 index 1        
        inet 127.0.0.1 netmask ff000000                                        
dmfe0: flags=19040803<UP,BROADCAST,MULTICAST,DEPRECATED,IPv4,NOFAILOVER,FAILED>
mtu 1500 index 2                                                                
        inet 10.76.88.95 netmask ffffff00 broadcast 10.76.88.255                
        groupname ipmp0                                                        
        ether 0:3:ba:1f:94:f1                                                  
dmfe1: flags=9040843<UP,BROADCAST,RUNNING,MULTICAST,DEPRECATED,IPv4,NOFAILOVER>
mtu 1500 index 3                                                                
        inet 10.76.88.59 netmask ffffff00 broadcast 10.76.88.255                
        groupname ipmp0                                                        
        ether 0:3:ba:1f:94:f2                                                  
dmfe1:1: flags=1000842<BROADCAST,RUNNING,MULTICAST,IPv4> mtu 1500 index 3      
        inet 10.76.88.94 netmask ffffff00 broadcast 10.76.88.255                
dmfe1:2: flags=1000843<UP,BROADCAST,RUNNING,MULTICAST,IPv4> mtu 1500 index 3    
        inet 10.76.88.94 netmask ffffff00 broadcast 10.76.88.255

root@jm091# more /etc/hosts                                                    
#                                                                              
# Internet host table                                                          
#                                                                              
127.0.0.1       localhost                                                      
10.76.88.95     jm091-int0                                                      
10.76.88.59     jm091-int1                                                      
10.76.88.94     jm091-ext0 jm091 jm091.xxxxxcf.com      loghost                
root@jm091# more /etc/hostname.dmfe0                                            
jm091-int0 netmask + broadcast + group ipmp0 deprecated -failover up \          
addif jm091-ext0 netmask + broadcast + failover up                              
root@jm091# more /etc/hostname.dmfe1                                            
jm091-int1 netmask + broadcast + group ipmp0 deprecated -failover up \          
addif jm091-ext0 netmask + broadcast + failover up

The above output was after I pulled the cable from dmfe0 and it failed over to dmfe1 and never missed a ping.  I am wanting to have active active traffic down both interfaces.  I don't understand how it is possible to only use one ip address which is what dns points to.  Can someone clarify if it is possible and how to do this?

0
 
LVL 81

Expert Comment

by:arnold
ID: 22962356
Not sure what you are asking.  The whole idea is to reference a single IP and have the system accessible at all times without regard to which Network Interface is up.
usually each network inteface is connected to a different switch such that a switch failure does not make the system inaccessible.
0
 

Author Comment

by:kkatula
ID: 22962480
True,  I want failover and load balancing.  The load balancing is the part I don't seem to have at this point.
0
 
LVL 81

Expert Comment

by:arnold
ID: 22962592
When both interfaces are up, do you not have the same IP on index 1 of dmfe0 and dmfe1?  Check the switch, it may record a conflict. note when dmfe0 is failed, you have dmfe1:1 and dmfe1:2 reflecting the same IP address.  If your switch is managed, look at the traffic on the ports to which dmfe0 and dmfe1 are plugged in.
0
 

Author Comment

by:kkatula
ID: 22963269
This is what I have when the server first boots with no failures.  I would like the traffic to load balance across the two network interfaces.

root@jm091# ifconfig -a
lo0: flags=1000849<UP,LOOPBACK,RUNNING,MULTICAST,IPv4> mtu 8232 index 1
        inet 127.0.0.1 netmask ff000000
dmfe0: flags=9040843<UP,BROADCAST,RUNNING,MULTICAST,DEPRECATED,IPv4,NOFAILOVER> mtu 1500 index 2
        inet 10.76.88.95 netmask ffffff00 broadcast 10.76.88.255
        groupname ipmp0
        ether 0:3:ba:1f:94:f1
dmfe0:1: flags=1000843<UP,BROADCAST,RUNNING,MULTICAST,IPv4> mtu 1500 index 2
        inet 10.76.88.94 netmask ffffff00 broadcast 10.76.88.255
dmfe1: flags=9040843<UP,BROADCAST,RUNNING,MULTICAST,DEPRECATED,IPv4,NOFAILOVER> mtu 1500 index 3
        inet 10.76.88.59 netmask ffffff00 broadcast 10.76.88.255
        groupname ipmp0
        ether 0:3:ba:1f:94:f2
dmfe1:1: flags=1000842<BROADCAST,RUNNING,MULTICAST,IPv4> mtu 1500 index 3
        inet 10.76.88.94 netmask ffffff00 broadcast 10.76.88.255
0
 
LVL 81

Expert Comment

by:arnold
ID: 22963840
How are the two network interfaces connected? Two separate switches?
Are you using managed switches?  Do you monitor port traffic? Are the switch ports bonded?

 
0
 

Author Comment

by:kkatula
ID: 22964000
Today, they are on the same switch.  We would like to get this working and move one nic to a second switch.
Are you using managed switches?
Yes
Do you monitor port traffic?
No but we can sniff a port if needed.
Are the switch ports bonded?
Not the (2) switch ports this server is plugged into.

Thanks,
0
 
LVL 81

Expert Comment

by:arnold
ID: 22964025
Look at the port usage which will tell you whether the setup is "load balanced" or not.  I.e. if the traffic on both ports is close, or if the traffic on one is higher than the other, you will know which mode.  Don't you have errors on the switch dealing with the IP? How about on the firewall?  Is this a level 2 switch?
0
 

Author Comment

by:kkatula
ID: 22984533
This 2 switches are Cisco 6509's, which are  Layer 3 switches.  We looked at the port usage and only one of the interfaces is being used with the configuration now.  My ideal would be for both nics to be used in a load balancing configuration with failover as well.
0
 
LVL 81

Accepted Solution

by:
arnold earned 1500 total points
ID: 22986461
What is the value in /etc/default/mpathd?
try ping 192.168.5.20 from two different systems and then get the arp table (arp -a) to see whether the arp entry reflects the same interface.

The IPMP is not a distributive load balancer it simply makes the IPMP IP always available and accessible.  Could you check the firewall if any to see whether you are locking the IP to a specific MAC address as well as whether there are errors dealing with the same IP with different MACs conflict?
0

Featured Post

 The Evil-ution of Network Security Threats

What are the hacks that forever changed the security industry? To answer that question, we created an exciting new eBook that takes you on a trip through hacking history. It explores the top hacks from the 80s to 2010s, why they mattered, and how the security industry responded.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Hi there, This article summarizes what you need if you are going to set up your home or small business Network Attached Storage (NAS) to be accessible from the internet. Of course there are configuration differences based on your NAS or router ma…
#Citrix #Citrix Netscaler #HTTP Compression #Load Balance
Learn several ways to interact with files and get file information from the bash shell. ls lists the contents of a directory: Using the -a flag displays hidden files: Using the -l flag formats the output in a long list: The file command gives us mor…
Learn how to get help with Linux/Unix bash shell commands. Use help to read help documents for built in bash shell commands.: Use man to interface with the online reference manuals for shell commands.: Use man to search man pages for unknown command…
Suggested Courses
Course of the Month20 days, 2 hours left to enroll

873 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question