• Status: Solved
  • Priority: Medium
  • Security: Public
  • Views: 2289
  • Last Modified:

Compatability Issue - VMware ESX 3.5.0 fails to discover iSCSI target on Netgear ReadyNAS 2100

Dear VMware 3.4 guru's,

GOAL:             I am attemtpting to get my VMware ESX 3.5 (update #5) hosts (which are either running on HP G3 DL360 (P31 bios) or G3 DL380 (P29 bios) with (4) 1000MB NICs, connected to a 48-port Gig-E managed switch (Netgear GS748TR) with all ports belonging to the same VLAN (for now), to DISCOVER the iSCIS (LUN's) I've created on the Netgear ReadyNAS (4TB model: 2100 firmware is RAIDiator 4.2.8) which is also plugged into the GS748TR switch.  


CURRENT RESULT (as of Dec 16, 2009 5pm EST):  I have tried a million things and I can't get ESX to discover any iSCSI LUNs I create on the ReadyNAS 2100.  However, my Windows 7 Enterprise software iSCSI identifies the Netgear iSCSI LUNs in about 4 seconds!   If you can answer, "What does Windows 7's iSCSI software initiator do...that VMware's ESX 3.5.0 update 5 does NOT do...when scanning (the virtual HBA) for iSCSI targets - and you'll likely solve this riddle!"

However, I want to give you the accurate impression that I did try everything you can think of to ensure it isn't "operator error".  I assume "operator error" at nearly every turn.  I think I've found some type of incompatibility, but I'd sure like some fresh (way smarter) brains attacking my problem.  
 
   Physical Layer:  I have checked, rechecked, & triple checked the physical layer.  I have (one by one) unplugged each NIC card of each server to correctly map each NIC to a switch port (so I can implement vlan in the future).  In VMware vSwitch0 is just the service console and it belongs to the 10.1.1.x subnet, vswitch1 has the VMKernel port for iSCSI and it also belongs to the 10.1.1.x subnet - I have DISABLED VMotion.  I can log into the Linux BASH shell of these servers and ping to everything (GS748TR, firewall, & hosts).  I have run vmkping X.X.X.X command to verify connectivity to my iSCSI VMKernel port!  Also, I've run esxcfg-swiscsi -e enables, -d disables, -q is service up or down, -s scans for targets, -k kills/clears iSCSI stack, -r restores sw iSCSI stack, - h help.  

     Inside VMware using VIC client -> Configure Tab -> Network Tab
           NIC 0 = Service Console, ip 10.1.1.201/24,  Port # 6 on GS748TR
           NIC 1 = VMKernel for software iSCSI, ip 10.1.1.202/24
           NIC 2 = unassigned
           NIC 3 = unassigned

    Inside VMware using VIC client -> Configure Tab -> Storage Adapter Tab -> Select iSCSI -> click on Properties - > Configure - click Enable - > Dynamic Discovery Tab -> enter 10.1.1.1 (port #1 on ReadyNAS) and 10.1.1.2 (port #2 on ReadyNAS) (which I ping to  verify connectivity long before), CHAP Tab - ensure CHAP is disabled.  
 
    Inside the Netgear ReadyNAS 2100 -> Network -> Interface -NIC 1 Tab - ensure ip address is 10.1.1.1 vlan & jumbo-frames are DISABLED. Same for NIC 2 (10.1.1.2).

Inside ReadyNAS 2100 -> Services -> Standard File Protocols -> CIFS, NFS, FTP, HTTP, HTTPS are all enabled.

Inside ReadyNAS 2100 -> Volumes - > Volume Settings -> iSCSI Tab -> Create iSCSI Target (aka LUN), activate or disable iSCSI LUN target, choose name - vmlunxx in my case, choose size - 50 GB in my case, CHAP settings - all disabled, Access Controls - where you can identify the name of only the software initiators that can see this LUN/target.  

The Windows 7 software initiator can see iSCSI LUN/Targets on the ReadyNAS as long as they are "active" without regard for whether the Microsoft iSCSI initiator has NOT been included in the list of initiator to authorize.  

The symtom:  I tell the VMware ESX 3.5.0 (update #5) host to rescan the virtual iSCSI HBA for new devices (targets).  The task usually times out.  It has NEVER found an iSCSI target

"Why don't I just give up and migrate to vSphere 4 where software iSCSI is supposedly much easier to configure?"  Good question, but the answer is because HP G3 DL360's & DL380's have 32-bit XEONs (which suprised me) - vSphere4 requires 64-bit CPUs - and before a smart aleck corrects me - I don't mean true 64 bit - I mean x64.  
0
buymycorp
Asked:
buymycorp
  • 6
  • 4
  • 3
  • +3
1 Solution
 
ringwCommented:
Have you opened the ISCSI firewall port on the ESX Server:

esxcfg-firewall -e swiSCSIClient
0
 
buymycorpAuthor Commented:
The first line in my previous quote was a typo:  it should have read VMware 3.5.
 
And yes ringw the swiscsi firewall port is open.  
0
 
Paul SolovyovskyCommented:
do you seen any logs on the netgear?  I have seen some issues on the small NAS unites and wound up going with a Iomega IX2-200 because it was on the HCL LIst.

Something to check.   Your hostname on the ESX host -- is it all lowercase or does it have capital letters?  

I agree that you can't go to vsphere because of G3 limiitations.  You should also upgrade the BIOS on the HP's I've seen some nic issues if you don't on the newer models but still doesn't hurt.

Also when you zone the LUNs on the Netgear I would do via IP Address, just to make sure it's not a DNS issue
0
VIDEO: THE CONCERTO CLOUD FOR HEALTHCARE

Modern healthcare requires a modern cloud. View this brief video to understand how the Concerto Cloud for Healthcare can help your organization.

 
za_mkhCommented:
Question:....
You say on your ISCSI LUN presented by the netgear, that CHAP is disabled. Have you checked to see if CHAP is configured on the ESX host for your ISCSI connection? I wonder if there is a requirement for you to have authentication enabled for ISCSI to work.  Basic searches on the Internet seem to indicate so. Maybe enabling CHAP on the Netgear and then configuring ESX as appropriate will help?
I personally don't use ISCSI so am very green on ... but this will be something I will look up !
0
 
andyalderSaggar makers bottom knockerCommented:
Service console has to be connected to the iSCSI NIC or it won't be able to login to the iSCSI target, also you should have your LAN and SAN on different subnets.
0
 
ringwCommented:
Not true, andyalder. Service Console need not be connected to the iSCSI NIC in buymycorp's environment. As long as the Service Console can reach the iSCSI Target (via other NIC or via routing), it's fine. As I understood buymycorp has just one flat subnet on one single switch and can ping the iSCSI Target from the Service Console, so if the iSCSI Firewall Port is open for outgoing traffic, it should work.
0
 
Paul SolovyovskyCommented:
@ringw.  I agree with Andy on this. With ESX 3.5 you do need a service console on the iSCSI vswitch, this has been fixed with vSphere.  This second service console must be on the same subnet as the iSCSI vmkernel port.
0
 
Paul SolovyovskyCommented:


http://www.vmwareinfo.com/2009/04/more-new-stuff-in-vsphere.html


excert:

Storage Enhancements for vSphere

 They have also rewritten the iSCSI initiator with a number of performance enhancements.  It no longer needs a service console connection and it also supports jumbo frames and multipathing.  
0
 
ringwCommented:
@paulsolov: I dont agree with you. IMHO its effectively the same to either put the SC on the same vSwitch as the kernel port, or putting the SC on a different vSwitch and connecting the two vSwitches together in the backend with a physical switch. As long as the SC can reach the iSCSI target, you should be fine.

@bymycorp: give it a try and move the SC to the same vSwitch as the kernel port and let us know if that changed anything.
0
 
Paul SolovyovskyCommented:
@ringw:  Most customers don't route to the iSCSI network from production network (as where most SC are located).  This has caused many issues with DNS, etc.. when not configured properly.  I agree it will work if it can route but generally not a good idea (especially since CHAP can be broken easily)
0
 
ringwCommented:
@paulsolov: I agree with you. Not a best practice how buymycorp has configured it, but it should work that way.
0
 
buymycorpAuthor Commented:
Thanks for the replies:
1.)  Paulsolovo:  
a.  ReadyNAS Logs:  (Good idea)  I've looked at the logs but unfortunately the Netgear ReadyNAS only seems to create log entries for things it does (examples: sync started, iscsi target vmlun01 successfully created, etc.).  However, I do NOT see any log entries for external activities such as "Windows 7 software initiator successfully discovered vmlun01, vmlun02, vmlun03, vmlun04 targets".  
b.  My host names are ip addresses:  10.1.1.201, 10.1.1.211, 10.1.1.221, 10.1.1.231, 10.1.1.241.  DNS is working but I'm not using it in the iSCSI discovery process - I'm using ip addresses.
c.  I'm loosely referring to the iSCSI targets on the Netgear ReadyNAS 2100 LUNs.  However, you shouldn't get the idea that the ReadyNAS appliance is a full-featured SAN device.  Therefore, your suggestion to zone the LUNs - is not possible in the ReadyNAS.  However, I take your meaning - I'm using physical ip address versus DNS.    
2.)  za_mkh:
a. CHAP :  sorry for the confusion.  CHAP is disabled on the ESX-side.  I will be happy to try a test where I've enabled CHAP on both sides.  I'll let you know.  However, I'm pretty skeptical that that is going to work.  My thinking is that if Windows 7's built-in iSCSI software initiator can "see" vmlun01, vmlun02, vmlun03, vmlun04 in a blink of an eye - what standards are at play in ESX 3.5 iSCSI discovery process?  
Also, I've worked with VMware technical support at my distributor.  My distributor sold $7.5 million in VMware licenses Tuesday evening!  I guarantee that my technical resource is well-versed in iSCSI and she spent a couple hours with me yesterday "trying everything she knew" and she's convinced that I have a compatibility issue.
3.)  andyalder:  
a.)  I've read the (22) page iSCSI Design Considerations and Deployment Guide for VI3.
On page (7) under the Software iSCSI Features and Limitations section - there's a confusing bullet point that I'd like to mention.  It states that the software initiator only supports a single storage interface (vmmhba40).  Guess what?  When I enable iSCSI on the Storage Adapters Tab - the iSCSI storage interface points to vmhba30.  There is no way (I can see) to change it????  
"Configuring a Software iSCSI Initiator" begin on page 11.  
Step (1) say to make sure and enable VMotion & IP Storage licenses.  DONE!
Step (2) say to configure a VMotion & IP Storage connection in the networking configuration.  That is a vague request!  Since I have seen instructions that say that VMotion must NOT be enabled on the VMKernel port group you intend to use for IP Storage.  In my environment, I'm not even trying to get VMotion working until I have IP storage working.  
Step (3) says to add a service console port to the same virtual switch used in step 2!  The guide goes on to say that whichever service console port that occupies the same vSwitch as your iSCSI VMKernel port - must belong to the same subnet.  These instructions seemed erroneous to the tech I was working with at my distributor.  She said that she's never worried about putting a service console port on the vswitch with the iSCSI VMkernel port.  So I can honestly say - I've tried this both ways.  
Going by the guide:  When trying to go by the guide - I just add an iSCSI VMkernel port group to vswitch0 and make it belong to the 10.1.1.x subnet.  This satisfies the condition that a service console shares the vswitch that the IP Storage VMKernel port resides on.  And it satisfies the requirement that both ports belong to the same subnet.  However, the result is always the same - the discovery process times out and NEVER discovers any targets.  Please keep in mind - that before "rescanning the virtual iSCSI HBA I always Putty to the host and ping the ReadyNAS NIC 1 & NIC 2 & perform a vmkping of the iSCSI VMKernel port.  All pings are successful.  Physical connectivity is NOT the problem.  
Going by the tech at my distributor:  I leave the service console alone by itself on vswitch0.  I add the IP Storage VMKernel port to vswitch1 and assign it an ip in the 10.1.1.x subnet (the same as the service console on vswitch0).  This has also never worked!  
I always list both of the ReadyNAS NIC ports in the ESX 3.5 iSCSI Dynamic Discovery Tab using the default iSCSI port 3260.
Thanks
-buymycorp
0
 
Paul SolovyovskyCommented:
If you get bogged down and you need to get this unit up and running you may want to try NFS.  Setup an extent, configure it enable root squash (annon access basically) and you can get it working.

Can you do a screenshot of your networking and iscsi initiator configuration.
0
 
thribhuCommented:
Check for port 3260 if it is blocked at VM . ESX again is redhat linux , so try flushing IP tables or disabling IP tables(Firewall).

0
 
buymycorpAuthor Commented:
I have given up.  NFS works but obviously i cannot use DRS, HA, vmotion, or storage vmotion.  I truly believe it is a compatiblity issue.  Thanks for trying.
0
 
Paul SolovyovskyCommented:
You sure can use DRS, HA, vmotion and storage vmotion with NFS.  Works just like iscsi or fiber

Just map the the NFS to your ESX hosts and you're good to go
0

Featured Post

Windows Server 2016: All you need to know

Learn about Hyper-V features that increase functionality and usability of Microsoft Windows Server 2016. Also, throughout this eBook, you’ll find some basic PowerShell examples that will help you leverage the scripts in your environments!

  • 6
  • 4
  • 3
  • +3
Tackle projects and never again get stuck behind a technical roadblock.
Join Now