I'm trying to setup iSCSI connections to my NAS and SAN devices and need to know how to do this properly.

Hi,

but when I try to connect to my EMC device it connects however it does not show any drive space nor hardware acceleration

unclear for me..

You need to configure the Unity and your ESXi hosts for iSCSI following best practices.
Also this is not really complex, but you cannot do it in a few minutes nor without reading some EMC/VMware guides....
You could start here : https://www.emc.com/collateral/TechnicalDocument/docu69326.pdf

iSCSI has something equivalent to FC zoning, if you are familiar with this concept.
So on Unity you have 2 Storage Processor, each with 2 iSCSI ports you should use.
You got to have a dedicated IP range (e.g. and switches (or at least a dedicated vlan) for iSCSI, with 2 distinct sub-network.
For each SP you will have an iSCSI port in a sub network and the other port in the other sub network.
Attached general scheme could help you.

Your ESXi hosts got to have 2 iSCSI IP too, one in each sub network (with iSCSI software or hardware adapter).
I suppose Andrew's links on iSCSI on ESXi should help you to do that, also I did not check them.

When your network config is done you will have to check if your hosts are seen in Unisphere by the Unity array, or create them
Then you will have the option to create and map LUN to your hosts...

I am going through the suggestions above and have a question on the QNAP and NFS. The QNAP NAS devices we are currently using in production do not have NFS turned on and would like to know if I can turn it on now and use that functionality or is it too late since we are already using it via iSCSI for our data?

The QNAP NAS devices we are currently using in production do not have NFS turned on and would like to know if I can turn it on now and use that functionality or is it too late since we are already using it via iSCSI for our data?

You can now turn it on, and use it, and test NFS versus iSCSI (if you have not allocated all the space on the NAS!)

on the same hosts! (e.g. NFS and iSCSI)

All the space on the NAS has been allocated to to a Thick LUN RAID 5 so I am to assume then I cannot use NFS.

Well you can enable NFS, but you'll have nothing to EXPORT, because you've allocated ALL the NAS space!

Please also note that RAID 5 is not considered Enterprise safe, because it tolerates only a single disk failure!

I am in such a difficult place. I no extra NAS devices to move data around and at the time of configuration we needed to as much disk space as possible so we went with RAID 5 VS 10. I had no idea how to configure the NAS with VMware so I just trusted the Consultant knew what he was doing. We no longer use this consultant but I am now tasked to move data around and add two new VM Hosts to the network. At this point I can't change the configuration so I might as well leave it alone.

I want to move our SQL database from a virtual drive on our SAN to local SSD drives on one of the Hosts but first I need to create an iSCSI connection from our SAN to the new VM host server but have been having issues creating the iSCSI connection from the SAN to the new VM host. I want the SQL Data to be on the SSD's but all the other drives to continue to reside on the SAN and that is why I need to create the iSCSI connection. I have the QNAP NAS devices connected but the SAN is a little more difficult. You have given me a bunch of reading material but I am having a hard time understanding some of this information.

Okay, so lets put a line under QNAP (nothing can be done here!)

EMC SAN issue....

can you ping the IP Address of the EMC SAN ? (from the console of ESXi server, you must use VMKPING not ping).

All iSCSI can use the same network and IP Addresses, bad idea but it can to keep it simple....

do you have any dedicated network switches to use as a dedicated storage network?

We have one 10GB switch that the NAS and SAN connect to as well as the VM Hosts. 2 of the hosts have 10GB nics and the other three have 1GB NIC's. All hosts have 4 NIC's to each physical Host.  I hope that is what you are asking.

Okay, so with all this connectivity, is the network flat, e.g. one subnet network ?

can you vmkping the SAN ? from the hosts ?

Yes, for the most part this is a flat network. We have a separate subnet for our VOIP phones but we really don't use it for our data network.

Now as for Pinging the SAN from the Hosts, how is that done? On a Windows machine I can go to the command prompt and run the ping but on the Host I don't know where that can be done?

You need to login to the console or remotely via SSH, and issue the command VMKPING on the ESXi server.

So I connected to the server using PuTTY and when I issue the command VMKPING it comes back telling my the VMKPING not found.

Nevermind on that last post. It doesn't like the command to be in capitals. Anyway I am able to ping the SAN through the console when connected to the host through PuTTY

and that was using vmkping, and not ping this is very important ?

what is the IP Address of the SAN, and IP Address of the VMKernel Portgroup ?

Hi,
+1! vmkping, not ping command.
And you must vmkping the Unity's iSCSI IP, not the Unity's management IP of course.
And with wmkping I think you must specify the source IP, using the ESX iSCSI vmkernel number in the command
example in ssh from ESXi : vmkping -I vmk1 Unity_iSCSI_IP
(https://kb.vmware.com/s/article/1003728)

But IMHO implementing the Unity properly should better be done by a specialist..

OK, I am very confused right now. My EMC has 4 different IP addresses. When I go to Storage Adapters and click on the iSCSI Software Adapter then click Properties then dynamic discovery and add one of the 4 IP addresses to the target it adds it but does not show as a datastore. I have gone on the EMC Unity and added the Host to ESXi Hosts which added the IP as a Localhost.domain.com but does not show the vCenter IP. I don't know how to get this to add to my new Host in vCenter.

The IP address of the SAN I am really confused on. When I look at a host that has a connection to the SAN it shows 4 IP addresses. 192.168.0.95, 96, 97, 83. The IP address of the host is 192.168.0.64. In the documentation above (HOW TO: Add an iSCSI Software Adaptor and Create an iSCSI Multipath Network in VMware vSphere Hypervisor ESXi 5.0) I have followed the instructions however when I go to Static Discovery and add the IP address of the EMC device it wants an iSCSI Target name. I don't know what to put there.

use Dynamic Discovery, and just add IP Address of SAN and Port number 3260.

You will need to add your hosts and configure LUNs on the EMC Unity SAN.

I added one of the IP's that I sent you .95 and had the system rescan but nothing popped up in the Details section of that HOST. When I did the rescan it popped up with the EMC in the details section but it shows as unmounted with 0 capacity

Firstly, The re-scan, will just re-scan for LUNs (that you have to configure on the SAN).

Next configuration, is to select Add (new) Datastore (new is missing).

When you click Add Datastore - does anything appear ?

Have you configured any LUNs on the SAN ?

So I found that I needed to mount the iSCSI connection but after mounting the connection the EMC shows no disk space. I tried to Add a new datastore yet I didn't have to do that when I connected to the NAS devices but when I go add the datastore nothing appears.

I am missing something. I really don't need to create a new databstore but to have access to the data that is stored on the SAN. Just to recap what I am trying do is to move the VM's that are on one host I am trying to reconfigure to a new host temporarily so I can change the configuration on the one host then move one of the VM's that were on that host back to it and move a drive that is sitting on the SAN to an SSD datastore I am creating on the current host. Since this is our production database that holds all the business information I trying to be careful.

The SAN already has one LUN configured which holds all the data for 4 VM's.

So I found that I needed to mount the iSCSI connection but after mounting the connection the EMC shows no disk space. I tried to Add a new datastore yet I didn't have to do that when I connected to the NAS devices but when I go add the datastore nothing appears.

I'm not sure how datastores would have been created using the NAS, without using Add Datastore.

Ah, so you already have LUNs created on the EMC SAN ?

In that case all you need to do is make sure that your hosts (new host) has been granted access, by adding the IQN of this host to the LUN.

Hi,

So I found that I needed to mount the iSCSI connection but after mounting the connection the EMC shows no disk space. I tried to Add a new datastore yet I didn't have to do that when I connected to the NAS devices but when I go add the datastore nothing appears.

yes, before adding a new datastore, you got to create a LUN ad map it to the right hosts on your Unity
And before that you got to add your hosts (with their iSCSI identifier, IQN), in Unity GUI (Unisphere).

The SAN already has one LUN configured which holds all the data for 4 VM's.

So your hosts are already created on the Unity ? unclear...

I think I have figured it out to some degree however when I go to add the datastore to this host it now see the EMC however I just want to add it to my HOST and not format the whole drive. When It says formatting and the choice is the whole drive is it going to format the whole drive or is it just going to allow me to access it. I think I am almost there but don't want to screw up the whole unit since it is in production and holds Mission critical servers on it.

You should NEVER have to click Add Datastore if the Datastore already exists on other hosts!

Add Datastore should really Read ADD NEW DATASTORE!

Check in Devices under Storage Devices does the LUN exist ?

A Re-scan should find the Datastore already..... if it does not there are other reasons it may not mount it....

Did you add the IQN then correctly ?

I'm glad I didn't throw caution to the wind and proceeded.

When I check in Devices under Storage Devices I see the LUN and it is mounted with the transport as iSCSI.

As for adding the IQN correctly well I really don't know. I went back into my Unity and was clicking through all my choices in Storage and found that I did not add access to the Pool for the storage unit. I had added access to Block and VMware but not pools. Once I added access to that host on Pools the storage unit appeared in Datastores so I assume we are good.

What can I do to assure that this configuration is correct?

I have a related question about my Unity and Initiators. When I look at the initiators there are 10 Initiators that are visible but it shows that they are no Bound and the iSCSI Type is Dependent and not Software. Do I need these? Can I delete them? How many Initiators should there be on each Host?

Okay.... this is good that their is a LUN, in storage devices - which could be a Unity LUN...

There is an initiator per host (usually).

There could be a good reason for WHY, it just does not appear as a datastore...... and to diagnose that we need to deep dive into the logs...

So before we start looking at logs... you need to get access to the console or remotely via SSH to run some commands

(and before you ask, this is sometimes normal!)

Hi,

If you have hardware dependent iSCSI initiators on your ESX well why not, in fact this is better from an ESX cpu performance side because some load will be offload to the iscsi hardware adapter (instead of software iSCSI).

if you have 10 initiators in Unisphere and only 3 ESX, well, there is too many..!
From best practices side you should have 2 initiators per ESX.
I did not remember for sure, but if you were to use ESXi software iSCSI, you have only one IQN, so one initiator per host in Unisphere.
Even if the ESXi soft iSCSI is linked to 2 vmnic (which is also a basic best practices for redundancy).

I Unisphere when you create a host, you specify which are its IQN. So you got to identify which IQN belong to which ESX.
And as already said it seems you already have at least one ESX created in Unisphere because you have a Unity datastore in vsphere.

Thanks for the answers. I still really don't know what to do about the extra initiators. I have been trying to correct some of the configuration errors done on each VM Host but it is difficult since I really don't understand some of the details of the VM environment. On three of the machines I have fail over NIC's and configured them using this document from above which has been really helpful .

HOW TO: Add an iSCSI Software Adaptor and Create an iSCSI Multipath Network in VMware vSphere Hypervisor ESXi 5.0

How do you delete extra initiators on VM Hosts or is there a way? Right now I don't think I need to worry about them but I would like to fine tune these hosts so that they work optimally in the end.

So much to learn on the fly.

okay, so what about SSH access so we can check this LUN/Datastore ?

I have installed the CLI onto the vSphere server and have been able to ping the server so yes we can do whatever tests you might suggest. What tests to I need to run?

I'm afraid you need to login to the console of the ESXi server (e.g. at the keyboard and monitor of the server).

or remotely via SSH (e.g. PuTTY login)

I have PuTTY installed on my workstation and have used it to connect to the server but don't know what you want me to run.

Connect to the ESXi server, and I'll tell you what to type, let me know when you are connected.

OK, I am connected to one of the ESXi host machines.

Please type the following;-

esxcfg-volume -l 

Select allOpen in new window

hopefully it will respond with something like...

~ # esxcfg-volume -l
VMFS3 UUID/label: 4cc89503-f054d0d4-2865-00221958e471/nas3-vmfs
Can mount: Yes
Can resignature: No (the volume is being actively used)
Extent name: naa.600140588e9bab7d2b7ed4b53dadc1de:1     range: 0 - 1048319 (MB)

do you get any output ?

How do I get to the # prompt? When I try to run the command above it tells me -sh: escvfg-volume: not found. I changed directory to Bin and got the same result. Also, when I connected to the ESXi Host that was the original host for the guests I want to move back to it comes up with this alert:

The server's host key is not cached in the registry. You have no guarantee that the server is the computer you think it is. The server's rsa2 key fingerprint is: ssh-rsa 2048 <Long Hexadecimal number>
If you trust this host, hit yes to add the key to PuTTY's cache and carry on connecting.

It just sounds like a warning so I clicked yes.

I found that I fat fingered the command but when corrected it just comes back to the prompt.

oh dear, no datastores found.

looks like log time, to be honest with you....

and this is very difficult to try and fault find this environment, via a forum...

I think there are many things wrong here...

OK, I am confused and maybe we are going down the wrong path. Right now when I look at the Original Host > iSCSI Software Adapter > Details pane (Devices) I see the stores. When I go to Storage in the Hardware section I see both QNAP stores, EMC store and the new store I created using the local SSD drives on the Host. This was the main reason I moved all the guests off the host because I wanted to create this new store so I could move our SQL database to those SSD drives to increase performance.

Would you like to see screen shots?

If you move your SQL servers off shared storage onto local storage, you lose ALL VM functions with them in terms of Cluster features ?

Screenshots may help...

but need them of ALL hosts, networking, storage, datastores

I don't understand. Wouldn't a datastore that I created on a host device be considered shared? I feel i have put myself I a bad spot. I have already moved my SQL server off a clustered machine to an un-clustered host so now you are saying I cannot move it back or are you saying I can't move just the SQL data to a new datastore I made on a hosts SSD drives? Does the whole guest (all data and drives) need to be on one SAN/NAS?

It's only shared if it can be accessed by more than one host.

e.g. Fibre Channel, iSCSI and NFS are Shared Storage, vSAN is shared storage.

anything like SATA, SAS, SSD is Local Storage.

You can move it back, or should be able to move it anywhere

I see. At this point I am not really concerned about v Motion and more on getting better performance on my SQL server since it serves our entire organization. Since the Host that has the SSD drives is in a cluster would you recommend removing that server from the cluster or maybe getting another NAS that is fully SSD and flash and add it to the network then give access to all hosts?

okay, I think we deviated off the original question, because SQL performance is a large topic!

You are right. When I look at the topic header I needed help with setting up iSCSI which I have accomplished. I will need to start a new topic on this. Thanks for all your help on this question though and I will start a new one on SQL performance however it deals with VM and placement of the database but I will address it in the new question.

lets discuss in new question, and get some SQL DBA input for you!

Thanks to the both of you for assisting me with this solution. You both provided great information which helped me solve my problem.