We help IT Professionals succeed at work.

Check out our new AWS podcast with Certified Expert, Phil Phillips! Listen to "How to Execute a Seamless AWS Migration" on EE or on your favorite podcast platform. Listen Now

x

SAN freezes with Server Disk Management.  Worked before rebuilding Server.

High Priority
88 Views
Last Modified: 2020-06-08
Hello, I have an old EMD Astra ES 12 SAN which has been working for the last 2 years no issues. My DC went down last weekend and I had to rebuild it. I started from scratch Server 2012 R2. There is a Qlogic Fiber card in there which attached directly to the SAN.

The SAN is setup RAID 6 16TB and the Qlogic sees it and it assigned to the card..

I am NOT very experienced with SANS but enough I have always maintained them and able to fix minor issues. I was able to build this in the first place using old parts due to budget constraints.

The SAN is so old and the company is not around and there are no firmware updates, etc. I have the latest drivers for the Qlogic.

At first I had a problem with Disk Management not seeing the drive. I found out this was Mcafee so for now I have uninstalled that (this is what I used in the past but it was installed AFTER the SAN was working).

Now one of 2 things happens.. Either I open Disk management and it freezes at "connecting to virtual disk service" and I can close it and re-open but it does the same thing.. The other thing I get is that It open and sees the 16TB drive and it is online but when I go to assign a drive letter it freezes.

Either way, Disk management does not like this device.

Tried windows updates, Qlogic updates, etc. Nothing seems to help. I have tried shutting down the SAN and them powering on AFTER I am in disk management and it sees it, but then locks up when I click on refresh.

This data is critical and our backup failed 2 weeks ago. I am moving to a new SAN but need the data off it. There is about 10TB data.

ANY ideas? Suggestions?

Event log shows:
SYSTEM - The IO operation at logical block address 0x40918 for Disk 1 (PDO name: \Device\0000005e) was retried.

  - Provider
   [ Name]  disk
  - EventID 153
   [ Qualifiers]  32772

Everything was working until I lost the DC. The DC was upgraded to 2012 R2 last year so the SAN was grandfathered in from Server 2003, but I built in a few years back. I dont remember doing anything differently. Same drivers, same SAN, same server, Same OS,

I tried new fiber cable. I tried another Qlogic card. I am using SanSurfer and the QCC GUI. Those detect the SAN and 16TB.

I am stuck. Worked on this a full week. I will get this to another drive ASAP but need to get to the data,

Thanks ahead.
Comment
Watch Question

andyaldersaggar maker's framemaker
CERTIFIED EXPERT
Distinguished Expert 2019

Commented:
I do not know of EMD Astra but I found a line drawing of one and it looked very much like a LSI Engenio, can you paste a photo of it from the back so we can confirm.
johnny prescottIT Director

Author

Commented:
Thank you. This is the bad one, but they are identical. I am only using the fiber port for connectivity and Ethernet port for management. I believe I have a manual too but its on the SAN. ugh.The controllers and power supplies are all testing out OK. No errors .



andyaldersaggar maker's framemaker
CERTIFIED EXPERT
Distinguished Expert 2019

Commented:
It's not LSI Engenio based, the line drawing I saw was nothing like that.
johnny prescottIT Director

Author

Commented:

So some new info... 
CERTIFIED EXPERT
Distinguished Expert 2019

Commented:
Check LUNs allocations to see whether the new setup is authorized to access the LUN
johnny prescottIT Director

Author

Commented:
I believe the Luns are allocated properly but not sure. I can get it to this point (see pic)

johnny prescottIT Director

Author

Commented:
I dont know if this has anything to do with it but there is only one Qlogic card in there. I dont know why it shows so many Hostnames.

CERTIFIED EXPERT
Distinguished Expert 2019

Commented:
How many systems access the SAN?
johnny prescottIT Director

Author

Commented:
Fo one second I though I had something. It shows up in Disk management as D: drive. There was a D drive before and it acted like there was nothing there so I went to change the Drive letter and it locked up again. This was the first time I not only got it to show up in disk management (there were a few times it showed up OFFLINE) but something still isnt right. 

johnny prescottIT Director

Author

Commented:
Id be happy to pay someone for an hour of their time if they really know SANS and could do a screen share. Venmo or whatever. I can show you my company is legit, etc.
johnny prescottIT Director

Author

Commented:
Im not proficient in this so hopefully Im making sense, but this is literally a SAN, Raid 6 with a fiber connection to a Qlogic card in Server 2012 which is our Domain Controller. From there we connect it through file shares and that's all its used for. The iSCSI cable is not being used. It was working before with it plugged in not being used. Only one system is connected.

I have a plan to move away from this but need to access the data to move it elsewhere.
johnny prescottIT Director

Author

Commented:
After Disk Management locks up I typically have to reboot the server. And then things get funky. For example this time the configuration changes and the SAN shows no initiator logged in. Rebooting the SAN typically gets it back to where the Qlogic shows it online again
Everything just seems so volatile.


johnny prescottIT Director

Author

Commented:
By the way, there's nothing I could have done to wipe out the file system on it. The only thing I have done is tried to bring it online or try to change the drive letter. Nothing else.
CERTIFIED EXPERT
Distinguished Expert 2019

Commented:
Different sans have different ways to achieve things. Check the qlogic settings, configuration.


Confirm the mapping of LUNs are right.
Potentially the LUN is allocated based on the qlogic ID....
Does the qlogic drivers, utility installed on the server, cannot see the LUN, what does it say?
johnny prescottIT Director

Author

Commented:
Im going to try this. Please let me know if this is a bad idea:

https://docs.microsoft.com/en-us/previous-versions/windows/it-pro/windows-server-2012-r2-and-2012/jj134187(v=ws.11) 

Deploy Hyper-V over SMB


I read it may work when you run into this.

CERTIFIED EXPERT
Distinguished Expert 2019

Commented:
Did the prior setup was virtualized?
Smb network share?
169.254 is an address block when'dhcp is unavailable on a DHCP configured network..
johnny prescottIT Director

Author

Commented:
Previously I did have Hyper-V installed and had a virtual server running but not related to the SAN. I understand the 169.x block. The DC is assigned a static IP and DHCP comes from the firewall and is working. Does the Qlogic need an IP address in our network range to work?

I dont know where that 169 address is coming from. I do an ipconfig/all on the server and there is no reference to it.

The server is also DNS.
CERTIFIED EXPERT
Distinguished Expert 2019

Commented:
Wastage SAN used asthe data repository for the hyper-v host?
The data should remain as long as you did not initialize the LUN

To reconstruct, one has to have some background on what or how it was used....
johnny prescottIT Director

Author

Commented:
arnold. Im sorry I did not understand your post.
CERTIFIED EXPERT
Distinguished Expert 2019

Commented:
The SAN you are using, I am unable to locate info on it.

Reconstructing how it was used before.

Are you certain the SaN was not previously used by a hyper-v host as storage space for vhdx files.

Looking through the San interface, options logs if any.
Qlogic. Has a command line interface if not mistaken that deals with querying the SAN to see what is available.

Is there another system successfully accessing. Using san resources?
johnny prescottIT Director

Author

Commented:
No, it previously was not used by hyper-v host. It previously had a drive letter in Windows Server 2012 and was shared out.

Last night I ran Diskinternals Partition recovery and it sees the NTFS Partition fine and says there is nothing wrong with it. But Windows doesnt see it. Here is what SysInternals sees it as:



andyaldersaggar maker's framemaker
CERTIFIED EXPERT
Distinguished Expert 2019

Commented:
See if the trial version of https://www.runtime.org/captain-nemo.htm sees the data.
CERTIFIED EXPERT
Distinguished Expert 2019

Commented:
IT IS missing drivers to handle the LUN presented to it. It sees it as a RAW partition while it should be seeing it in  a different way.
The LUN contains several GPT parttions based on your earlier image.

the 2012 you installed, BIOS or EFI?

THE LUN mapping might be that you need to access the portion that identifies the GPT partition an.....
johnny prescottIT Director

Author

Commented:
It is a Dell R900 and does not have UEFI. This is the same server it was previously working on before reinstalling the OS.




CERTIFIED EXPERT
Distinguished Expert 2019

Commented:
Did the SAN Qlogic HBA came with software..
You loaded the qlogic driver. To handle the LUN access, you may have had software that handles ePartitiis to present tothe system.
I
You are missing something.

I.e. EMC sans have powerpath though that deals with handling multiple path.

The LUN contains to GPT partitions
Was the system before booting from the SAN or the disks you have inthe system arethesameones.

Though itis obtain indication as one ould have a bootable HD that half way through transitions to the SAN

Look at test-disk utility to see if you can attach it to the smaller GPt partition to see the content of that partition to see what is there I.e. Windows ...
johnny prescottIT Director

Author

Commented:
I am looking into it now. Here is the card and drivers section for it:

http://driverdownloads.qlogic.com/QLOGICDRIVERDOWNLOADS_UI/ResourceByOS.aspx?productid=934&oemid=65&oemcatid=58603 

I currently installed sansurfer and QConvergeConsole  but I havent seen anything like you are talking about as far as being able to test and see partitions, etc.
andyaldersaggar maker's framemaker
CERTIFIED EXPERT
Distinguished Expert 2019

Commented:
It's single controller so no software is needed to handle multipath, whether server is in UEFI or BIOS mode does not matter since you are not trying to boot from it. You would be unlikely to see the partition table if you had the wrong drivers for the Qlogic card. It's probably corrupt from the server crashing which is why I suggested looking with Captain Nemo or EaseUS data recovery wizard or similar.
CERTIFIED EXPERT
Distinguished Expert 2019

Commented:
If the server crashed, the LUN is an inclusive RAW partition, therehastobe soemthing that strips/deals with the overlay and presents to the system the partition 1002 and  Volume 2 as two separate partitions.

The multipath example was provided as an example.

In the qlconvergeConsole, can you link, present to the OS just the two volumes mentioned GPt partition versus the entire LUN

In the current way, Windows does not have
CERTIFIED EXPERT
Distinguished Expert 2019

Commented:
On the back, you have an FC connection, a network contnection
What is the blue tagged connection, a display, serial or a sas connector?
andyaldersaggar maker's framemaker
CERTIFIED EXPERT
Distinguished Expert 2019

Commented:
It's a SAS cable.
CERTIFIED EXPERT
Distinguished Expert 2019

Commented:
Andy, are you familiar with the SAN in use? Spec referenc?
The system seems to have another FC card slot, does this mean it has TWO controllers /paths for redundancies.
The sas cable connected to an external TAPE or to the Dell R900 via an external SAS hba?
andyaldersaggar maker's framemaker
CERTIFIED EXPERT
Distinguished Expert 2019

Commented:
I don't have to be familiar with it to recognise a SAS cable, it should go to an additional enclosure although I can't see one in the rack, There's always 2 or more FC ports since if there were two controllers and a pair of FC switches one cable from each controller would go to each switch.
johnny prescottIT Director

Author

Commented:
The SAS cable is not being used. Ive actually removed it since I took the photos.
The SAN has 2 controllers. I am using one controller only. One fiber connection to a single Qlogic Fiber card. I will be trying the Caption Nemo software this evening. I tried another partition recovery software this morning and it saw the NTFS partition but didnt know what to do with it.

Also looking for Qlogic software which will let me connect to a certain GPT partition. 
CERTIFIED EXPERT
Distinguished Expert 2019

Commented:
is there a FC switch, zoning related thing that needs to be addressed.
The disks need to be presented by other means if the way the current stooge is being accessed is as a DAS via sas cable
CERTIFIED EXPERT
Distinguished Expert 2019

Commented:
Ok, sas not in use.

It is unlikely to be software from qlogic, it would be software that came with the San
Note the structure
Unallocated
 GPT partition
Unallocated
GPT partition
Unallocated

Do you have a backup 17TB of the San. Tape?
When used before, did you have two additional disks 128MB and a 16TB volumes?
johnny prescottIT Director

Author

Commented:
I created the SAN directly from the web interface GUI and only created one partition on my end.
It is pretty basic setup. Add some drives and create a logical drive. In this case RAID 6 and there are a few settings and that's really it. No other software. Theres not much more to it. 

You can see the basic setup. Not my original settings, but I just clicked on it to show here.



johnny prescottIT Director

Author

Commented:
My offer is still out there.. Ill pay someone to take a look at it. :)
CERTIFIED EXPERT
Distinguished Expert 2019

Commented:
Look on page 16 of the documents
Your setup right now is as a Direct Attached storage which explains why the entire 16.3TB volume from your storage enclosure is being presented.

Let's try it this way, the R900 if not mistaken has four network ports did you conect and configure one of them to connect to the network switch 169.254.127. Or 169.254.198. IP address to reflect the hostnamess in the FC/hba info?

Did not go further to see which software beyond.....
Based on the document, 2003 was the last Windows supported by the device.

You need to get your configuration of the 2012 to match the outlined process for SAN functionality versus DAS.

Will continue to look through the document and provide info in the event it might help.
CERTIFIED EXPERT
Distinguished Expert 2019

Commented:
Do you have the CD with AstraPath installed?
andyaldersaggar maker's framemaker
CERTIFIED EXPERT
Distinguished Expert 2019

Commented:
>Do you have the CD with AstraPath installed?

Albert, they only have one FC connection, multipath software is not needed. They are using it as a DAS, there is a minor data corruption in a partition table or the GUID table that DR software can recover from.

I do wonder why they had a SAN in the first place though, a good RAID controller plus external disk shelf would have been cheaper and faster than this when they bought it 10 or so years ago and still is if they only have one host server.
CERTIFIED EXPERT
Distinguished Expert 2019

Commented:
Setup can be used as a SA S attached DAS, or an FC DAS, it can also be used as a SAN and all depends on how they connect the thing
The SAS channel is 3GB not sure which FC mode they have 2,4,8 GB channel.

Not sure what the overall partition table will be if the volumes within are GPTs.
johnny prescottIT Director

Author

Commented:
Andyalder..

I can look for the software. I do not have anything for the SAN or card..

Why we have it is.. about 10-15 years ago the company I now work for did e-discovery and we had about 500TB across EMC Sans. The company had about 200 employees. I came on a few years ago and there is only 5 employees and we left our datacenter to a small office with a closet. I had a zero budget and these were laying around and they sold off all the larger equipment. But we still had about 8TB of Client data so this is what I had to work with.

Arnold. We dont have anything on 169.x.x.x I dont know where that IP is coming from. Would it matter in the way I am using the storage?

I agree 2003 was the last version supported but I did have it running fine on Server 2012 r2.

Thank you everyone. Hopefully I can get back into the office tonight. otherwise Ill check on some of these items tomorrow and get back to you.


CERTIFIED EXPERT
Distinguished Expert 2019

Commented:
Did you setup th system when Windows server 2012 r2 was installed?

The 169.254 might be a way the astrpath in combination with the qlogic FC HBA connect the specific 16TB partition as the D drive. Getting the thing to function as a SAN <storage array Network) versus what it appears now as a direct attached storage where the server sees the entire storage space while you are trying to access a specific GPt partition.
johnny prescottIT Director

Author

Commented:
originally set this up as a 2003 server and I didn’t have the CD at the time. about a year ago I switched to 2012 because that’s all I had access to and I didn’t have the CD.
johnny prescottIT Director

Author

Commented:
maybe I could try server 2003 again just to see if it works? i’ll try a couple of the other suggestions tonight first.
Believe it or not we had another one of these units I was backing up to just a week ago and the unit completely failed (hardware failure). it took four drives with it. needless to say it’s been a rough two weeks LOL
CERTIFIED EXPERT
Distinguished Expert 2019
Commented:
Unlock this solution with a free trial preview.
(No credit card required)
Get Preview
johnny prescottIT Director

Author

Commented:
OK everyone. First of all thank you everybody.. Here is where I am at.

Still havent got the SAN working BUT I took Arnold advice on the backup SAN I had and forced the drives online and replaced 2 drives that were definitely bad. So that is rebuilding and shows OK. I telnet in which I had not thought about before so this was a lifesaver for me.

I may go back and work on the original issue but it really was just about getting my data off of there and I can do that now. That you VERY much everyone. You were a huge help and saved my butt. 
andyaldersaggar maker's framemaker
CERTIFIED EXPERT
Distinguished Expert 2019

Commented:
WTF!  Question is about this SAN not some backup one you may have,
johnny prescottIT Director

Author

Commented:
Yes, but I mentioned a backup SAN in my post earlier and thought it was completely dead. Arnold walked me through checking something and got that one up and running.

I understand the "WTF" comment but wouldn't have resolved my problem without going through all those steps. Isnt the end result supposed to be getting someone back up and running?

 So once again thank you everyone.
andyaldersaggar maker's framemaker
CERTIFIED EXPERT
Distinguished Expert 2019

Commented:
Going through all those steps???

I just downloaded and ran Captain Nemo in 45 seconds!
johnny prescottIT Director

Author

Commented:
andyalder.. what is your point?

I was at home and the equipment is at work.. nearly an hour away.

Why are you being argumentative? Is that how these boards work? I thought people pay to get advice here and many times to get their business back up in running in an emergency, which is what I did.
Unlock the solution to this question.
Thanks for using Experts Exchange.

Please provide your email to receive a free trial preview!

*This site is protected by reCAPTCHA and the Google Privacy Policy and Terms of Service apply.

OR

Please enter a first name

Please enter a last name

8+ characters (letters, numbers, and a symbol)

By clicking, you agree to the Terms of Use and Privacy Policy.