Question

Cannot delete NAS Storage Reference

Asked by: wct296

I recently upgraded one of my servers from Vmware ESX 3.0.2 to ESX 3.5... Before the upgrade I had in place a NAS storage reference which was working fine.. After the upgrade something went wrong... Now I have a file in the /vmfs/volumes folder that I cannot delete and as a result I cannot re-link ESX to the NAS storage...

I know the NAS storage box is working because on my other ESX server I can connect to it fine..

The main problem I have is a linux one... There is a file in the directory that does not list properly.. (see below for e24c5a25-3aa7c729)

[root@ESX1 volumes]# ls -l
ls: e24c5a25-3aa7c729: No such file or directory
total 1024
drwxrwxrwt    1 root     root         7140 Jan  3 14:30 473b1a29-80b7f018-f3ce-001d09142fc7
lrwxr-xr-x    1 root     root           17 Jan  4 12:13 CLARKE E DRIVE -> e24c5a25-3aa7c729
lrwxr-xr-x    1 root     root           35 Jan  4 12:13 storage1 -> 473b1a29-80b7f018-f3ce-001d09142fc7

and
[root@ESX1 volumes]# ls
473b1a29-80b7f018-f3ce-001d09142fc7  CLARKE E DRIVE  e24c5a25-3aa7c729  storage1

Thinks I have tried

1. rm -rf file
2. Deleting the VmKernel (used to create the NAS)
3. chown to root (failed)
[root@ESX1 volumes]# chown root e24c5a25-3aa7c729
chown: failed to get attributes of `e24c5a25-3aa7c729': No such file or directory

Its starting to point to corruption... and I'm at a loss as to how to delete a file that the system thinks isnt there.. My next step is to completely rebuild the box from scratch, which is a huge amount of work and downtime for the organisation.

Any ideas on how to delete this file or recover from this situation greatly appreciated.



This Question has been solved and asker verified All Experts Exchange premium technology solutions are available to subscription members.

Subscribe now for full access to Experts Exchange and get

Instant Access to this Solution

  • Plus...
  • 30 Day FREE access, no risk, no obligation
  • Collaborate with the world's top tech experts
  • Unlimited access to our exclusive solution database
  • Never be left without tech help again

Subscribe Now

Asked On
2008-01-03 at 19:24:51ID23057703
Tags

VMWare

,

ESX Server

,

3.5

Topics

VMware

,

Linux

Participating Experts
4
Points
500
Comments
13

Trusted by hundreds of thousands everyday for fast, accurate and reliable tech support.

  • "The time we save is the biggest benefit of Experts Exchange to Warner Bros. What could take multiple guys 2 hours or more each to find is accessed in around 15 minutes on Experts Exchange." Mike Kapnisakis, Warner Bros.
  • "Our team likes having a resource that is more secure than just using Google and most experts using this service really know their stuff. It's nice to look here first versus using Google." Dayna Sellner, Lockheed Martin
  • "Anytime that I've been stumped with a problem, 9 out of 10 times Experts Exchange has either the accepted solution or an open discussion of the potential solution to the problem." Kenny Red, eBay Inc.

See what Experts Exchange can do for you.

Got a question?

We've got the answer.

Experts Exchange has been collecting answers to technology questions since 1996…3 million and counting! If you have a question, chances are we already have your answer.

Screenshot of Experts Exchange Knowledgebase

Need individual assistance?

Our experts are ready to help.

If you can't find the exact answer you're looking for, ask our exclusive community of 50,000 experts. You’ll get a personalized answer from a trusted professional.

Screenshot of Experts Exchange Knowledgebase

Want to learn from the best?

Read articles from industry experts.

Thousands of free tech tips, tricks, how-to’s and tutorials are available in our peer reviewed articles section. See for yourself how smart our experts are, no login required.

Screenshot of an Article

Working on a long term project?

Store your work and research.

Save solutions to your questions, answers you’ve discovered through searching plus helpful articles in your personal knowledgebase for easy future access.

Screenshot of Experts Exchange Knowledgebase

Access the answers to your technology questions today.

Subscribe Now

30-day free trial. Register in 60 seconds.

What Makes Experts Exchange Unique?

Members of the expert community talk about why the experience at Experts Exchange is different than what you will find anywhere else.

Trusted by the world's most respected brands.

image of each brand's logo

Faithfully serving IT professionals since 1996.

Experts Exchange Logo

Try it out and discover for yourself.

Subscribe Now

30-day free trial. Register in 60 seconds.

Related Solutions

  1. Connecting ESX to NAS
    Hello, I have a 3 X HP DL585 server setup with ESX 3.5 with 2 X 146 GB drives (Raid 1) each and a NAS device HP Storageworks AiO1200 with 4TB of space. I need to create the VMs from the ESX server on the NAS device. How do I get the ESX servers to see the NAS device as a ...
  2. ESX 3.5 and network nas
    I'm trying to do some studying of ESX 3.5 on my own hardware at home. I have two boxes ready for ESX 3.5. And I was wondering if ESX servers will support a NAS for VM storage if the connection between the Nas and the severs are gigabyte Ethernet instead of Fiber. I like to tr...
  3. NAS Enclosure for ESX Server
    can someone recommend a NAS enclosure that supports the following: NFS v3 TCP only support for NFS No root squash so ESX can have full access in writing, reading from NAS. I just want a cheap solution currently have a EMC clarion FC for production this NAS would be f...
  4. Mounting NFS NAS in ESX 3
    I am testing ESX 3 (Not using ESXi 4 due to not having a 64 bit server). It is up and running and everyting seems to be fine. I have a Western Digital 4TB NAS that wanted to use as a datastore. The NAS has NFS and it is running and allowing connection from the correct IP's. E...
  5. Using a NAS with VMWare
    Hi Experts. I am getting into the action a little late with VMWare and just setup my first server and guest OS. Now that I have the itch, I realize I am going to be out of hdd space soon. I planned on purchasing a NAS as I am running low on my file server but with the addi...

Free Tech Articles

  1. WARNING: 5 Reasons why you should NEVER fix a computer for free.
    It is in our nature to love the puzzle. We are obsessed. The lot of us. We love puzzles. We love the challenge. We thrive on finding the answer. We hate disarray. It bothers us deep in our soul. W...
  2. SCCM OSD Basic troubleshooting
    SCCM 2007 OSD is a fantastic way to deploy operating systems, however, like most things SCCM issues can sometimes be difficult to resolve due to the sheer volume of logs to sift through and the dispe...
  3. Migrate Small Business Server 2003 to Exchange 2010 and Windows 2008 R2
    This guide is intended to provide step by step instructions on how to migrate from Small Business Server 2003 to Windows 2008 R2 with Exchange 2010. For this migration to work you will need the fo...
  4. Create a Win7 Gadget
    This article shows you how to create a simple "Gadget" -- a sort of mini-application supported by Windows 7 and Vista. Gadgets can be dropped anywhere on the desktop to provide instant information, ...
  5. Outlook continually prompting for username and password
    There have been a lot of questions recently regarding Outlook prompting for a username and password whilst using Exchange 2007. There are a few reasons why this would happen and I will try to cover t...
  6. Backup Exchange 2010 Information Store using Windows Backup
    There seems to be quite a lot of confusion around the ability to backup Exchange 2010 using the built in Windows Backup feature. This stems from the omission of this feature prior to Exchange 2007 s...

Cloud Class Webinars

  1. Avoiding Bugs in Microsoft Access
    Alison Balter takes and in-depth look at avoiding bugs in Access. In this webinar you will learn about using the immediate window to debug your applications, invoking the debugger, using breakpoints to troubleshoot, stepping through code, setting the next statement to execute, ...
  2. Top 10 Best New Features in Visio 2010
    Scott Helmers gives live demonstrations of the top 10 new features in Visio 2010. This webinar will teach you how to create compelling diagrams by adding shapes to the page with a single click, linking the shapes in a diagram to data in Excel (or SQL Server, or SharePoint), ...
  3. IT Consultant Business Secrets Revealed
    Michael Munger, Experts Exchange tech pro and IT consultant, pulls back the curtain on his very successful businesses and answers question on every IT consultant and business owner should know about. He shares secrets on what he did to solve the 5 most common problems in IT, ...
  4. Disaster Recovery and Business Continuity
    Quest CTO, Mike Billon, gives an overview of the steps involved in building a dunamic disaster recovery plan. Through case studies and an examination of software/hardware tooles for monitoring and testing, you'll gain a better understandin of where you are, where you want ...
  5. Organize Your Visio Diagrams with Containers and Lists
    Scott Helmers uses cross functional flowcharts, wireframe diagrams, data graphic legends and seating charts to teach you: how to ustilize all three new structured diagram components in Visio 2010, the best practices for organizeing shapes in previous version of Visio, how to organize ...
  6. How to Us Objects, Properties, Events and Methods in Microsoft Access
    Alison Dalter gives an in-depbth look at objects, properties, events and methods in Microsoft Access. In this webinar you will learn about using the object browser, referring to objects, working with properties and methods, working with object variables, understanding the ...

Join the Community

Give a Little. Get a Lot.

Join the community of experts here and help other tech pros by answering question in your area of expertise. You can earn FREE access to all Experts Exchange's premium features and resources.

Join the Community

Answers

 

by: murugesandinsPosted on 2008-01-03 at 20:34:26ID: 20580004

It may be useful for the viewers to know the output of the following commands:

pwd
ls -latr
echo $?
ls -ld "e24c5a25-3aa7c729"
echo $?
ls -ld "CLARKE E DRIVE"
echo $?
pwd

 

by: wct296Posted on 2008-01-03 at 21:36:24ID: 20580190

Commands and output below - thanks

[root@ESX1 volumes]# pwd
/vmfs/volumes
[root@ESX1 volumes]# ls -latr
ls: e24c5a25-3aa7c729: No such file or directory
total 1024
drwxrwxrwt    1 root     root         7140 Jan  3 14:30 473b1a29-80b7f018-f3ce-0                                             01d09142fc7
drwxrwxrwt    1 root     root          512 Jan  4 10:32 ..
lrwxr-xr-x    1 root     root           35 Jan  4 14:33 storage1 -> 473b1a29-80b                                             7f018-f3ce-001d09142fc7
lrwxr-xr-x    1 root     root           17 Jan  4 14:33 CLARKE E DRIVE -> e24c5a                                             25-3aa7c729
drwxr-xr-x    1 root     root          512 Jan  4 14:33 .


[root@ESX1 volumes]# echo $?
0

[root@ESX1 volumes]# ls -ld "e24c5a25-3aa7c729"
ls: e24c5a25-3aa7c729: No such file or directory
[root@ESX1 volumes]#

[root@ESX1 volumes]# ls -ld "CLARKE E DRIVE"
lrwxr-xr-x    1 root     root           17 Jan  4 14:35 CLARKE E DRIVE -> e24c5a25-3aa7c729


[root@ESX1 volumes]# echo $?
1
[root@ESX1 volumes]# pwd
/vmfs/volumes
[root@ESX1 volumes]#



 

by: bleeuwenPosted on 2008-01-04 at 01:21:01ID: 20580744

Did you try to rescan the san?

And make sure there are no active virtual machines anymore on that volume.

 

by: wct296Posted on 2008-01-04 at 03:36:13ID: 20581178

There never was any active vm's on it (hadnt moved any yet)... And yes I did try re-scanning...

I really think its some sort of drive corruption or FAT mismatch...

I think a re-build will solve this and some of my other problems from the upgrade to version 3.5...


 

by: WeHePosted on 2008-01-04 at 05:14:29ID: 20581765

What is the output of 'rm "/vmfs/volumes/CLARKE E DRIVE"' ?
It looks like you have a link from CLARKE E DRIVE to a nonexistent file (e24...)

 

by: BestWayPosted on 2008-01-04 at 12:39:23ID: 20585885

Hi

I think the problem in on your NAS not in the VMware host

Have you delete that LUNs? This ID e24c5a25-3aa7c729 is from a LUN, is this LUN still active in the NAS?

Jail

 

by: wct296Posted on 2008-01-04 at 14:26:06ID: 20586587

I have tried re-creating the NAS, and I've tried connecting to other NAS's using the same system... My other ESX server can connect to the CLARKE NAS as well as any others I've setup..  So I'm pretty sure there is nothing wrong with the NAS, the ESX server just as a left over reference to it... which I cannot delete..

Thanks for your comment - I really appreciate it..

WeHe - will have the answer to your question soon - however I get similar issues "Operation Not Permitted" or "access denied" when I try to delete the symbolic link... That said, I did delete it once, but after a reboot it came back... dont know how..

 

by: BestWayPosted on 2008-01-04 at 14:47:11ID: 20586716

Hi

Some times deleted LUNs and files get freeze in VMware, have you tried to restart the  VMware server? And check the files again?

In VMware 2.x had a bug like this, only after a restart and a storage rescan we get the files out.

Try to restart, NAS, VMware, and then On the Virtual Center in Storage adapter rescan, then in the Storage option, try to refresh the storage

And check the files again

Jail

 

by: wct296Posted on 2008-01-04 at 15:02:29ID: 20586821

I have done two restarts... but I will try your suggestion again.. Thanks very much!

 

by: bleeuwenPosted on 2008-01-05 at 02:05:51ID: 20588604

Try to set your zoning right (so the esx can't see the san drive anymore), then rescan the san with ESX (or a reboot, which will do the same).

 

by: wct296Posted on 2008-01-05 at 02:17:04ID: 20588636

Sorry - but can you explain setting zoning?
Thanks

 

by: bleeuwenPosted on 2008-01-05 at 03:24:56ID: 20588797

Sorry you have a nas instead of a san. with san's you use zoning, don't now if a nas is also setup in this way. You also can disconnect the nas from the esx, then rescan/reboot and reattach the nas to the esx. Then rescan the esx for new volumes.

 

by: BestWayPosted on 2008-01-05 at 06:45:01ID: 20589190

Hi

If all the information that exist in your NAS is not important, then you can remove the Storage Adapters, and remove all Storage in the Virtual Center, then recreate again all, and restart both, and check the files

If you have any Storage groups in your NAS, or LUNs etc., remove all.

Jail

20120131-EE-VQP-002

3 Ways to Join

30-Day Free Trial

The Experts

98% positive feedback on 31,087 answers since March 2000. angeliii is a Microsoft Most Valuable Professional for his work with MS SQL Server & Develoment.

He has also proven his knowledge of Visual Basic Programming, PHP Scripting and Oracle Databases.

The Experts

97% positive feedback on 10,752 answers since July 2000. lrmoore has more than 18 years experience in the networking industry.

The six-time Mircosoft MVPs specialties include firewalls, virtual private networking, and network management.

Testimonials

"...and excellent source for support... Kind of like having your very own IT dept." Electriciansnet

Testimonials

"I was apprehensive at signing up at first. However... it has already made my life as an IT administrator much easier." JaCrews

Testimonials

"WOW! You guys have great, active, and knowledgeable people on here." moore50

Business Clients

Business Clients

In the Press

"If you’ve got a question... Experts Exchange can supply an answer.”

In the Press

"...an invaluable aid for both IT professionals and those who require tech support."

In the Press

"where IT professionals provide quick answers on just about any topic"

Business Account Plans

Loading Advertisement...