<div>
we are unable to remove such log file or vmdk ,because file allocation table or root dir seems to be corrupted ? and VMFS unable to locate all blocks and entries for the files to be deleted.<br />
What could be reason behind it ?
</div>


<div>
You can use RV Tools, to check for orphaned files and delete them, if not in use.<br />
<br />
<a href="http://www.robware.net/" rel="ugc">http://www.robware.net/</a>
</div>


<div>
I am able to locate orphaned files ,but when trying to delete no luck ?<br />
<br />
is it related to ghsot file and what could be done to get this fixed ?
</div>


<div>
yes, they r not in use,but yet not able to delete n files are like log files n vmdk etc not in use??
</div>


<div>
log files from what ?<br />
<br />
screenshots please
</div>


<div>
normally we should have 10 folder on data store &nbsp;as there r 10 vms created on datastore ,but here we found 12 folders n under those 2 extra folder we found few vmdk, vmk or logs which r created for any virtual machine , so when we try to delete them it seem files r locked ..unable to delete ??
</div>


<div>
you need to check if they are in use, does RV Tools suggest they are orphaned.
</div>


<div>
Strange that file/vmdk identified by rvtool n powergui as orphaned are like...<br />
<br />
&gt;few of them are of cource not in use n dirty/duplicate file to be removed <br />
&gt;but few of them are attached to running vms as second hard drives ..but why those are consider as orphaned ?<br />
<br />
can in place or scratch upgrade cause a datastore to be corrupted ?
</div>


<div>
okay, further i need to use voma to look for more detail ..<br />
<br />
Can we use voma on running infra..i mean will there be any impact or prerequisites if required ?<br />
<br />
i need to check vmfs consistency if my datsore is okay &nbsp; using voma tool.<br />
<br />
please confirm if we have other &nbsp;better way to check if lun is okay with no issues n impcat ?
</div>


<div>
Yes, you can use VOMA to check for VMFS corruption.
</div>


<div>
got your point ,Thanks<br />
<br />
Can we use voma for running infra if vms are running on datasore with prod infra ?<br />
<br />
or if any better tool to locate n fix the problem @datasore level in case there is any corruption or crash due to orphaned ghost files ??<br />
<br />
i mean m looking for healthcheck for my data store, further orphaned i can remove accordingly <br />
<br />
can we get orphaned due to vaai feature enabled in vmfs 5 ..causes svmotion delayed due to drivers or etc not compatible ?
</div>


<div>
yes ,its big question here, so need your expert advise on this, as yet there are 30+ vms running on same dataore ,but we have have orphaned disk/files for 3-4 vms on same datastore [which was recently prepared from scratch with vmfs5]<br />
<br />
Now if we look for below given logs...<br />
<i><span>FS3_ADDR_INVALID &nbsp;<br />
<br />
Below errors were seen after running VMFS check using VOMA tool by someone else.<br />
Phase 5: Checking resource reference counts.<br />
ON-DISK ERROR: FB inconsistency found: (8010,0) allocated in bitmap, but never used<br />
ON-DISK ERROR: FB inconsistency found: (8021,27) allocated in bitmap, but never used</span></i><br />
<br />
and logs like<br />
<i><span>Mapping for file abcd.vmdk (34376533 bytes in size):<br />
[ &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; 0: &nbsp; &nbsp;12582912] --&gt; [VMFS -- LVID:56bacca5-61a9bfe9-c08<wbr />3-5cf3fc5b<wbr />9b14/56bac<wbr />ca5-51e8ab<wbr />c6-4675-5c<wbr />f3fc5b9b14<wbr />/1:( 1184909754368 --&gt; &nbsp;1184922337280)]<br />
[ &nbsp; &nbsp;12582912: &nbsp; &nbsp; 8388608] --&gt; [VMFS Z- LVID:56bacca5-61a9bfe9-c08<wbr />3-5cf3fc5b<wbr />9b14/56bac<wbr />ca5-51e8ab<wbr />c6-4675-5c<wbr />f3fc5b9b14<wbr />/1:( 1184922337280 --&gt; &nbsp;1184930725888)]<br />
[ &nbsp; &nbsp;20971520: &nbsp; &nbsp;15728640] --&gt; [VMFS -- LVID:56bacca5-61a9bfe9-c08<wbr />3-5cf3fc5b<wbr />9b14/56bac<wbr />ca5-51e8ab<wbr />c6-4675-5c<wbr />f3fc5b9b14<wbr />/1:( 1184930725888 --&gt; &nbsp;1184946454528)]</span></i><br />
<br />
so above logs indicate its all due to corruption @datasore level ?<br />
If Yes -How can we resolve/fix it with minimal efforts and avoid such issue in future ?<br />
If No -How to locate exact issue behind it....<br />
&gt;any tool we can use in running infra &nbsp;?<br />
&gt;&nbsp;due to VAAI feature enabled on newly formated lun<br />
&gt;due to in place upgrade<br />
&gt;due to SAN firmware upgrade<br />
&gt;due to patching &nbsp;of vc/esxi and vms pending since long time including vmtool alos
</div>


<div>
if you cannot answer that question yourselves.<br />
<br />
<b>MOVE THE VMs OFF IMMEDIATELY.</b>
</div>


<div>
we did vmfs upgrade almost &nbsp;6 month before and it was &nbsp;90% we did in place upgrade but 10% we did from scratch adding new luns<br />
<br />
earlier we had few orphaned disk..but now days [in june-july]getting more and more <br />
<br />
how can we control on creation of such orphaned n ghost files
</div>


<div>
i believe its not lun/datasore corruption , can we use any better tool to look n fix more on this within running infra
</div>


<div>
Thanks Andrew<br />
<br />
yes will check for that, i am not directly involved team in this issue,else i could have done more,but will try for sure<br />
<br />
Many Thanks for all your support on this.<br />
<br />
let me try to look more into this
</div>


<div>
can we remove this question ?
</div>


<div>
Why, select an Answer and Award Points.
</div>


<div>
<div class="content wysiwyg-content">
Related to vSphere 5.5[upgraded from 4.1] ..recently we did in place vmfs upgrade &amp;&nbsp;there was firmware upgrade done @san level as well<br />
now from last few days...we are getting &nbsp;few errors<br />
<br />
&quot;ON-DISK ERROR: FB inconsistency found: (8010,0) allocated in bitmap, but never used&quot;<br />
<br />
and due to this we are unable to delete few orphaned vmdk from data store, we can see same folder there on multiple lun.<br />
<br />
Can we remove such file using any valid method wich can be action with no interruption to running services ?<br />
also would like to confirm for below given quires...<br />
<br />
&gt;What is ghost file and corrupted log &nbsp;files in vmfs data store? and what could be reason behind it ?<br />
&gt;Can we avoid such file creation ?<br />
&gt;Will that ghost or corrupted file cause datastore corruption or any other major impact ?<br />
&gt;is Orphaned vmdk related &nbsp;to such ghost or corrupted files ?<br />
&gt;How can we remove such files easily with no service interruption or how to fix this issue on permanent basis ?<br />
&gt;what could be reason behind such ghost or corrupted files ? is in place upgrade reason behind it ?<br />
In Place upgrade is not recommended by VMware ?
</div>
</div>


Related to vSphere 5.5[upgraded from 4.1] ..recently we did in place vmfs upgrade & there was firmware upgrade done @san level as well
now from last few days...we are getting  few errors

"ON-DISK ERROR: FB inconsistency found: (8010,0) allocated in bitmap, but never used"

and due to this we are unable to delete few orphaned vmdk from data store, we can see same folder there on multiple lun.

Can we remove such file using any valid method wich can be action with no interruption to running services ?
also would like to confirm for below given quires...

>What is ghost file and corrupted log  files in vmfs data store? and what could be reason behind it ?
>Can we avoid such file creation ?
>Will that ghost or corrupted file cause datastore corruption or any other major impact ?
>is Orphaned vmdk related  to such ghost or corrupted files ?
>How can we remove such files easily with no service interruption or how to fix this issue on permanent basis ?
>what could be reason behind such ghost or corrupted files ? is in place upgrade reason behind it ?
In Place upgrade is not recommended by VMware ?

To locate and remove Ghost &amp; corrupted log files files in VMFS Datstore

VMware, a software company founded in 1998,  was one of the first commercially successful companies to offer x86 virtualization. The storage company EMC purchased VMware in 1994. Dell Technologies acquired EMC in 2016. VMware’s parent company is now Dell Technologies.

VMware has many software products that run on desktops, Microsoft Windows, Linux, and macOS, which allows the virtualizing of the x86 architecture.  Its enterprise software hypervisor for servers, VMware vSphere Hypervisor (ESXi), is a bare-metal hypervisor that runs directly on the server hardware and does not require an additional underlying operating system.

VMware

Computer data storage, often called storage or memory, is a technology consisting of computer components and recording media used to retain digital data. In addition to local storage devices like CD and DVD readers, hard drives and flash drives, solid state drives can hold enormous amounts of data in a very small device. Cloud services and other new forms of remote storage also add to the capacity of devices and their ability to access more data without building additional data storage into a device.

Storage

Virtualization is the act of creating a virtual (rather than actual) version of something, including (but not limited to) a virtual computer hardware platform, operating system (OS), storage device, or computer network resources. Virtualization is usually the creation of a system that executes separate from the underlying hardware resources, or the creation of an entire desktop for systems located elsewhere, similar to thin clients.

To locate and remove Ghost &amp; corrupted log files files in VMFS Datstore

To locate and remove Ghost & corrupted log files files in VMFS Datstore