It is an honor to be featured in Gartner 2019 Magic Quadrant for Datacenter Backup and Recovery Solutions. Gartner’s MQ sets a high standard and earning a place on their grid is a great affirmation that Acronis is delivering on our mission to protect all data, apps, and systems.
Xen Server Storage Issue
Hi,
I've just upgraded a XenServer host from 5.5.0 to 5.6.0 SP2, however on restart, I get the following message in XenCenter:
Failed to attached storage on server start
This refers to a 2 TB lun on a NetApp filer via Fibre Channel. Other hosts in the pool (not yet upgraded) can see the lun ok, yet it's status is broken as the host reports to be "unplugged" from the storage.
This is 1 of 8 luns accessible - other FC luns are ok.
No idea what to do now, so any help would be greatly appreciated!
Thanks.
I've just upgraded a XenServer host from 5.5.0 to 5.6.0 SP2, however on restart, I get the following message in XenCenter:
Failed to attached storage on server start
This refers to a 2 TB lun on a NetApp filer via Fibre Channel. Other hosts in the pool (not yet upgraded) can see the lun ok, yet it's status is broken as the host reports to be "unplugged" from the storage.
This is 1 of 8 luns accessible - other FC luns are ok.
No idea what to do now, so any help would be greatly appreciated!
Thanks.
Do more with
EXPERT OFFICE® is a registered trademark of EXPERTS EXCHANGE®
I'm sure you tried right clicking the storage and elect "repair". What did that do?
Have you checked the log files on your upgraded host?
You'll find the important ones here:
/var/log/messages
/var/log/xensource.log
You can use a tool like WinSCP to connect to the Dom0 and look at the content.
Check for any errors, especially around boot time, and post back.
Have you checked the log files on your upgraded host?
You'll find the important ones here:
/var/log/messages
/var/log/xensource.log
You can use a tool like WinSCP to connect to the Dom0 and look at the content.
Check for any errors, especially around boot time, and post back.
Yes, I tried repair - the error mentioned PBDs
/var/log/messages mentioned:
very big device
xensource.log extract attached, where it tries to attach the SR. PBD is mentioned in here also.
Thanks for the quick reply by the way.
xensource-extract.txt
/var/log/messages mentioned:
very big device
xensource.log extract attached, where it tries to attach the SR. PBD is mentioned in here also.
Thanks for the quick reply by the way.
xensource-extract.txt
Try to look up the messages in SMLog, right around the time the xensource log reported the error. SMlog is located in the same folder /var/log/
I forgot that storage operations are mostly logged in SMLog.
I forgot that storage operations are mostly logged in SMLog.
Extract below, but the full file is attached:
[14279] 2012-12-22 22:50:51.246524 SUCCESS
[14279] 2012-12-22 22:50:51.246669 Synching Metadata volume
[14279] 2012-12-22 22:50:51.246766 ['/usr/sbin/lvchange', '-ay', '/dev/VG_XenStorage-25b74c46-b305-c164-1070-6bf5abb49c5b/MGT']
[14279] 2012-12-22 22:50:51.341808 SUCCESS
[14279] 2012-12-22 22:50:51.342376 lock: released /var/lock/sm/25b74c46-b305-c164-1070-6bf5abb49c5b/sr
[14279] 2012-12-22 22:50:51.351858 ***** sr_attach: EXCEPTION exceptions.AssertionError,
File "/opt/xensource/sm/SRCommand.py", line 94, in run
return self._run_locked(sr)
File "/opt/xensource/sm/SRCommand.py", line 131, in _run_locked
return self._run(sr, target)
File "/opt/xensource/sm/SRCommand.py", line 223, in _run
return sr.attach(self.params['sr_uuid'])
File "/opt/xensource/sm/LVMoHBASR", line 124, in attach
LVHDSR.LVHDSR.attach(self, sr_uuid)
File "/opt/xensource/sm/LVHDSR.py", line 309, in attach
self._checkMetadataVolume(self.sm_config)
File "/opt/xensource/sm/LVHDSR.py", line 201, in _checkMetadataVolume
self._synchMetaData(map)
File "/opt/xensource/sm/LVHDSR.py", line 211, in _synchMetaData
xml = metadata.retrieveXMLfromFile(self.mdpath)
File "/opt/xensource/sm/metadata.py", line 103, in retrieveXMLfromFile
_testHdr(hdr)
File "/opt/xensource/sm/metadata.py", line 41, in _testHdr
assert(hdr[2] <= MD_MAJOR)
[14279] 2012-12-22 22:50:51.357473 lock: closed /var/lock/sm/25b74c46-b305-c164-1070-6bf5abb49c5b/sr
[14451] 2012-12-22 22:50:51.373612 LVMCache created for VG_XenStorage-76c86946-ce7a-6690-ba60-2692a0b5e303
<14451> 2012-12-22 22:50:51.514441 Host f8b94db1-26eb-4bcb-96ac-293075c52204 not enabled
<14451> 2012-12-22 22:50:51.514677 Pool is not ready, exiting
<14451> 2012-12-22 22:50:51.514750 SR 76c8 ('NetApp 2.5TB 1st floor') (0 VDIs in 0 VHD trees): no changes
[14491] 2012-12-22 22:50:51.699425 LVMCache created for VG_XenStorage-c06d1738-f71a-4b4a-d7df-3685db30e2a8
[14491] 2012-12-22 22:50:51.708770 ['/usr/sbin/vgs', 'VG_XenStorage-c06d1738-f71a-4b4a-d7df-3685db30e2a8']
[14491] 2012-12-22 22:50:51.841909 SUCCESS
[14491] 2012-12-22 22:50:51.842105 lock: acquired /var/lock/sm/c06d1738-f71a-4b4a-d7df-3685db30e2a8/sr
[14491] 2012-12-22 22:50:51.842172 LVMCache: will initialize now
[14491] 2012-12-22 22:50:51.842218 LVMCache: refreshing
[14491] 2012-12-22 22:50:51.842288 ['/usr/sbin/lvs', '--noheadings', '--units', 'b', '-o', '+lv_tags', '/dev/VG_XenStorage-c06d1738-f71a-4b4a-d7df-3685db30e2a8']
[14491] 2012-12-22 22:50:51.921928 SUCCESS
[14491] 2012-12-22 22:50:51.922361 lock: released /var/lock/sm/c06d1738-f71a-4b4a-d7df-3685db30e2a8/sr
[14491] 2012-12-22 22:50:51.922611 lock: closed /var/lock/sm/c06d1738-f71a-4b4a-d7df-3685db30e2a8/sr
[14491] 2012-12-22 22:50:51.922684 LVMCache created for VG_XenStorage-c06d1738-f71a-4b4a-d7df-3685db30e2a8
[14491] 2012-12-22 22:50:51.932351 ['/usr/sbin/vgs', 'VG_XenStorage-c06d1738-f71a-4b4a-d7df-3685db30e2a8']
[14491] 2012-12-22 22:50:52.008071 SUCCESS
[14491] 2012-12-22 22:50:52.008258 lock: acquired /var/lock/sm/c06d1738-f71a-4b4a-d7df-3685db30e2a8/sr
[14491] 2012-12-22 22:50:52.008326 LVMCache: will initialize now
[14491] 2012-12-22 22:50:52.008371 LVMCache: refreshing
[14491] 2012-12-22 22:50:52.008442 ['/usr/sbin/lvs', '--noheadings', '--units', 'b', '-o', '+lv_tags', '/dev/VG_XenStorage-c06d1738-f71a-4b4a-d7df-3685db30e2a8']
[14491] 2012-12-22 22:50:52.147745 SUCCESS
[14491] 2012-12-22 22:50:52.148196 lock: released /var/lock/sm/c06d1738-f71a-4b4a-d7df-3685db30e2a8/sr
[14491] 2012-12-22 22:50:52.148327 lock: acquired /var/lock/sm/c06d1738-f71a-4b4a-d7df-3685db30e2a8/sr
[14491] 2012-12-22 22:50:52.148461 sr_attach {'sr_uuid': 'c06d1738-f71a-4b4a-d7df-3685db30e2a8', 'subtask_of': 'OpaqueRef:94fadc0d-5104-cea7-8f83-46c3f9ab1b5c', 'args': [], 'host_ref': 'OpaqueRef:a69330e3-757f-8ff5-6099-d397d54caf85', 'session_ref': 'OpaqueRef:e1791520-24a6-f91e-3670-373e4153111f', 'device_config': {'device': '/dev/disk/mpInuse/360a98000572d5038636f58795a6c3753', 'SCSIid': '360a98000572d5038636f58795a6c3753', 'SRmaster': 'true'}, 'command': 'sr_attach', 'sr_ref': 'OpaqueRef:ac0b5d46-1ac7-7051-83c0-d1e720b5779f'}
[14491] 2012-12-22 22:50:52.148766 lock: acquired /var/lock/sm/c06d1738-f71a-4b4a-d7df-3685db30e2a8/running
[14491] 2012-12-22 22:50:52.148830 lock: acquired /var/lock/sm/c06d1738-f71a-4b4a-d7df-3685db30e2a8/sr
[14491] 2012-12-22 22:50:52.158463 RESET for SR c06d1738-f71a-4b4a-d7df-3685db30e2a8 (master: True)
[14491] 2012-12-22 22:50:52.195078 lock: released /var/lock/sm/c06d1738-f71a-4b4a-d7df-3685db30e2a8/sr
[14491] 2012-12-22 22:50:52.195159 lock: released /var/lock/sm/c06d1738-f71a-4b4a-d7df-3685db30e2a8/running
[14491] 2012-12-22 22:50:52.195211 lock: closed /var/lock/sm/c06d1738-f71a-4b4a-d7df-3685db30e2a8/sr
[14491] 2012-12-22 22:50:52.195261 lock: closed /var/lock/sm/c06d1738-f71a-4b4a-d7df-3685db30e2a8/running
[14491] 2012-12-22 22:50:52.202113 set_dirty 'OpaqueRef:ac0b5d46-1ac7-7051-83c0-d1e720b5779f' succeeded
[14491] 2012-12-22 22:50:52.202177 MPATH: multipath activate called
[14491] 2012-12-22 22:50:52.202227 ['/opt/xensource/sm/mpathHBA', 'enable']
[14491] 2012-12-22 22:50:52.221720 SUCCESS
From /var/log/messages
Dec 22 22:50:51 bcxenhostb3 fe: 14283 (/opt/xensource/sm/LVMoHBA
Dec 22 22:50:51 bcxenhostb3 fe: 14279 (/opt/xensource/sm/LVMoHBA
Dec 22 22:50:51 bcxenhostb3 xapi: [error|bcxenhostb3|0 thread_zero|creating storage D:aeefcbfe0dce|xapi] Could not plug in pbd '8f3869e9-38c6-e041-da4e-5
Dec 22 22:50:52 bcxenhostb3 multipathd: sdam: add path (operator)
Dec 22 22:50:52 bcxenhostb3 multipathd: sdam: spurious uevent, path already in pathvec
Dec 22 22:50:51 bcxenhostb3 fe: 14283 (/opt/xensource/sm/LVMoHBA
Dec 22 22:50:51 bcxenhostb3 fe: 14279 (/opt/xensource/sm/LVMoHBA
Dec 22 22:50:51 bcxenhostb3 xapi: [error|bcxenhostb3|0 thread_zero|creating storage D:aeefcbfe0dce|xapi] Could not plug in pbd '8f3869e9-38c6-e041-da4e-5
Dec 22 22:50:52 bcxenhostb3 multipathd: sdam: add path (operator)
Dec 22 22:50:52 bcxenhostb3 multipathd: sdam: spurious uevent, path already in pathvec
Unfortunately, there is not much info in the file that I could help you with. I would highly suggest you open an account with the official citrix forum and post your file outputs there if not already done so.
There are Citrix employees on these forums that might be able to help you further.
Also, I assume you disabled HA on your Server before doing the upgrade? How about Multipathing? If Multipathing is still active on your xenservers, try disabling that on all of them and then try to attach the storage again.
There are Citrix employees on these forums that might be able to help you further.
Also, I assume you disabled HA on your Server before doing the upgrade? How about Multipathing? If Multipathing is still active on your xenservers, try disabling that on all of them and then try to attach the storage again.
Thanks, I have posted all of this information over there to see if anything comes out of it,
HA isn't licensed as it is the free version of xenserver, unsure on multipathing? This only affects the one I upgraded to 5.6 from 5.5, so I still have the option to rollback, and try again I suppose.
HA isn't licensed as it is the free version of xenserver, unsure on multipathing? This only affects the one I upgraded to 5.6 from 5.5, so I still have the option to rollback, and try again I suppose.
Yes, you can start over, especially if you have run a "Xe pool-dump-database" and saved the resulting file somewhere offsite.
Then you can simply reinstall the old version and run the restore of the PoolDB on that host.
Then you can simply reinstall the old version and run the restore of the PoolDB on that host.
Do i have to restore the DB? I have reverted the pool master back to 5.5 but still have the SR issue
Premium Content
You need an Expert Office subscription to comment.Start Free Trial