HP-UX Replace a faulty mirror disk

Hi I have an HPUX rp3440 server with 2 x Hot Pluggable hard disks that are mirrored , (everything is mirrored on both) and they are both boot disks.
HP-UX version B.11.11 9000/800/rp3440
B2491BA          B.11.11           MirrorDisk/UX
LVM disk                   Device file        HW Addr      size   vol. grp
    HP 73.4GST373453LC         /dev/dsk/c2t0d0    0/1/1/0.0.0  70007  /dev/vg00
    HP 73.4GST373453LC         /dev/dsk/c2t1d0    0/1/1/0.1.0  70007  /dev/vg00

# lvlnboot -v
Boot Definitions for Volume Group /dev/vg00:
Physical Volumes belonging in Root Volume Group:
        /dev/dsk/c2t0d0 (0/1/1/0.0.0) -- Boot Disk
        /dev/dsk/c2t1d0 (0/1/1/0.1.0) -- Boot Disk

I need to replace one of the disks (c2t0d0) and I want to know what would be the best option to take.
I have read the following articles but still not 100% sure of which method  i should use.
•      http://wiki-ux.info/wiki/How_to_replace_a_LVM_mirror_boot_disk  (section 6.2)  
•      When Good Disks Go Bad - Document

# lvlnboot -v
Boot Definitions for Volume Group /dev/vg00:
Physical Volumes belonging in Root Volume Group:
        /dev/dsk/c2t0d0 (0/1/1/0.0.0) -- Boot Disk
        /dev/dsk/c2t1d0 (0/1/1/0.1.0) -- Boot Disk

pvdisplay -v is showing PV status as unavailable
vgdisplay -v is showing LV Status as available/stale for some but not all of the volumes (others are available/syncd)
lvdisplay - v is showing some of the logical extents as Stale the rest are Current.
if it I try an lvsync I get the following error

lvsync: Couldn't re-synchronize stale partitions of the logical volume:
I/O error
lvsync: Couldn't resynchronize logical volume "/dev/vg00/lvol1".
Also tried running this dd if=/dev/rdsk/c2t0d0 of=/dev/null bs=1024k but it hung (also cant kill it)

Are these my options ? : -
•      Shut down the server and replace the faulty disk. Does the replacement disk have to be clean and what commands do i need to run before shutting down or bringing the server back up.
•      Install these LVM OLR patches on the server (HP-UX 11i v1) which look to make it easier to do disk swapping. PHCO_35955, PHKL_35970.  But patches look like i need to reboot server anyway
•      Use the pvchange and vgcfgrestore commands as per the wiki url link  above (section 6.2) . Have had some feedback that these commands can hang so not advisable to use.

I think i would prefer to use the first option but would like some advice please.
I was advised to repost this question as last one was neglected.

Who is Participating?

[Product update] Infrastructure Analysis Tool is now available with Business Accounts.Learn More

I wear a lot of hats...

"The solutions and answers provided on Experts Exchange have been extremely helpful to me over the last few years. I wear a lot of hats - Developer, Database Administrator, Help Desk, etc., so I know a lot of things but not a lot about one thing. Experts Exchange gives me answers from people who do know a lot about one thing, in a easy to use platform." -Todd S.

Hot-pluggable should not be replaced on-line the whole purpose of hot-plugable is to allow the system  to continue to run. I would avoid trying to remedy this issue with the system powered off.
Do you have space to add a new drive or do you have one that is a hot spare?
You would need to partition the new/replacement and add each respective partition to be a member of a mirror.
Break each mirror, eject/remove the failed drive. Replace the drive, clone partition layout on the new drive.
rebuild the mirrors.
(Make sure you have a good recent backup just in case)

Experts Exchange Solution brought to you by

Your issues matter to us.

Facing a tech roadblock? Get the help and guidance you need from experienced professionals who care. Ask your question anytime, anywhere, with no hassle.

Start your 7-day free trial
stopmAuthor Commented:
Thanks Arnold,  in that case would i need to vgreduce, lvreduce commands which I thought as per above  url (extract below) can have issues in that the commands can hang? If that is the scenario which I expect it will be as other commands such as lvlnboot hang when I try and run them.

"Is has been a common practice, before the appearance of the OLR feature in 11i v1, to reduce the logical volumes from the failed disk and reduce the physical volume from the volume group to assure that no I/O requests are going to the failed disk.

This procedure has several drawbacks:
1. The disk should be accessible and the PVRA/VGRA should be working.
2. This approach usually produces more harm than benefits. It’s not uncommon that the system hangs during these tasks.
3. The procedure is prune to errors. You need to lvreduce the logical volume(s) mirror(s), vgreduce the logical volume group, replace the physical volume, pvcreate the physical volume, vgextend the volume group and lvextend the logical volume(s) mirror(s). That is particulary complex on Integrity systems when disk should be partitioned with the idisk command. "
Usually, you are not changing sizes of volumes, you are breaking mirrors, removing failed components.
If you have space in the server to add the replacement drive, that might be better.
is /dev/dsk/c2t0d0 reflected as dead can you access it on a low level to verify its functionality?
Unfortunately, hp-ux is not a system I am familiar with to the extent necessary.

can you list the partitions on the drive, or does that process also hang? do you have messages/syslog even reflecting issues with the drive?

In the document it does not cover/list the type of raid that scheme was altering.
it has a guide to run a command sasmgr to confirm that the disk is reflected as off as well as having a replacement added prior to removing the failed one.
stopmAuthor Commented:
Hi Arnold sorry for delay in replying I have been off sick.

sorry a little confused here. I thought I had to use lvreduce and vgreduce to break the mirror?

I don't seem to have the sasmgr command available on this server.

I can list the partitions ok.

there is space to install an additional disk

I am not getting regular error messages in syslogs just the occasional one 'vmunix: SCSI: Async read and write error's'

Are you able to assist or would an HPUX specialist be able to help

It's more than this solution.Get answers and train to solve all your tech problems - anytime, anywhere.Try it for free Edge Out The Competitionfor your dream job with proven skills and certifications.Get started today Stand Outas the employee with proven skills.Start learning today for free Move Your Career Forwardwith certification training in the latest technologies.Start your trial today
Unix OS

From novice to tech pro — start learning today.