AIX - lsvg showing open/stale

root@server[/]# lsvg -p rootvg
rootvg:
PV_NAME           PV STATE          TOTAL PPs   FREE PPs    FREE DISTRIBUTION
hdisk0            missing           319         232         63..40..01..64..64
hdisk4            active            319         232         63..40..01..64..64

Do I have to unmirror the disks first, before I run varyonvg rootvg?

Why didn't you read my comment above up to its end?

Run "cfgmgr" and check if the disks are active again.
Then run "varyonvg rootvg"
Wait a little while and recheck.

No reason to unmirror anything (or to get into panic - it's AIX, after all!)

I did listen to you, I just forgot to post that I ran cfgmgr. Apologies.

However, after running cfgmgr..it still shows missing.

root@server[/]# cfgmgr
root@server[/]# lsvg -p rootvg
rootvg:
PV_NAME           PV STATE          TOTAL PPs   FREE PPs    FREE DISTRIBUTION
hdisk0            missing           319         232         63..40..01..64..64
hdisk4            active            319         232         63..40..01..64..64
root@server[/]#

It is a SAN disk.

root@server[/]# lspath -l hdisk0
Enabled hdisk0 vscsi0
Enabled hdisk0 vscsi1
root@server[/]# lsdev -Cl hdisk0
hdisk0 Available  Virtual SCSI Disk Drive
root@server[/]# errpt -a |more |grep hdisk0
root@server[/]#

It's a VIO disk, so the underlying issue must be searched at the VIO servers.

Did you check there? It looks quite as if both VIO servers cannot access the backing device,
although the virtual SCSI link is functional nonetheless.

I saw such things with disks whose reserve policy was not "no_reserve".

I found out that we should only have one rootvg disk.

I tried to unmirror it but its not working. Any help?

root@server[/]# unmirrorvg rootvg
0516-1246 rmlvcopy: If hd5 is the boot logical volume, please run 'chpv -c <diskname>'
        as root user to clear the boot record and avoid a potential boot
        off an old boot image that may reside on the disk from which this
        logical volume is moved/removed.
0516-1155 lreducelv: Last good copy of a partition cannot reside on a missing disk.
Try again after reactivating the disk using chpv and varyonvg.
0516-922 rmlvcopy: Unable to remove logical partition copies from
        logical volume hd6.
0516-1135 unmirrorvg: The unmirror of the volume group failed.
        The volume group is still partially or fully mirrored.

Then I ran:
root@server[/]# chpv -c hdisk0
root@server[/]# unmirrorvg rootvg
0516-1155 lreducelv: Last good copy of a partition cannot reside on a missing disk.
Try again after reactivating the disk using chpv and varyonvg.
0516-922 rmlvcopy: Unable to remove logical partition copies from
        logical volume hd6.
0516-1135 unmirrorvg: The unmirror of the volume group failed.
        The volume group is still partially or fully mirrored.

hd6 is paging. What is needed to complete the unmirroring?

The message was trying to tell you that you should run

chpv -v a hdisk0

to make it available, but this will not work, because the disk is not there.

It looks as if you removed a disk from rootvg which contained a not cleanly mirrored paging space!

Please post the output of

lslv -l hd6
lslv -m hd6

Let's see if we can run a manual rmlvcopy.

I think you already used the disk for other purposes, so you cannot reattach it just for cleaning up hd6?

I when ahead chpv on hdisk0. Than manually ran rmlvcopy on all the LV Names. That worked! I was able to free up hdisk0 and rmdev it. Now everything is open/syncd.

However, I'm getting this error. Do you have an other ideas of what I can do?

I set the bootlist correctly and ran bosboot on hdisk4. As you can see at the bottom.

0503-409 installp:  bosboot verification starting...
0503-497 installp:  An error occurred during bosboot verification processing.

install_all_updates: Checking for recommended maintenance level 6100-07.
install_all_updates: Executing /usr/bin/oslevel -rf, Result = 6100-06
install_all_updates: ATTENTION, the system recommended maintenance level
does not correspond to the highest level known to install_all_updates.
For more details, execute /usr/bin/oslevel -rl 6100-07.

install_all_updates: Log file is /var/adm/ras/install_all_updates.log
install_all_updates: Result = FAILURE
root@server[/]# bootlist -m normal -o
hdisk4 blv=hd5 pathid=0
hdisk4 blv=hd5 pathid=1
root@server[/]# bosboot -ad hdisk4

bosboot: Boot image is 47118 512 byte blocks.
root@server[/]#

I hope you ran reducevg before rmdev!

Yes, manually running rmlvcopy instead of unmirrorvg would have been my suggestion (see above)!

But I'm very sure that running chpv (which flag did you use?) didn't have any effect at all.

That's quite a different kind of problem.

You're missing several filesets in order to reach the level 6100-07

"oslevel -rl 6100-07" will show you which ones have still to be installed.

Run "installp -C" first to cleanup, then run "lppchk -v" Any messages?

This was the specific error at the top:
0503-409 installp:  bosboot verification starting...
0503-497 installp:  An error occurred during bosboot verification processing.

I ran this and it fixed my error.

#ln -f /dev/rhdisk0 /dev/ipldevice

Very misleading.

OK, this "ipldevice" thing is very common after unmirroring a rootvg.