Sharif Odinaev
asked on
AIX not load error 555
i have AIX 6.1 operation system on IBM Power 740. There are configured Mirroing rootvg. the aix not loading. I get error 555. PLS help.
we have failed disk: hdisk0 is failed.
I load from Installation Disk.
Access a Root Volume Group
Type the number for a volume group to display the logical volume information
and press Enter.
1) Volume Group 00f6aaf900004c00000001304f 99eb37 contains these disks:
hdisk1 286102 01-08-00
Choice: 1
Volume Group Information
-------------------------- ---------- ---------- ---------- ---------- ---------- --
Volume Group ID 00f6aaf900004c00000001304f 99eb37 includes the following
logical volumes:
hd5 hd6 hd8 hd4 hd2 hd9var
hd3 hd1 hd10opt hd11admin lg_dumplv livedump
hd12 hd11 fwdump
-------------------------- ---------- ---------- ---------- ---------- ---------- --
When i choos "Access this Volume Group and start a shell" i get following :
Importing Volume Group...
PV Status: hdisk1 00f6aaf94f99ea43 PVACTIVE
00f6aaf94f99ea03 NONAME
varyonvg: Volume group rootvg is varied on.
0516-510 synclvodm: Physical volume not found for physical volume
identifier 00f6aaf94f99ea030000000000 000000.
0516-548 synclvodm: Partially successful with updating volume
group rootvg.
0516-1281 synclvodm: WARNING, lv control block of hd5
has been over written.
0516-1281 synclvodm: WARNING, lv control block of hd8
has been over written.
0516-1281 synclvodm: WARNING, lv control block of hd4
has been over written.
0516-1281 synclvodm: WARNING, lv control block of hd2
has been over written.
0516-1281 synclvodm: WARNING, lv control block of hd9var
has been over written.
0516-1281 synclvodm: WARNING, lv control block of hd3
has been over written.
0516-1281 synclvodm: WARNING, lv control block of hd1
has been over written.
0516-1281 synclvodm: WARNING, lv control block of hd10opt
has been over written.
0516-1281 synclvodm: WARNING, lv control block of hd11admin
has been over written.
0516-1281 synclvodm: WARNING, lv control block of livedump
has been over written.
0516-1281 synclvodm: WARNING, lv control block of hd12
has been over written.
0516-1281 synclvodm: WARNING, lv control block of hd11
has been over written.
0516-1281 synclvodm: WARNING, lv control block of fwdump
has been over written.
0516-622 synclvodm: Warning, cannot write lv control block data.
0516-622 synclvodm: Warning, cannot write lv control block data.
0516-622 synclvodm: Warning, cannot write lv control block data.
0516-622 synclvodm: Warning, cannot write lv control block data.
0516-622 synclvodm: Warning, cannot write lv control block data.
0516-622 synclvodm: Warning, cannot write lv control block data.
0516-622 synclvodm: Warning, cannot write lv control block data.
0516-622 synclvodm: Warning, cannot write lv control block data.
0516-622 synclvodm: Warning, cannot write lv control block data.
0516-622 synclvodm: Warning, cannot write lv control block data.
0516-622 synclvodm: Warning, cannot write lv control block data.
0516-622 synclvodm: Warning, cannot write lv control block data.
0516-622 synclvodm: Warning, cannot write lv control block data.
rootvg
ln: /dev/r: No such file or directory
getlvodm: option requires an argument -- l
Usage: getlvodm [-a LVdescript] [-B LVdescript] [-b LVid] [-c LVid]
[-C] [-d VGdescript] [-e LVid] [-F] [-g PVid] [-h] [-j PVdescript]
[-k] [-L VGdescript] [-l LVdescript] [-m LVid] [-p PVdescript]
[-r LVid] [-s VGdescript] [-t VGid] [-u VGdescript] [-v VGdescript]
[-w VGid] [-y LVid] [-G LVdescript]
0516-104 lquerylv: Incorrect LV identifier. The format must be
VG_identifier.LV_minor_num ber
Could not find "/" and/or "/usr" filesystems.
Exiting to shell.
#
OUTPUT of lsvg command:
lsvg
/usr/bin/ksh: lsvg: not found
OUTPUT of bootlist command:
# bootlist -m normal -o
Killed
Please help
we have failed disk: hdisk0 is failed.
I load from Installation Disk.
Access a Root Volume Group
Type the number for a volume group to display the logical volume information
and press Enter.
1) Volume Group 00f6aaf900004c00000001304f
hdisk1 286102 01-08-00
Choice: 1
Volume Group Information
--------------------------
Volume Group ID 00f6aaf900004c00000001304f
logical volumes:
hd5 hd6 hd8 hd4 hd2 hd9var
hd3 hd1 hd10opt hd11admin lg_dumplv livedump
hd12 hd11 fwdump
--------------------------
When i choos "Access this Volume Group and start a shell" i get following :
Importing Volume Group...
PV Status: hdisk1 00f6aaf94f99ea43 PVACTIVE
00f6aaf94f99ea03 NONAME
varyonvg: Volume group rootvg is varied on.
0516-510 synclvodm: Physical volume not found for physical volume
identifier 00f6aaf94f99ea030000000000
0516-548 synclvodm: Partially successful with updating volume
group rootvg.
0516-1281 synclvodm: WARNING, lv control block of hd5
has been over written.
0516-1281 synclvodm: WARNING, lv control block of hd8
has been over written.
0516-1281 synclvodm: WARNING, lv control block of hd4
has been over written.
0516-1281 synclvodm: WARNING, lv control block of hd2
has been over written.
0516-1281 synclvodm: WARNING, lv control block of hd9var
has been over written.
0516-1281 synclvodm: WARNING, lv control block of hd3
has been over written.
0516-1281 synclvodm: WARNING, lv control block of hd1
has been over written.
0516-1281 synclvodm: WARNING, lv control block of hd10opt
has been over written.
0516-1281 synclvodm: WARNING, lv control block of hd11admin
has been over written.
0516-1281 synclvodm: WARNING, lv control block of livedump
has been over written.
0516-1281 synclvodm: WARNING, lv control block of hd12
has been over written.
0516-1281 synclvodm: WARNING, lv control block of hd11
has been over written.
0516-1281 synclvodm: WARNING, lv control block of fwdump
has been over written.
0516-622 synclvodm: Warning, cannot write lv control block data.
0516-622 synclvodm: Warning, cannot write lv control block data.
0516-622 synclvodm: Warning, cannot write lv control block data.
0516-622 synclvodm: Warning, cannot write lv control block data.
0516-622 synclvodm: Warning, cannot write lv control block data.
0516-622 synclvodm: Warning, cannot write lv control block data.
0516-622 synclvodm: Warning, cannot write lv control block data.
0516-622 synclvodm: Warning, cannot write lv control block data.
0516-622 synclvodm: Warning, cannot write lv control block data.
0516-622 synclvodm: Warning, cannot write lv control block data.
0516-622 synclvodm: Warning, cannot write lv control block data.
0516-622 synclvodm: Warning, cannot write lv control block data.
0516-622 synclvodm: Warning, cannot write lv control block data.
rootvg
ln: /dev/r: No such file or directory
getlvodm: option requires an argument -- l
Usage: getlvodm [-a LVdescript] [-B LVdescript] [-b LVid] [-c LVid]
[-C] [-d VGdescript] [-e LVid] [-F] [-g PVid] [-h] [-j PVdescript]
[-k] [-L VGdescript] [-l LVdescript] [-m LVid] [-p PVdescript]
[-r LVid] [-s VGdescript] [-t VGid] [-u VGdescript] [-v VGdescript]
[-w VGid] [-y LVid] [-G LVdescript]
0516-104 lquerylv: Incorrect LV identifier. The format must be
VG_identifier.LV_minor_num
Could not find "/" and/or "/usr" filesystems.
Exiting to shell.
#
OUTPUT of lsvg command:
lsvg
/usr/bin/ksh: lsvg: not found
OUTPUT of bootlist command:
# bootlist -m normal -o
Killed
Please help
ASKER
Good day Dear dfke. Thank you for reply.
i start with bootable media in service mode and choose:
2) Access this Volume Group and start a shell before mounting filesystems
after i get the following:
Importing Volume Group...
PV Status: hdisk1 00f6aaf94f99ea43 PVACTIVE
00f6aaf94f99ea03 NONAME
varyonvg: Volume group rootvg is varied on.
0516-510 synclvodm: Physical volume not found for physical volume
identifier 00f6aaf94f99ea030000000000 000000.
0516-548 synclvodm: Partially successful with updating volume
group rootvg.
0516-1281 synclvodm: WARNING, lv control block of hd5
has been over written.
0516-1281 synclvodm: WARNING, lv control block of hd8
has been over written.
0516-1281 synclvodm: WARNING, lv control block of hd4
has been over written.
0516-1281 synclvodm: WARNING, lv control block of hd2
has been over written.
0516-1281 synclvodm: WARNING, lv control block of hd9var
has been over written.
0516-1281 synclvodm: WARNING, lv control block of hd3
has been over written.
0516-1281 synclvodm: WARNING, lv control block of hd1
has been over written.
0516-1281 synclvodm: WARNING, lv control block of hd10opt
has been over written.
0516-1281 synclvodm: WARNING, lv control block of hd11admin
has been over written.
0516-1281 synclvodm: WARNING, lv control block of livedump
has been over written.
0516-1281 synclvodm: WARNING, lv control block of hd12
has been over written.
0516-1281 synclvodm: WARNING, lv control block of hd11
has been over written.
0516-1281 synclvodm: WARNING, lv control block of fwdump
has been over written.
0516-622 synclvodm: Warning, cannot write lv control block data.
0516-622 synclvodm: Warning, cannot write lv control block data.
0516-622 synclvodm: Warning, cannot write lv control block data.
0516-622 synclvodm: Warning, cannot write lv control block data.
0516-622 synclvodm: Warning, cannot write lv control block data.
0516-622 synclvodm: Warning, cannot write lv control block data.
0516-622 synclvodm: Warning, cannot write lv control block data.
0516-622 synclvodm: Warning, cannot write lv control block data.
0516-622 synclvodm: Warning, cannot write lv control block data.
0516-622 synclvodm: Warning, cannot write lv control block data.
0516-622 synclvodm: Warning, cannot write lv control block data.
0516-622 synclvodm: Warning, cannot write lv control block data.
0516-622 synclvodm: Warning, cannot write lv control block data.
rootvg
ln: /dev/r: No such file or directory
getlvodm: option requires an argument -- l
Usage: getlvodm [-a LVdescript] [-B LVdescript] [-b LVid] [-c LVid]
[-C] [-d VGdescript] [-e LVid] [-F] [-g PVid] [-h] [-j PVdescript]
[-k] [-L VGdescript] [-l LVdescript] [-m LVid] [-p PVdescript]
[-r LVid] [-s VGdescript] [-t VGid] [-u VGdescript] [-v VGdescript]
[-w VGid] [-y LVid] [-G LVdescript]
0516-104 lquerylv: Incorrect LV identifier. The format must be
VG_identifier.LV_minor_num ber
Could not find "/" and/or "/usr" filesystems.
Exiting to shell.
In this point i run FSCK command and get follow:
# fsck -y /dev/hd1
fsck: Cannot find the vfs value for file system /dev/hd1
this error occured with all LV's.
OUTPUT of LSPV commad for failed disk
lspv -l 00f6aaf94f99ea03
0516-304 : Unable to find device id 00f6aaf94f99ea030000000000 000000 in the Device
Configuration Database
After i did following steps from "Removing and Replacing a Fixed Disk":
# getlvodm -v rootvg
00f6aaf900004c00000001304f 99eb37
# putlvodm -p 00f6aaf900004c00000001304f 99eb37 00f6aaf94f99ea03
after this i did again lspv -l 00f6aaf94f99ea03:
# lspv -l 00f6aaf94f99ea03
pvid=00f6aaf94f99ea03:
LV NAME LPs PPs DISTRIBUTION MOUNT POINT
hd9var 16 16 00..16..00..00..00 N/A
hd2 16 16 00..00..16..00..00 N/A
hd4 20 20 00..00..20..00..00 N/A
hd8 1 1 00..00..01..00..00 N/A
hd6 64 64 00..00..64..00..00 N/A
hd5 1 1 01..00..00..00..00 N/A
hd10opt 16 16 00..16..00..00..00 N/A
hd1 8 8 00..08..00..00..00 N/A
hd3 8 8 00..00..08..00..00 N/A
hd11admin 1 1 00..00..01..00..00 N/A
livedump 1 1 00..01..00..00..00 N/A
lg_dumplv 13 13 09..04..00..00..00 N/A
hd11 20 20 00..20..00..00..00 N/A
hd12 60 60 14..45..01..00..00 N/A
fwdump 2 2 00..02..00..00..00 N/A
# lspv -l hdisk1
hdisk1:
LV NAME LPs PPs DISTRIBUTION MOUNT POINT
hd9var 16 16 00..16..00..00..00 N/A
hd2 16 16 00..00..16..00..00 N/A
hd4 20 20 00..00..20..00..00 N/A
hd8 1 1 00..00..01..00..00 N/A
hd6 64 64 00..00..64..00..00 N/A
hd5 1 1 01..00..00..00..00 N/A
hd10opt 16 16 00..16..00..00..00 N/A
hd1 8 8 00..08..00..00..00 N/A
hd3 8 8 00..00..08..00..00 N/A
hd11admin 1 1 00..00..01..00..00 N/A
livedump 1 1 00..01..00..00..00 N/A
lg_dumplv 13 13 09..04..00..00..00 N/A
hd11 20 20 00..20..00..00..00 N/A
hd12 60 60 12..47..01..00..00 N/A
at this point i don't know what to do?
i start with bootable media in service mode and choose:
2) Access this Volume Group and start a shell before mounting filesystems
after i get the following:
Importing Volume Group...
PV Status: hdisk1 00f6aaf94f99ea43 PVACTIVE
00f6aaf94f99ea03 NONAME
varyonvg: Volume group rootvg is varied on.
0516-510 synclvodm: Physical volume not found for physical volume
identifier 00f6aaf94f99ea030000000000
0516-548 synclvodm: Partially successful with updating volume
group rootvg.
0516-1281 synclvodm: WARNING, lv control block of hd5
has been over written.
0516-1281 synclvodm: WARNING, lv control block of hd8
has been over written.
0516-1281 synclvodm: WARNING, lv control block of hd4
has been over written.
0516-1281 synclvodm: WARNING, lv control block of hd2
has been over written.
0516-1281 synclvodm: WARNING, lv control block of hd9var
has been over written.
0516-1281 synclvodm: WARNING, lv control block of hd3
has been over written.
0516-1281 synclvodm: WARNING, lv control block of hd1
has been over written.
0516-1281 synclvodm: WARNING, lv control block of hd10opt
has been over written.
0516-1281 synclvodm: WARNING, lv control block of hd11admin
has been over written.
0516-1281 synclvodm: WARNING, lv control block of livedump
has been over written.
0516-1281 synclvodm: WARNING, lv control block of hd12
has been over written.
0516-1281 synclvodm: WARNING, lv control block of hd11
has been over written.
0516-1281 synclvodm: WARNING, lv control block of fwdump
has been over written.
0516-622 synclvodm: Warning, cannot write lv control block data.
0516-622 synclvodm: Warning, cannot write lv control block data.
0516-622 synclvodm: Warning, cannot write lv control block data.
0516-622 synclvodm: Warning, cannot write lv control block data.
0516-622 synclvodm: Warning, cannot write lv control block data.
0516-622 synclvodm: Warning, cannot write lv control block data.
0516-622 synclvodm: Warning, cannot write lv control block data.
0516-622 synclvodm: Warning, cannot write lv control block data.
0516-622 synclvodm: Warning, cannot write lv control block data.
0516-622 synclvodm: Warning, cannot write lv control block data.
0516-622 synclvodm: Warning, cannot write lv control block data.
0516-622 synclvodm: Warning, cannot write lv control block data.
0516-622 synclvodm: Warning, cannot write lv control block data.
rootvg
ln: /dev/r: No such file or directory
getlvodm: option requires an argument -- l
Usage: getlvodm [-a LVdescript] [-B LVdescript] [-b LVid] [-c LVid]
[-C] [-d VGdescript] [-e LVid] [-F] [-g PVid] [-h] [-j PVdescript]
[-k] [-L VGdescript] [-l LVdescript] [-m LVid] [-p PVdescript]
[-r LVid] [-s VGdescript] [-t VGid] [-u VGdescript] [-v VGdescript]
[-w VGid] [-y LVid] [-G LVdescript]
0516-104 lquerylv: Incorrect LV identifier. The format must be
VG_identifier.LV_minor_num
Could not find "/" and/or "/usr" filesystems.
Exiting to shell.
In this point i run FSCK command and get follow:
# fsck -y /dev/hd1
fsck: Cannot find the vfs value for file system /dev/hd1
this error occured with all LV's.
OUTPUT of LSPV commad for failed disk
lspv -l 00f6aaf94f99ea03
0516-304 : Unable to find device id 00f6aaf94f99ea030000000000
Configuration Database
After i did following steps from "Removing and Replacing a Fixed Disk":
# getlvodm -v rootvg
00f6aaf900004c00000001304f
# putlvodm -p 00f6aaf900004c00000001304f
after this i did again lspv -l 00f6aaf94f99ea03:
# lspv -l 00f6aaf94f99ea03
pvid=00f6aaf94f99ea03:
LV NAME LPs PPs DISTRIBUTION MOUNT POINT
hd9var 16 16 00..16..00..00..00 N/A
hd2 16 16 00..00..16..00..00 N/A
hd4 20 20 00..00..20..00..00 N/A
hd8 1 1 00..00..01..00..00 N/A
hd6 64 64 00..00..64..00..00 N/A
hd5 1 1 01..00..00..00..00 N/A
hd10opt 16 16 00..16..00..00..00 N/A
hd1 8 8 00..08..00..00..00 N/A
hd3 8 8 00..00..08..00..00 N/A
hd11admin 1 1 00..00..01..00..00 N/A
livedump 1 1 00..01..00..00..00 N/A
lg_dumplv 13 13 09..04..00..00..00 N/A
hd11 20 20 00..20..00..00..00 N/A
hd12 60 60 14..45..01..00..00 N/A
fwdump 2 2 00..02..00..00..00 N/A
# lspv -l hdisk1
hdisk1:
LV NAME LPs PPs DISTRIBUTION MOUNT POINT
hd9var 16 16 00..16..00..00..00 N/A
hd2 16 16 00..00..16..00..00 N/A
hd4 20 20 00..00..20..00..00 N/A
hd8 1 1 00..00..01..00..00 N/A
hd6 64 64 00..00..64..00..00 N/A
hd5 1 1 01..00..00..00..00 N/A
hd10opt 16 16 00..16..00..00..00 N/A
hd1 8 8 00..08..00..00..00 N/A
hd3 8 8 00..00..08..00..00 N/A
hd11admin 1 1 00..00..01..00..00 N/A
livedump 1 1 00..01..00..00..00 N/A
lg_dumplv 13 13 09..04..00..00..00 N/A
hd11 20 20 00..20..00..00..00 N/A
hd12 60 60 12..47..01..00..00 N/A
at this point i don't know what to do?
This question needs an answer!
Become an EE member today
7 DAY FREE TRIALMembers can start a 7-Day Free trial then enjoy unlimited access to the platform.
View membership options
or
Learn why we charge membership fees
We get it - no one likes a content blocker. Take one extra minute and find out why we block content.
It looks like your hdisk0 is either removed missing or failed.
To diagnose and fix these typical problems, you must boot from bootable media and run fsck to fix any file systems that may be corrupted. If fsck fails when calling logredo then run logform on /dev/hd8.
WARNING: Do not use this instruction if the system is a /usr client, diskless client, or dataless client.
1. Power down your system. If it has a key mode select switch (that is, if it is a Microchannel-based system), turn the key to the Service position.
2. With bootable media of the same version and technology level as the system, boot the system.
The bootable media can be any one of the following:
- Bootable CD-ROM
- NON_AUTOINSTALL mksysb
- Bootable Install Tape
Follow the screen prompts to the following menu:
Welcome to Base Operating System
Installation and Maintenance
3. Choose Start Maintenance Mode for System Recovery (Option 3). The next screen has the Maintenance Menu.
Choose Access a Root Volume Group (Option 1).
The next screen displays a warning that indicates you will not be able to return to the Base OS menu without rebooting.
Choose 0 continue.
The next screen displays information about all volume groups on the system.
Select the root volume group by number.
Choose Access this volume group and start a shell before mounting the filesystems (Option 2).
If you get errors indicating that a physical volume is missing from the rootvg, run diagnostics on the physical volumes to find out if you have a bad disk. Do not continue with the rest of the steps in this document.
If you get other errors from the preceding option, do not continue with the rest of the steps in this document. Correct the problem causing the error. If you need assistance correcting the problem, contact one of the following:
your local branch office
your point of sale
your AIX support center
4. Next, run the following commands to check and repair file systems. (The -y option gives fsck permission to repair file systems when necessary.)
fsck -y /dev/hd1
fsck -y /dev/hd2
fsck -y /dev/hd3
fsck -y /dev/hd4
fsck -y /dev/hd9var
5. If fsck fails during the logredo phase, you will have to format the default jfslog (or j2fslog) for the rootvg JFS (or JFS2) file systems.
/usr/sbin/logform /dev/hd8
Answer yes when asked if you want to destroy the log.
6. Type exit. The file systems will automatically mount after you type exit.
7. If you are running the Andrew File System (AFS), use the following commands to find out whether you have more than one version of the v3fshelper file. Otherwise, skip to step 8.
cd /sbin/helpers
ls -l v3fshelper*
If you have only one version of the v3fshelper file (for example, v3fshelper), proceed to step 8.
If there is a version of v3fshelper marked as original (for example, v3fshelper.orig), run the following commands:
copy v3fshelper v3fshelper.afs
copy v3fshelper.orig v3fshelper
8. If rootvg is mirrored, verify that all drives in the volume group are active:
lsvg -p rootvg
If either one of the drives is either in a "Missing" or "Removed" state, or if you see just the PVID along with the following error:
0516-304 Unable to find device id in the device configuration database
You will need to perform the approriate procedures for reducing that drive out of rootvg. This procedure can be found in the document, "Removing and Replacing a Fixed Disk." Once that has been completed, you can proceed to the next step.
9. Determine which disk is the boot disk with the lslv command. The boot disk will be shown in the PV1 column of the lslv output.
lslv -m hd5
WARNING: Do not proceed further if the system is a /usr client, diskless client, or dataless client.
10. Recreate the boot image and alter the boot list. (hdisk# is the boot disk determined in step 9.)
bosboot -a -d /dev/ipldevice
bootlist -m normal <hdisk#>
11. If you copied the v3fshelper file in step 7, copy AFS file system helper back to v3fshelper:
copy v3fshelper.afs v3fshelper
12. With the key in Normal position, run
shutdown -Fr
If you followed all of the preceding steps and the system still stops at an LED 551, 555, or 557 during a reboot in Normal mode, you may want to consider reinstalling your system from a recent backup. Isolating the cause of the hang could be excessively time-consuming and may not be cost-effective in your operating environment.
To isolate the possible cause of the hang would require a debug boot of the system. Instructions for doing this are included in the document "Capturing Boot Debug", available at http://techsupport.services.ibm.com/server/aix.srchBroker.
It is still possible, in the end, that isolation of the problem may indicate a restore or reinstall of AIX is necessary to correct it.
Cheers