Link to home
Start Free TrialLog in
Avatar of keith_opswat
keith_opswatFlag for United States of America

asked on

FSTAB Missing... Can't boot in Fedora Core

So I started this job around 2 months ago and even though this Software Development company had been around for 7 years they have never had someone hired on whose main duty was System/Network Administration & Network Security. So needless to say having a couple different people randomly doing things when they need to be done creates a huge unorganized mess after 7 years.

So besides trying to move into a network & organize everything as well as learn what's going on I'm having to deal with things that just quit working for no apparent reason except probably it was setup by someone who doesn't know what they're doing.

So anyways, we have this server that's running our wiki database, our bot controller, as well as some other random things. But last week most of the services on it went down so after some looking around I found that it had run out of space. It was using LVM file system and to tell you how long ago this was setup it was running Fedora Core 2 and was on a 160GB Sata 1.5 hard drive. Holy crap... It's also in some ancient dell tower which was sideways sitting on top of a switch that was mounted in the rack. ohhh me I have too much work to do here.

So anyways I cloned the disk over to a 1.5 TB drive and attempted to extend the Volume Group... I asked around on here and the directions I got seemed to match up with ones on other websites. I did the commands and I think I must have forgot something that has to do with fstab because now I'm getting this message upon boot...


Reading All Physical Volumes.... This could take a while.

Couldn't find device with UID FKTS-etc etc... (I'm guessing this is because I changed hard drives..)

Couldn't find all physical volumes for volume group
VolGroup00
Volume Group 00 not found

ERROR: /bin/lvm exited abnormally

mount:error 6 mounting ext3
mount:error 2 mounting none

switchroot: mount failed : 22

umount /initrd/dev failed : 2

Open in new window


Now even when I try to boot into Fedora Rescue CD it looks for old Linux Filesystems and says it didn't find any linux partitions and that it would restart after I exited the shell.

The shell came up as sh-3.1#

I tried to mount /dev/sda and it says couldn't find /etc/fstab

Sure enough... /etc/fstab is not there. It doesn't seem to mount my disk for use anywhere... I don't know how to fix this issue. I get the grub loader when I boot the computer and everything.

If I do the fdisk -l command I get this...

Device    Boot              Start              End        Blocks      Id       System
/dev/sda1    *                     1            13024   31257568+  83        Linux
/dev/sda2                     13025         78084  156143767+  8e        Linux LVM
/dev/sda3                    130238      130238   125169600   8e        Linux LVM


So this is obviously on the new drive... (I accidentally left the old one I had cloned it too plugged into SATA and accidentally ran some commands that well... Also ruined the VolGroups. I'm awesome I know)

So /dev/sda3 was made by me following the directions I got. I created it with the rest of the unpartitioned space. Here are the commands I did... (Well pretty sure I did... maybe I left one of them out and that's why everything is messed up)

This is the output on the guys computer... So he's doing sdb and I had sda etc.. You get the drift.

Step 1: Create a new physical volume to take up all that lovely free space

# fdisk -u /dev/sdb

WARNING: DOS-compatible mode is deprecated. It's strongly recommended to
         switch off the mode (command 'c').

Command (m for help): p

Disk /dev/sdb: 80.0 GB, 80026361856 bytes
164 heads, 27 sectors/track, 35298 cylinders, total 156301488 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x45434952

   Device Boot      Start         End      Blocks   Id  System
/dev/sdb1               1    78295039    39147519+  8e  Linux LVM

Command (m for help): n
Command action
   e   extended
   p   primary partition (1-4)
p
Partition number (1-4): 2
First sector (78295040-156301487, default 78295040):
Using default value 78295040
Last sector, +sectors or +size{K,M,G} (78295040-156301487, default 156301487):
Using default value 156301487

Command (m for help): t
Partition number (1-4): 2
Hex code (type L to list codes): 8e
Changed system type of partition 2 to 8e (Linux LVM)

Command (m for help): p

Disk /dev/sdb: 80.0 GB, 80026361856 bytes
164 heads, 27 sectors/track, 35298 cylinders, total 156301488 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x45434952

   Device Boot      Start         End      Blocks   Id  System
/dev/sdb1               1    78295039    39147519+  8e  Linux LVM
/dev/sdb2        78295040   156301487    39003224   8e  Linux LVM

Command (m for help): w
The partition table has been altered!

Calling ioctl() to re-read partition table.

WARNING: Re-reading the partition table failed with error 16: Device or resource busy.
The kernel still uses the old table. The new table will be used at
the next reboot or after you run partprobe(8) or kpartx(8)
Syncing disks

You can just accept the default values like I did, thats why it shows no reply to some of the questions, I just pressed enter and it will make a new partition that takes up all the space.

Step 2: Create a new PHYSICAL VOLUME for use with your VOLUME GROUP.

bob:~ # pvcreate /dev/sdb2
  Physical volume "/dev/sdb2" successfully created

If you receive an error about ignored by filtering, reboot your machine and try again.

Step 3: Add new volume to volume group

bob:~ # vgextend VolGroup00 /dev/sdb2
  Volume group "VolGroup00" successfully extended

(My volume group has a cool name, but I put yours in so you can copy and paste)

Step 4: Add space to the logical volume you want. Probably your root one LogVol00

bob:~ # lvs
  LV   VG   Attr   LSize  Origin Snap%  Move Log Copy%  Convert
  root pc   -wi-ao 53.23g                                      
  swap pc   -wi-ao 12.00g                                      

Those show mine, you want to add space to LogVol00 though so you go:

bob:~ # lvextend -l +100%FREE /dev/mapper/VolGroup00-LogVol00
  Extending logical volume root to 99.72 GiB
  Logical volume root successfully resized

bob:~ # lvs
  LV   VG   Attr   LSize  Origin Snap%  Move Log Copy%  Convert
  root pc   -wi-ao 99.72g                                      
  swap pc   -wi-ao 12.00g                                      
bob:~ #


So that's what I did or am pretty sure I did.

So any help at least telling me what I did wrong would be cool.

Also... now since I accidentally jacked my original install I still have it on a 1.5TB drive but the only issue is I told clonezilla to make the filesystem use the rest of the free space for the partition and now /dev/sda2 on that drive takes up all the space, but it's still saying it has 0 disk space. So I need to either make sda2 see all the rest of the space or extract all the information and somehow get it working properly.

Thanks in advance... This topic seems to always take a lot of work for both the person answering and myself. Lol
Avatar of arnold
arnold
Flag of United States of America image

My guess based on the age and the size of the drive you are using that it is the issue.
you made the /boot partion the linux (83)  type too big.


My suggestion would be to place the old drive back and boot into singleuser and cleanup the system to free up some space.  My guess is that there is a huge log file that was never setup for logrotate and it grew too much.

Additional option is to add a drive if possible and then use find /path | cpio -pdvmu /mnt where /mnt is a new partition that will be mounted over the /path i.e.
mv /path /path.old mkdir /path, mount /dev/sdbx /path.
Avatar of keith_opswat

ASKER

Well no... I know my post was hella long so it's easy to miss some of the details.

The original 160GB drive I accidentally ruined... The only working copy I have left is on a 1.5TB disk and supposedly sda2 is taking up around 900GB of space for it's partition but inside the file system it still shows it in it's 140 something GB form.

How do I make it realize that it has all that extra space? It's not unused anymore according to any partition viewer, but it's not able to be written to because I didn't set it up right with LVM I think.
run lvdisplay.

You may fair better by using explicit numbers to extend the logical volume.

what is the output from:
pvscan
vgscan
vgdisplay
lvscan
lvdisplay


Thanks so much for your help here arnold... Here's the information you asked for. As it looks like to me all the extra like 1.3TB of data are apart of one of the partitions but the LogVol00 isn't using it or something... Not 100% on it. My life will seriously be 10000 times easier if this can be figured out so I'm hoping it's fixable.

Thanks again... And here's the info!!


[root@database ~]# pvscan
  PV /dev/hda2   VG VolGroup00   lvm2 [148.91 GB / 128.00 MB free]
  Total: 1 [148.91 GB] / in use: 1 [148.91 GB] / in no VG: 0 [0   ]


[root@database ~]# vgscan
  Reading all physical volumes.  This may take a while...
  Found volume group "VolGroup00" using metadata type lvm2


[root@database ~]# vgdisplay
  --- Volume group ---
  VG Name               VolGroup00
  System ID
  Format                lvm2
  Metadata Areas        1
  Metadata Sequence No  3
  VG Access             read/write
  VG Status             resizable
  MAX LV                0
  Cur LV                2
  Open LV               2
  Max PV                0
  Cur PV                1
  Act PV                1
  VG Size               148.91 GB
  PE Size               32.00 MB
  Total PE              4765
  Alloc PE / Size       4761 / 148.78 GB
  Free  PE / Size       4 / 128.00 MB
  VG UUID               ZnMJ2o-Z3SM-URJU-kNWX-qpLP-XOdo-aaZ2YE


[root@database ~]# lvscan
  ACTIVE            '/dev/VolGroup00/LogVol00' [146.84 GB] inherit
  ACTIVE            '/dev/VolGroup00/LogVol01' [1.94 GB] inherit


[root@database ~]# lvdisplay
  --- Logical volume ---
  LV Name                /dev/VolGroup00/LogVol00
  VG Name                VolGroup00
  LV UUID                OoCy3z-v83g-7Ofz-2Yi2-peom-7vcb-z4qvvY
  LV Write Access        read/write
  LV Status              available
  # open                 1
  LV Size                146.84 GB
  Current LE             4699
  Segments               1
  Allocation             inherit
  Read ahead sectors     0
  Block device           253:0

  --- Logical volume ---
  LV Name                /dev/VolGroup00/LogVol01
  VG Name                VolGroup00
  LV UUID                apIUQI-OO0J-H23Y-dRpU-R5gd-t5Oy-6a5l75
  LV Write Access        read/write
  LV Status              available
  # open                 1
  LV Size                1.94 GB
  Current LE             62
  Segments               1
  Allocation             inherit
  Read ahead sectors     0
  Block device           253:1
Just an update... I no longer have the FSTAB missing issue where I can't find VolGroups anymore.

I still have that issue on my original 160GB drive but I was able to get the OS back up on my 1.5 TB drive but the partition is still messed up. Still has the same partition size and I need to expand it out to have more free space. if at all possible.

Thanks again, just wanted to make sure that fact was clear. So it's not 100% necessary anymore to know how to fix the FSTAB. I'm going to be 100x more careful in the future with what I run and making sure I have a viable backup of something before making such dangerous changes to it.
According to pvscan, volgroup00 is configured with only 149GB but is referencing /dev/hda2

You ran vgextend to use /dev/sdb2 but it is not reflected.


Ooops... I had the hard drive in a different computer when I was working on it. It referenced the drive with a different name. Here's the new pvscan

[root@database ~]# pvscan
  PV /dev/sda2   VG VolGroup00   lvm2 [148.91 GB / 128.00 MB free]
  Total: 1 [148.91 GB] / in use: 1 [148.91 GB] / in no VG: 0 [0   ]

I don't think I ever ran any of those commands on this drive... Wow. I thought my clone I had now had been taken after those commands were run.

Dude, I'm so sorry for giving you false information. I just realized what the story is now. I had so many different clones of it on different hard drives, trying to expand the file system that I got lost in my disorganization.

So if you're kind enough to hang with me still... I understand if not. But if you do here's the actual story as of now. The stuff I posted above was done to my hard drive that is cooked now... Also the sdb2 was the command the guy who was telling me how to do it used. My drive is and has always been sda except for in that other computer because it was using a different sata driver.

So the story as of now.

Had the 160GB drive that filled with disk space. Cloned it with clonezilla onto different drives trying to get clonezilla to automatically resize the partitions which obviously didn't work. So the drive it's on now was done by using clonezilla and telling clonezilla to fill the partition to the extent of the drive free space.

So even though clonezilla set the partition to take up the rest of drive obviously the filesystem knows nothing of this. it only knows of the original  149GB partition that it had on the previous drive.

So I will repost all the commands that you asked for above but this time getting the info from the drive when it's in the correct computer, with it's real names, and real info at the moment. And remember.... None of the vgcreate, vgextend commands have been run at all. So is there ANY way at all to make it be able to use any of that space?

I feel really really stupid by the way.. I can't believe I made that mistake and led you on this wild goose chase. I'm really sorry.

PVSCAN
[code]
[root@database ~]# pvscan
  PV /dev/sda2   VG VolGroup00   lvm2 [148.91 GB / 128.00 MB free]
  Total: 1 [148.91 GB] / in use: 1 [148.91 GB] / in no VG: 0 [0   ]
[\code]

VGSCAN
[code]
[root@database ~]# vgscan
  Reading all physical volumes.  This may take a while...
  Found volume group "VolGroup00" using metadata type lvm2
[/code]

VGDISPLAY
[code]
[root@database ~]# vgdisplay
  --- Volume group ---
  VG Name               VolGroup00
  System ID
  Format                lvm2
  Metadata Areas        1
  Metadata Sequence No  3
  VG Access             read/write
  VG Status             resizable
  MAX LV                0
  Cur LV                2
  Open LV               2
  Max PV                0
  Cur PV                1
  Act PV                1
  VG Size               148.91 GB
  PE Size               32.00 MB
  Total PE              4765
  Alloc PE / Size       4761 / 148.78 GB
  Free  PE / Size       4 / 128.00 MB
  VG UUID               ZnMJ2o-Z3SM-URJU-kNWX-qpLP-XOdo-aaZ2YE
[/code]

LVSCAN
[code]
[root@database ~]# lvscan
  ACTIVE            '/dev/VolGroup00/LogVol00' [146.84 GB] inherit
  ACTIVE            '/dev/VolGroup00/LogVol01' [1.94 GB] inherit
[/code]

LVDISPLAY
[code]
[root@database ~]# lvdisplay
  --- Logical volume ---
  LV Name                /dev/VolGroup00/LogVol00
  VG Name                VolGroup00
  LV UUID                OoCy3z-v83g-7Ofz-2Yi2-peom-7vcb-z4qvvY
  LV Write Access        read/write
  LV Status              available
  # open                 1
  LV Size                146.84 GB
  Current LE             4699
  Segments               1
  Allocation             inherit
  Read ahead sectors     0
  Block device           253:0

  --- Logical volume ---
  LV Name                /dev/VolGroup00/LogVol01
  VG Name                VolGroup00
  LV UUID                apIUQI-OO0J-H23Y-dRpU-R5gd-t5Oy-6a5l75
  LV Write Access        read/write
  LV Status              available
  # open                 1
  LV Size                1.94 GB
  Current LE             62
  Segments               1
  Allocation             inherit
  Read ahead sectors     0
  Block device           253:1
[/code]

Ok so now that I realize I haven't actually told LVM to add the extra drive space in there like I thought and it was just clonezilla which added the extra data onto it. So technically can't we just take the data back from the partition or extend the filesystem onto that space if we know what block or cylinder the current LV's end on?  Or am I completely off base.

Once again I'm sorry for the mis-information and hope you can still help me out.
So here's my fdisk -l which shows where the cylinders start and end for the partitions now according to LVM.

[root@database ~]# fdisk -l

Disk /dev/sda: 1500.3 GB, 1500301910016 bytes
255 heads, 63 sectors/track, 182401 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes

   Device Boot      Start         End      Blocks   Id  System
/dev/sda1   *           1         122      978853+  83  Linux
Partition 1 does not end on cylinder boundary.
/dev/sda2             122      182398  1464128879   8e  Linux LVM


So technically I should be able to take the blocks or something right? because the cylinders are off. It says it's using almost all the cylinders but it's obviously not seeing the data. I dunno this is kinda confusing me. I'm getting better but honestly haven't been actually digging into the linux file system for  long.

The most I ever did before this job really was install Ubuntu or the likes letting it partition everything for me then install all the programs I needed for my ethical hacking & network security courses.

But i'm really starting to like using Linux but the beginning part of the learning curve is steep. All though I'm finding that once I figure out how to do something, or get help from someone the other things become a lot easier. It's just learning the basics that take a bit.
ASKER CERTIFIED SOLUTION
Avatar of arnold
arnold
Flag of United States of America image

Link to home
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
Start Free Trial
I know there's another way to do this... But you took your time to help me out and technically your way will work.

So gonna give you the points at least since you took the time to help me.

Thanks a lot for taking the time to help me!!