Mesed up with Linux partition resizing...

Hi,

I messed up today with production server on Linux Ubuntu 10.04 LTS. I wanted to extend LVM partition on Virtual Linux server (ESX 5.0), something came out during my work and I I forgot to make snapshot!

What I did already?
First, I shut down Linux server (with intention to make snapshot...but I forgot), added some 35GB to virtual DISK space in ESX, and powered up machine.

This is BEFORE any changes:
root@enginex:/home/labsy# fdisk -l

Disk /dev/sda: 80.5 GB, 80530636800 bytes
255 heads, 63 sectors/track, 9790 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x000f1f94

   Device Boot      Start         End      Blocks   Id  System
/dev/sda1   *           1          32      248832   83  Linux
Partition 1 does not end on cylinder boundary.
/dev/sda2              32        2611    20719617    5  Extended
/dev/sda3            2611        5221    20967186+  83  Linux
/dev/sda5              32        2611    20719616   8e  Linux LVM

Open in new window

After Virtual disk RESIZING, I did this:
root@enginex:/home/labsy# fdisk /dev/sda

WARNING: DOS-compatible mode is deprecated. It's strongly recommended to
         switch off the mode (command 'c') and change display units to
         sectors (command 'u').

Command (m for help): c
DOS Compatibility flag is not set

Command (m for help): u
Changing display/entry units to sectors

Command (m for help): p

Disk /dev/sda: 80.5 GB, 80530636800 bytes
255 heads, 63 sectors/track, 9790 cylinders, total 157286400 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x000f1f94

   Device Boot      Start         End      Blocks   Id  System
/dev/sda1   *        2048      499711      248832   83  Linux
/dev/sda2          501758    41940991    20719617    5  Extended
/dev/sda3        41940992    83875364    20967186+  83  Linux
/dev/sda5          501760    41940991    20719616   8e  Linux LVM

Command (m for help): n
Command action
   l   logical (5 or over)
   p   primary partition (1-4)
p
Selected partition 4
First sector (499712-157286399, default 499712):
Using default value 499712
Last sector, +sectors or +size{K,M,G} (499712-501757, default 501757):
Using default value 501757

Command (m for help): w
The partition table has been altered!

Calling ioctl() to re-read partition table.

WARNING: Re-reading the partition table failed with error 16: Device or resource busy.
The kernel still uses the old table. The new table will be used at
the next reboot or after you run partprobe(8) or kpartx(8)
Syncing disks.

Open in new window

Rebooted again.

Then my situation was as follows:
root@enginex:/home/labsy# fdisk -l

Disk /dev/sda: 80.5 GB, 80530636800 bytes
255 heads, 63 sectors/track, 9790 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x000f1f94

   Device Boot      Start         End      Blocks   Id  System
/dev/sda1   *           1          32      248832   83  Linux
Partition 1 does not end on cylinder boundary.
/dev/sda2              32        2611    20719617    5  Extended
/dev/sda3            2611        5221    20967186+  83  Linux
/dev/sda4              32          32        1023   83  Linux
Partition 4 does not end on cylinder boundary.
/dev/sda5              32        2611    20719616   8e  Linux LVM

Partition table entries are not in disk order

Open in new window

Here I should have fixed partition table entries, but I did not!
MISTAKE!

So I messed up with the following commands, which I need to UNDO somehow:
root@enginex:/home/labsy# pvcreate /dev/sda4
  Physical volume "/dev/sda4" successfully created
root@enginex:/home/labsy# vgdisplay | grep "Name"
  VG Name               enginex

Open in new window

root@enginex:/home/labsy# vgextend enginex /dev/sda4
  Volume group "enginex" successfully extended
root@enginex:/home/labsy# vgdisplay enginex | grep "Free"
  Free  PE / Size       2 / 8.00 MiB

Open in new window

Please, advice, how to proceed.
As you see, I extended with WRONG volume, which is only 8MB in size, instead of 35GB.
I need to revert these TWLO last actions to get partition tables fixed first.
LVL 18
Andrej PirmanAsked:
Who is Participating?
I wear a lot of hats...

"The solutions and answers provided on Experts Exchange have been extremely helpful to me over the last few years. I wear a lot of hats - Developer, Database Administrator, Help Desk, etc., so I know a lot of things but not a lot about one thing. Experts Exchange gives me answers from people who do know a lot about one thing, in a easy to use platform." -Todd S.

Daniel HelgenbergerCommented:
I hope I did get this right, please verify:

1. Everything is still running fine.
2. You created a partition, but at the wrong place, its only 8MB
3. You did create a PV from this partition and added this to your VG.

Now, you want to remove the PV to remove the partition.

If this is the case, just do it the other way around:
vgreduce enginex /dev/sda4
pvremove /dev/sda4

Open in new window


Create your snapshot first, tough.

But the PV does no harm at all, why remove it?
0
arnoldCommented:
Your fdisk -l reflects the same disk size

Not sure why did not add a send drive instead of trying to resize the existing one..
0
Andrej PirmanAuthor Commented:
Yes,

to confirm:
1.) Everything is still working fine.
2.) I created partition, but partition tables did not reflect the correct layout.
3.) I should've fixed the partition tables BEFORE I created PV from this partition, so I stuck by extending VG with 8MB of "some" space, instead of wanted 35GB

Why remove?
Well, first of all, I do not know, which space I added. I obviously did something I do not understand, and would like to get back to the path I know :)

@Arnold:
Due to a simple fact - I am copy-paste google Linux guy, especially here at LVM. I understand Windows partition layout perfectly, but I am totally lost within these LVM partitions, volumes, physical etc.
For me, it is that BEFORE I have xy space for files, then do some mambo jambo in Linux console...and voila, I have more space available :)
0
10 Tips to Protect Your Business from Ransomware

Did you know that ransomware is the most widespread, destructive malware in the world today? It accounts for 39% of all security breaches, with ransomware gangsters projected to make $11.5B in profits from online extortion by 2019.

arnoldCommented:
LVM has two ways to extend.
One by adjusting the space of the existing drive and then going through the process of having the host filesystem to recognize the change.
You do not seem to have gone through the process of having the host system recognize that the disk allocation has increased from 80gb to 115GB.
The 8MB is the space that was left from your initial space.given you tried to make it a primary partition.

http://kb.vmware.com/selfservice/microsites/search.do?cmd=displayKC&docType=kc&docTypeID=DT_KB_1_1&externalId=1004071

http://www.redhat.com/archives/dm-devel/2010-January/msg00065.html

Echo 1 /proc/block/sda/device/rescan

The other way is you add a second drive /dev/sdb
You create a single Linux LVM for the entire volume
Pvcreate /dev/sdb1
Vgextend your_volumegroup /dev/sdb1
http://linux.die.net/man/8/vgextend
You can then use lvextend to increase the size of a logical partition.
http://linux.die.net/man/8/lvextend
0

Experts Exchange Solution brought to you by

Your issues matter to us.

Facing a tech roadblock? Get the help and guidance you need from experienced professionals who care. Ask your question anytime, anywhere, with no hassle.

Start your 7-day free trial
Andrej PirmanAuthor Commented:
Thanx Arnold.
I've been following this tutorial, which I once made after playing a lot with different methods of LVM extending: http://www.zimbra.com/forums/administrators/55983-create-new-partition-extend-existing-one.html#post243191
I guess it is the 2nd method you mentioned.

So, I went with SNAPSHOT first, then reduced VG and removed PV. Then I removed partition and after reboot recreated it back.
This time it settled down correctly and partition table correctly represented actual layout.
Then followed those commands:

pvcreate /dev/sda4
vgextend enginex /dev/sda4
vgdisplay enginex | grep "Free"
lvdisplay | grep "Name"
lvextend -L+35G /dev/enginex/root
resize2fs /dev/enginex/root

Open in new window


I expected FSCK to be forced upon reboot to check superblocks...but that did not happen. Weird. Just in case I forced fsck upon reboot, but it did only check /dev/sda1 and /dev/sda2

touch /forcefsck

Open in new window


Would it be fine to force also /dev/sda3 and /dev/sda4 to be checked?
If yes, how? :)
0
arnoldCommented:
You should check the logical volumes.
Df -k

Fsck /dev/mapper/volumegroup-logicalvolume

You seem to be trying/looking at running fsck on underlying devices that do not have a filesystem
/dev/sda3 is an extended partition
/dev/sda4 is another Linux LVMpartition.
0
Andrej PirmanAuthor Commented:
Hmmm...not like that "none":
root@enginex:/home/labsy# df -k
Filesystem           1K-blocks      Used Available Use% Mounted on
/dev/mapper/enginex-root
                      76257288  25076440  47326448  35% /
none                   3059172       184   3058988   1% /dev
none                   3064048         0   3064048   0% /dev/shm
none                   3064048        76   3063972   1% /var/run
none                   3064048         0   3064048   0% /var/lock
none                   3064048         0   3064048   0% /lib/init/rw
/dev/sda1               233191     20152    200598  10% /boot

Open in new window

Regarding FSCK, I guess I should have somehow schedule it to run before FS is mounted:
root@enginex:/home/labsy# fsck /dev/mapper/enginex-root
fsck from util-linux-ng 2.17.2
e2fsck 1.41.11 (14-Mar-2010)
/dev/mapper/enginex-root is mounted.

WARNING!!!  The filesystem is mounted.   If you continue you ***WILL***
cause ***SEVERE*** filesystem damage.

Do you really want to continue (y/n)? no

check aborted.

Open in new window

0
arnoldCommented:
All those referenced as none are fine.
0
Andrej PirmanAuthor Commented:
Thank you all!
0
It's more than this solution.Get answers and train to solve all your tech problems - anytime, anywhere.Try it for free Edge Out The Competitionfor your dream job with proven skills and certifications.Get started today Stand Outas the employee with proven skills.Start learning today for free Move Your Career Forwardwith certification training in the latest technologies.Start your trial today
Linux

From novice to tech pro — start learning today.