Expanding an Logical Volume on an expanded HP MSA60

sundoginteractive
sundoginteractive used Ask the Experts™
on
Ok the rundown:
Fedora 11
HP MSA60 attached to p800 controller.
Shelf had 8-750 GB drives in it for starters using RAID 6. Used LVM to create a VG.
All of that is working fine.

I added 4 more 750GB drives to the shelf. I used the hpacucli to expand the RAID array and everything is fine as far as the MSA60 and controller are concerned - it shows a nice 6.8TB array.

in Fedora, an fdisk -l /dev/cciss/c0d0 (the msa) shows this:
[root@vs02 ~]# fdisk -l /dev/cciss/c0d0

WARNING: GPT (GUID Partition Table) detected on '/dev/cciss/c0d0'! The util fdisk doesn't support GPT. Use GNU Parted.

Disk /dev/cciss/c0d0: 7501.2 GB, 7501224730624 bytes
255 heads, 63 sectors/track, 911971 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Disk identifier: 0x00000000

           Device Boot      Start         End      Blocks   Id  System
/dev/cciss/c0d0p1               1           1          16+  ee  GPT
/dev/cciss/c0d0p2   *           1      267350  2147483647+  8e  Linux LVM



lvdisplay -m shows this (for the relevant volume group):

  --- Logical volume ---
  LV Name                /dev/VolGroup01/LogVol00
  VG Name                VolGroup01
  LV UUID                WdHpL0-BF93-Efxs-Kn8d-CxMS-p0KW-s2sy8A
  LV Write Access        read/write
  LV Status              available
  # open                 1
  LV Size                4.09 TB
  Current LE             134131
  Segments               1
  Allocation             inherit
  Read ahead sectors     auto
  - currently set to     256
  Block device           253:2

  --- Segments ---
  Logical extent 0 to 134130:
    Type                linear
    Physical volume     /dev/cciss/c0d0p1
    Physical extents    0 to 134130



vgdisplay shows this:

 --- Volume group ---
  VG Name               VolGroup01
  System ID
  Format                lvm2
  Metadata Areas        1
  Metadata Sequence No  2
  VG Access             read/write
  VG Status             resizable
  MAX LV                0
  Cur LV                1
  Open LV               1
  Max PV                0
  Cur PV                1
  Act PV                1
  VG Size               4.09 TB
  PE Size               32.00 MB
  Total PE              134132
  Alloc PE / Size       134131 / 4.09 TB
  Free  PE / Size       1 / 32.00 MB
  VG UUID               6S1bpq-aw8j-nkJi-qex2-jQ7Q-npHC-2hCP3R


So basically, the system sees that there is a ~7.3TB disk that the current LV / VG resides on, though it's pointing to a weird GPT partition... not sure what that's all about...

Anyway, the total size of this array is actually 6.8TB, here's the output in the HP ACU cli utility of show config:
=> ctrl slot=6 show config

Smart Array P800 in Slot 6                (sn: P98690G9SV50EX)

   array A (SATA, Unused Space: 0 MB)

      logicaldrive 1 (6.8 TB, RAID 6 (ADG), OK)

      physicaldrive 2E:1:1 (port 2E:box 1:bay 1, SATA, 750 GB, OK)
      physicaldrive 2E:1:2 (port 2E:box 1:bay 2, SATA, 750 GB, OK)
      physicaldrive 2E:1:3 (port 2E:box 1:bay 3, SATA, 750 GB, OK)
      physicaldrive 2E:1:4 (port 2E:box 1:bay 4, SATA, 750 GB, OK)
      physicaldrive 2E:1:5 (port 2E:box 1:bay 5, SATA, 750 GB, OK)
      physicaldrive 2E:1:6 (port 2E:box 1:bay 6, SATA, 750 GB, OK)
      physicaldrive 2E:1:7 (port 2E:box 1:bay 7, SATA, 750 GB, OK)
      physicaldrive 2E:1:8 (port 2E:box 1:bay 8, SATA, 750 GB, OK)
      physicaldrive 2E:1:9 (port 2E:box 1:bay 9, SATA, 750 GB, OK)
      physicaldrive 2E:1:10 (port 2E:box 1:bay 10, SATA, 750 GB, OK)
      physicaldrive 2E:1:11 (port 2E:box 1:bay 11, SATA, 750 GB, OK)
      physicaldrive 2E:1:12 (port 2E:box 1:bay 12, SATA, 750 GB, OK)

Now all I want to do is expand my VG / LV to include the space from the 4 drives I just added... but i can't seem to find a way to do this. Can anyone help??



Comment
Watch Question

Do more with

Expert Office
EXPERT OFFICE® is a registered trademark of EXPERTS EXCHANGE®
Top Expert 2014

Commented:
Distinguished Expert 2017

Commented:
Caution should be taken.
Run pvdisplay to see whether the added space is seen.
pvresize can be used to stretch out the to use the additional space (pvresize /dev/cciss/c0d0p2)
vgdisplay should then show the added space as free space.
lvresize might be how you would stretch the logical partition to take more space within the volume group.

I think this is the right approach, but care should be taken in trying this as well as have full backup of your data just in case. If you have a test system with a small array that can be extended in a similar matter, you should test this process on it first.

Author

Commented:
pvdisplay shows only the 4TB:

  --- Physical volume ---
  PV Name               /dev/cciss/c0d0p1
  VG Name               VolGroup01
  PV Size               4.09 TB / not usable 12.58 MB
  Allocatable           yes
  PE Size (KByte)       32768
  Total PE              134132
  Free PE               1
  Allocated PE          134131
  PV UUID               E2sv8b-QswU-iyCZ-q73w-evh3-34bk-UPM7iu



How to Generate Services Revenue the Easiest Way

This Tuesday! Learn key insights about modern cyber protection services & gain practical strategies to skyrocket business:

- What it takes to build a cloud service portfolio
- How to determine which services will help your unique business grow
- Various use-cases and examples

Distinguished Expert 2017

Commented:
I think you need to run the pvresize to realize the additional space you added and seen by the OS.
pvresize /dev/cciss/c0d0p1

If you have a test system where you can test this, have a full backup prior to running the above.
Usually it should have no issues, but you never know.  Better to be prepared for the worst.

Author

Commented:
I've tried pvresize on everything that references that LV but i got the same result every time. I tried these three variations:
pvresize -v  -d  -t /dev/mapper/VolGroup01-LogVol00
pvresize -v  -d  -t /dev/cciss/c0d0
pvresize -v  -d  -t /dev/cciss/c0d0p2

and each one came back with
  Failed to read physical volume "/dev/cciss/c0d0p2" (or "/dev/mapper/VolGroup01-LogVol00", "/dev/cciss/c0d0" )
Distinguished Expert 2017

Commented:
if you use gnu parted, do you see the extra space on /dev/cciss/c0d0 for use if you want to define a new partition?

try using parted /dev/cciss/c0d0p2 resize.

You should try these on a different system or make sure you have a current backup of the data prior to running
parted /dev/cciss/c0d0 print
parted /dev/cciss/c0d0p1 print
parted /dev/cciss/c0d0/p2 print
parted /dev/cciss/c0d0p2 resize 2 1 max_cylinder_count_from_prior_prints

Author

Commented:
parted can't see/change that partition - but can see the partiton on the disk and correct total disk size:

parted /dev/cciss/c0d0 print:
Model: Compaq Smart Array (cpqarray)
Disk /dev/cciss/c0d0: 7501GB
Sector size (logical/physical): 512B/512B
Partition Table: gpt

Number  Start   End     Size    File system  Name  Flags
 1      17.4kB  4501GB  4501GB                     lvm


parted /dev/cciss/c0d0p1 print:
Error: /dev/cciss/c0d0p1: unrecognised disk label

parted /dev/cciss/c0d0p2 print
Error: Could not stat device /dev/cciss/c0d0p2 - No such file or directory.


Distinguished Expert 2017

Commented:
backup needed to test.
parted 1 resize 17.4kb 7501GB
 or something like that.
Alternatively you could create a second partition for the remaining 3000GB and then use lvextend to add this partition to the existing one.

Author

Commented:
# parted /dev/cciss/c0d0
GNU Parted 1.8.8
Using /dev/cciss/c0d0
Welcome to GNU Parted! Type 'help' to view a list of commands.
(parted) resize 1 17.4kB 7501GB
Error: Could not detect file system.

Already tried to make a partition in that space but this is what i get:
(parted) mkpart primary 4502GB 7501GB
Error: Unable to satisfy all constraints on the partition.

I've tried troubleshooting that error too but haven't been able to find anything that helps...

Distinguished Expert 2017

Commented:
Double check the new allocation in the HP insight that the space is being made available to the OS. Not sure what the issue is that is preventing the various tools, gparted and LVM tools from extending the use of the larger allocation.

Author

Commented:
The space IS available to the OS. Both fdisk and parted see the space out there but nothing can use it for some reason. There isn't a magic command in the HP stuff to show space to the OS and not allow it to use it...

Author

Commented:
Neither of those links helped me in any way...
Distinguished Expert 2017

Commented:
Try running pvresize -v -d -t /dev/VolGroup01

Author

Commented:
# pvresize -v -d -t /dev/VolGroup01/
  Test mode: Metadata will NOT be updated.
    Using physical volume(s) on command line
  Failed to read physical volume "/dev/VolGroup01/"
  0 physical volume(s) resized / 0 physical volume(s) not resized
    Test mode: Wiping internal cache
    Wiping internal VG cache

Author

Commented:
Here's some more info from sfdisk -l:
# sfdisk -l

WARNING: GPT (GUID Partition Table) detected on '/dev/cciss/c0d0'! The util sfdisk doesn't support GPT. Use GNU Parted.

Disk /dev/cciss/c0d0: 1795444 cylinders, 255 heads, 32 sectors/track
Warning: The partition table looks like it was made
  for C/H/S=*/255/63 (instead of 1795444/255/32).
For this listing I'll assume that geometry.
Units = cylinders of 8225280 bytes, blocks of 1024 bytes, counting from 0

   Device Boot Start     End   #cyls    #blocks   Id  System
/dev/cciss/c0d0p1          0+      0-      1-        16+  ee  GPT
                start: (c,h,s) expected (0,0,2) found (1023,254,63)
                end: (c,h,s) expected (0,0,34) found (1023,254,63)
/dev/cciss/c0d0p2   *      0+ 267349- 267350- 2147483647+  8e  Linux LVM
                start: (c,h,s) expected (0,0,35) found (1023,254,63)
/dev/cciss/c0d0p3          0       -       0          0    0  Empty
/dev/cciss/c0d0p4          0       -       0          0    0  Empty
Distinguished Expert 2017

Commented:
Did you use parted to create the initial LVM partition?
The various commands should have done what you wanted, but it is clear that something else is interfering and I am not sure what that could be.

Presumably a backup of the data on the array and then recreating it with the full size in use is not an option.

Is the OS you are running 64bit?

Author

Commented:
It's been a while, but i don't recall using parted to create it. ... but now that you mention it, i recall creating it when i setup the machine - and at that time i used the GUI to set it up... basically that boils down to 'I don't know'

I really don't want to tear it all don if i don't have to. That LVM is currently hosting a few live websites that I don't want to take down.

And yes the OS is 64bit
Distinguished Expert 2017

Commented:
Which fedora version are you using not sure if you have LVM or LVM2. Not sure if it even will make a difference.

Author

Commented:
Fedora 11
Distinguished Expert 2017

Commented:
Unfortunately, the above gparted/pvresize/lvextend/resize2fs etc should work in combination to streatch the physical,and logical volumes. Something that I am unclear on is preventing that.
Could post up the output of df -k?
http://www.howtoforge.com/linux_resizing_ext3_partitions_p2

Author

Commented:
# df -k
Filesystem           1K-blocks      Used Available Use% Mounted on
/dev/mapper/VolGroup00-LogVol01
                     233206488  11266268 209902896   6% /
/dev/mapper/VolGroup01-LogVol00
                     4257562032 1812803248 2224998556  45% /var/msa
/dev/cciss/c1d0p1       194442     23158    161245  13% /boot
tmpfs                  6616624         0   6616624   0% /dev/shm
/dev/sda1            961432072  55533120 857060952   7% /var/external/lacie1
/dev/sdb1            961432072    204628 912389444   1% /var/external/lacie2

Distinguished Expert 2017

Commented:
I suspect that the existence of the /boot partition on this volume is what might be causing the problem.
Does your system support internal drives?
What I am suggesting is to create a /boot copy to an internal drive and setup grub to be able to boot from either.  then attempt the prior ones while the boot is through the other drive to see whether that was preventing the physical expansion.

 
Distinguished Expert 2017

Commented:
I suspect that the existence of the /boot partition on this volume is what might be causing the problem.
Does your system support internal drives?
What I am suggesting is to create a /boot copy to an internal drive and setup grub to be able to boot from either.  then attempt the prior ones while the boot is through the other drive to see whether that was preventing the physical expansion.

 
Distinguished Expert 2017

Commented:
I suspect that the existence of the /boot partition on this volume is what might be causing the problem.
Does your system support internal drives?
What I am suggesting is to create a /boot copy to an internal drive and setup grub to be able to boot from either.  then attempt the prior ones while the boot is through the other drive to see whether that was preventing the physical expansion.

 

Author

Commented:
Actually THAT LV is working fine and can be expanded. That's just an array with 2 250gb disks. The LV that's the problem is the disk /dev/cciss/c0d0, not /dev/cciss/c1d0 (on which is the root partition: /dev/mapper/VolGroup00-LogVol01 and the boot partition: /dev/cciss/c1d0p1)
Distinguished Expert 2017
Commented:
In this case, I am not sure whether something in the manner in which the original 4TB storage was partitioned is the cause for the issue.
I.e. The GPT c0t0p1 might be holding all the cards preventing the expansion.
http://librenix.com/?page=Partition
Top Expert 2014
Commented:
That's why I referred to that other question in my post, it looks like something in the hardware array expansion throw some phantom GPT in. I think it's a problem with the ACU. It does naughties with Windows as well, trying to stretch partitions past 2TB when they're not using GPT.

Author

Commented:
Looks like i'm going to have to destory it all and remake it. Thanks for the ideas.

Author

Commented:
Not the solution i was hoping for, but there's only so much a person can do I guess.

Do more with

Expert Office
Submit tech questions to Ask the Experts™ at any time to receive solutions, advice, and new ideas from leading industry professionals.

Start 7-Day Free Trial