Want to protect your cyber security and still get fast solutions? Ask a secure question today.Go Premium

x
  • Status: Solved
  • Priority: Medium
  • Security: Public
  • Views: 1680
  • Last Modified:

RHEL 6 out of space

I have no more space in root, buts lots of space at home

[root@ex5b home]# df
Filesystem           1K-blocks      Used Available Use% Mounted on
/dev/mapper/vg_ex5b-lv_root
                      51606140  50650608         0 100% /
tmpfs                 66073880         0  66073880   0% /dev/shm
/dev/sda1               495844     60297    409947  13% /boot
/dev/mapper/vg_ex5b-lv_home
                     103986416    292588  98411628   1% /home

Open in new window


How can I move space from home to root

Linux does not have a chfs command

Thanks
0
Los Angeles1
Asked:
Los Angeles1
  • 7
  • 7
  • 2
  • +1
1 Solution
 
upanwarCommented:
You can check the logs they might be eating your systems space and you compress them or if not needed the remove them/

It might have some different issue so you can check which directory is eating more by issuing command.

# du -sh *

then you can go into that directory and remove unwanted files.

If you need further help, please update us.
0
 
wesly_chenCommented:
As root
# du -xsk /* | sort -nr
It will sort by the size of the directories under /.
Then you can cd to the biggest directory do
# du -xsk * | sort -nr
to find out what directory or files use the most of disk space.

Usually, /var and /tmp have a lot of temporary files, log files, which are ok to delete.

For shrink /home and extend /, please post the output of the following command.
# pvdisplay
# vgdisplay
# lvdisplay
0
 
Los Angeles1Author Commented:
Looks like I can take some space from the home LV and put it to the root lv, how can I do this ?

[root@ex5b /]# pvdisplay
  --- Physical volume ---
  PV Name               /dev/sda2
  VG Name               vg_ex5b
  PV Size               278.91 GiB / not usable 0
  Allocatable           yes (but full)
  PE Size               4.00 MiB
  Total PE              71400
  Free PE               0
  Allocated PE          71400
  PV UUID               w3rwHB-4mtU-0Ir2-Q3xe-2ZDJ-Uhvq-b4WgT4

[root@ex5b /]#
[root@ex5b /]# vgdisplay
  --- Volume group ---
  VG Name               vg_ex5b
  System ID
  Format                lvm2
  Metadata Areas        1
  Metadata Sequence No  4
  VG Access             read/write
  VG Status             resizable
  MAX LV                0
  Cur LV                3
  Open LV               3
  Max PV                0
  Cur PV                1
  Act PV                1
  VG Size               278.91 GiB
  PE Size               4.00 MiB
  Total PE              71400
  Alloc PE / Size       71400 / 278.91 GiB
  Free  PE / Size       0 / 0
  VG UUID               JhnNdh-9aKB-4YwE-xtTv-oLqZ-xUYS-wZAfFw

[root@ex5b /]#
[root@ex5b /]# lvdisplay
  --- Logical volume ---
  LV Name                /dev/vg_ex5b/lv_root
  VG Name                vg_ex5b
  LV UUID                yurcUj-lMPk-DPvL-Voo4-OEtS-Z92g-TPQURp
  LV Write Access        read/write
  LV Status              available
  # open                 1
  LV Size                50.00 GiB
  Current LE             12800
  Segments               1
  Allocation             inherit
  Read ahead sectors     auto
  - currently set to     256
  Block device           253:0

  --- Logical volume ---
  LV Name                /dev/vg_ex5b/lv_home
  VG Name                vg_ex5b
  LV UUID                4X2SCc-pTkY-x3mU-sdYY-6LF6-Z9xm-MD6wP0
  LV Write Access        read/write
  LV Status              available
  # open                 1
  LV Size                100.75 GiB
  Current LE             25792
  Segments               1
  Allocation             inherit
  Read ahead sectors     auto
  - currently set to     256
  Block device           253:2

  --- Logical volume ---
  LV Name                /dev/vg_ex5b/lv_swap
  VG Name                vg_ex5b
  LV UUID                I1zVVz-njSc-v4GB-r3VO-X0br-GmPg-PrAWtX
  LV Write Access        read/write
  LV Status              available
  # open                 1
  LV Size                128.16 GiB
  Current LE             32808
  Segments               1
  Allocation             inherit
  Read ahead sectors     auto
  - currently set to     256
  Block device           253:1

[root@ex5b /]# du -xsk /* | sort -nr
du: cannot access `/proc/6189/task/6189/fd/4': No such file or directory
du: cannot access `/proc/6189/task/6189/fdinfo/4': No such file or directory
du: cannot access `/proc/6189/fd/4': No such file or directory
du: cannot access `/proc/6189/fdinfo/4': No such file or directory
47460292        /var
1539524 /usr
1177784 /opt
206216  /lib
100492  /home
49832   /boot
29308   /lib64
25044   /etc
13604   /sbin
9044    /bin
5712    /root
244     /dev
20      /tmp
16      /lost+found
8       /mnt
4       /srv
4       /mp
4       /media
4       /cgroup
0       /sys
0       /selinux
0       /proc
0       /net
0       /misc
[root@ex5b /]# du -xsk * | sort -nr
du: cannot access `proc/6194/task/6194/fd/4': No such file or directory
du: cannot access `proc/6194/task/6194/fdinfo/4': No such file or directory
du: cannot access `proc/6194/fd/4': No such file or directory
du: cannot access `proc/6194/fdinfo/4': No such file or directory
47460292        var
1539524 usr
1177784 opt
206216  lib
100492  home
49832   boot
29308   lib64
25044   etc
13604   sbin
9044    bin
5712    root
244     dev
20      tmp
16      lost+found
8       mnt
4       srv
4       mp
4       media
4       cgroup
0       sys
0       selinux
0       proc
0       net
0       misc
[root@ex5b /]#

Open in new window




0
Independent Software Vendors: We Want Your Opinion

We value your feedback.

Take our survey and automatically be enter to win anyone of the following:
Yeti Cooler, Amazon eGift Card, and Movie eGift Card!

 
upanwarCommented:
Could you please show us.

# df -Th

and also update with how much size you want to reduce the home and increase the root so that we can suggest the steps.
0
 
Los Angeles1Author Commented:
I would likke to move half the size of home (50 GB) to root

[root@ex5b /]# df -Th
Filesystem    Type    Size  Used Avail Use% Mounted on
/dev/mapper/vg_ex5b-lv_root
              ext4     50G   49G     0 100% /
tmpfs        tmpfs     64G     0   64G   0% /dev/shm
/dev/sda1     ext4    485M   59M  401M  13% /boot
/dev/mapper/vg_ex5b-lv_home
              ext4    100G  286M   94G   1% /home

Open in new window

0
 
wesly_chenCommented:
> 47460292        var
You have 47GB on /var.
Do you have database (such as MySQL) installed in /var/lib/mysql ?

# du -xsk /var/* |sort -nr
0
 
wesly_chenCommented:
>   LV Name                /dev/vg_ex5b/lv_swap
>  LV Size                128.16 GiB

Woa, you have 128.16GB for swap.....

Anyway, you can
1. shrink  the filesystem on /dev/vg_ex5b/lv_home to 50GB
# resize2fs  /dev/vg_ex5b/lv_home 50G

2. shrink the logical volume  /dev/vg_ex5b/lv_home
# lvreduce -L 50G  /dev/vg_ex5b/lv_home
# resize2fs  /dev/vg_ex5b/lv_home

3. extend lv on /dev/vg_ex5b/lv_root
# lvextend -L 100G /dev/vg_ex5b/lv_root

4. Extend teh filesystem
# resize2fs /dev/vg_ex5b/lv_root 100G
0
 
Los Angeles1Author Commented:
I do not have any db's, but I do have an mq with a single qmgr, with lots of logging enabled

Previously I had several large qmgrs (with lots of logging enabled), but I deleted about 9 of them

[root@ex5b var]# du -xsk /var/* | sort -nr
44855736        /var/mqm
2067544 /var/log
361936  /var/mqsi
101108  /var/cache
71936   /var/lib
1800    /var/spool
204     /var/run
16      /var/lock
12      /var/yp
8       /var/empty
8       /var/db
4       /var/tmp
4       /var/report
4       /var/preserve
4       /var/opt
4       /var/nis
4       /var/local
4       /var/gdm
4       /var/games
4       /var/cvs
4       /var/crash
4       /var/account
0       /var/mail
[root@ex5b var]# dspmq
QMNAME(LQM3)                                              STATUS(Ended unexpectedly)
[root@ex5b var]#

Open in new window

0
 
Los Angeles1Author Commented:
The system does not allow me to do this

[root@ex5b var]# resize2fs  /dev/vg_ex5b/lv_home 50G
resize2fs 1.41.12 (17-May-2010)
Filesystem at /dev/vg_ex5b/lv_home is mounted on /home; on-line resizing required
On-line shrinking from 26411008 to 13107200 not supported.
[root@ex5b var]#

Open in new window

0
 
wesly_chenCommented:
Woo, forget to mentioned that you need to umount the /home first.
1. Directory login as root.
2. Make none else login
# who
3.
# umount /home

Then
1. shrink  the filesystem on /dev/vg_ex5b/lv_home to 50GB
# resize2fs  /dev/vg_ex5b/lv_home 50G

2. shrink the logical volume  /dev/vg_ex5b/lv_home
# lvreduce -L 50G  /dev/vg_ex5b/lv_home
# resize2fs  /dev/vg_ex5b/lv_home

3. extend lv on /dev/vg_ex5b/lv_root
# lvextend -L 100G /dev/vg_ex5b/lv_root

4. Extend teh filesystem
# resize2fs /dev/vg_ex5b/lv_root 100G
0
 
Los Angeles1Author Commented:
I performed the following, but ran into a problem

[root@ex5b ~]#
[root@ex5b ~]#
[root@ex5b ~]# umount /home
[root@ex5b ~]#
[root@ex5b ~]#
[root@ex5b ~]# resize2fs  /dev/vg_ex5b/lv_home 50G
resize2fs 1.41.12 (17-May-2010)
Please run 'e2fsck -f /dev/vg_ex5b/lv_home' first.

[root@ex5b ~]# e2fsck -f /dev/vg_ex5b/lv_home
e2fsck 1.41.12 (17-May-2010)
Pass 1: Checking inodes, blocks, and sizes
Pass 2: Checking directory structure
Pass 3: Checking directory connectivity
Pass 4: Checking reference counts
Pass 5: Checking group summary information
/dev/vg_ex5b/lv_home: 212/6602752 files (0.0% non-contiguous), 487551/26411008 blocks
[root@ex5b ~]# resize2fs  /dev/vg_ex5b/lv_home 50G
resize2fs 1.41.12 (17-May-2010)
Resizing the filesystem on /dev/vg_ex5b/lv_home to 13107200 (4k) blocks.
The filesystem on /dev/vg_ex5b/lv_home is now 13107200 blocks long.

[root@ex5b ~]# lvreduce -L 50G  /dev/vg_ex5b/lv_home
  WARNING: Reducing active logical volume to 50.00 GiB
  THIS MAY DESTROY YOUR DATA (filesystem etc.)
Do you really want to reduce lv_home? [y/n]: y
  Reducing logical volume lv_home to 50.00 GiB
  Limit for the maximum number of semaphores reached. You can check and set the limits in /proc/sys/kernel/sem.
  Unable to resume vg_ex5b-lv_home (253:2)
  Problem reactivating lv_home
[root@ex5b ~]# resize2fs  /dev/vg_ex5b/lv_home
resize2fs 1.41.12 (17-May-2010)

Open in new window



At this point, the resize2fs command has been running for well over an hour.  
It does not seem to be ending.  Seems to be in some sort of permanent loop.

On another putty session, I did the following:

[root@ex5b ~]# ps -ef | grep resize2fs
root      6535  6418  0 12:33 pts/0    00:00:00 resize2fs /dev/vg_ex5b/lv_home
root      6782  6744  0 14:01 pts/1    00:00:00 grep resize2fs
[root@ex5b ~]#

Open in new window



What should I do at this point ?

0
 
wesly_chenCommented:
> Limit for the maximum number of semaphores reached
As root,
# cat /proc/sys/kernel/sem
0
 
Los Angeles1Author Commented:
After the changes, it could not open th esession

[root@WEST5 ~]# cd /etc
[root@WEST5 etc]# cd security
[root@WEST5 security]# cat limits.conf
# /etc/security/limits.conf
#
#Each line describes a limit for a user in the form:
#
#<domain>        <type>  <item>  <value>
#
#Where:
#<domain> can be:
#        - an user name
#        - a group name, with @group syntax
#        - the wildcard *, for default entry
#        - the wildcard %, can be also used with %group syntax,
#                 for maxlogin limit
#
#<type> can have the two values:
#        - "soft" for enforcing the soft limits
#        - "hard" for enforcing hard limits
#
#<item> can be one of the following:
#        - core - limits the core file size (KB)
#        - data - max data size (KB)
#        - fsize - maximum filesize (KB)
#        - memlock - max locked-in-memory address space (KB)
#        - nofile - max number of open files
#        - rss - max resident set size (KB)
#        - stack - max stack size (KB)
#        - cpu - max CPU time (MIN)
#        - nproc - max number of processes
#        - as - address space limit (KB)
#        - maxlogins - max number of logins for this user
#        - maxsyslogins - max number of logins on the system
#        - priority - the priority to run user process with
#        - locks - max number of file locks the user can hold
#        - sigpending - max number of pending signals
#        - msgqueue - max memory used by POSIX message queues (bytes)
#        - nice - max nice priority allowed to raise to values: [-20, 19]
#        - rtprio - max realtime priority
#
#<domain>      <type>  <item>         <value>
#

#*               soft    core            0
#*               hard    rss             10000
#@student        hard    nproc           20
#@faculty        soft    nproc           20
#@faculty        hard    nproc           50
#ftp             hard    nproc           0
#@student        -       maxlogins       4

mqm    soft     nofile   10000
mqm    hard     nofile   unlimited
mqm    hard     nproc    4096

# End of file
[root@WEST5 security]#
[root@WEST5 security]# su mqm
could not open session
[root@WEST5 security]# su - mqm
could not open session
[root@WEST5 security]#

Open in new window





[root@WEST5 security]# ulimit -a
core file size          (blocks, -c) 0
data seg size           (kbytes, -d) unlimited
scheduling priority             (-e) 0
file size               (blocks, -f) unlimited
pending signals                 (-i) 1161386
max locked memory       (kbytes, -l) 64
max memory size         (kbytes, -m) unlimited
open files                      (-n) 1024
pipe size            (512 bytes, -p) 8
POSIX message queues     (bytes, -q) 819200
real-time priority              (-r) 0
stack size              (kbytes, -s) 10240
cpu time               (seconds, -t) unlimited
max user processes              (-u) 1024
virtual memory          (kbytes, -v) unlimited
file locks                      (-x) unlimited
[root@WEST5 security]#
[root@WEST5 security]# ulimit -Ha
core file size          (blocks, -c) unlimited
data seg size           (kbytes, -d) unlimited
scheduling priority             (-e) 0
file size               (blocks, -f) unlimited
pending signals                 (-i) 1161386
max locked memory       (kbytes, -l) 64
max memory size         (kbytes, -m) unlimited
open files                      (-n) 1024
pipe size            (512 bytes, -p) 8
POSIX message queues     (bytes, -q) 819200
real-time priority              (-r) 0
stack size              (kbytes, -s) unlimited
cpu time               (seconds, -t) unlimited
max user processes              (-u) 1161386
virtual memory          (kbytes, -v) unlimited
file locks                      (-x) unlimited

Open in new window

0
 
wesly_chenCommented:
It seems the wrong post.
Please post
# cat /proc/sys/kernel/sem
0
 
haroldpgaCommented:
You've already gotten plenty of advice on how to remove the extra space in /home.  My advice is to not immediately put all of it into /.  Leave some of this space un-allocated.  That way, if you run out of space in /home later you can put a little of it back.

When I build systems, I usually create two physical partitions on the disk.  The first is a small file system for /boot, and the rest of the disk goes into a Logical Volume Manager partition.  Everything else goes into this partition, managed by the LVM.  This includes file systems for swap, /, /usr, /var, /tmp, /home and any other needed systems like /opt or /pub.  I also usually put /usr/local and /var/www into their own file systems as well.  I try to size all of these systems so they start off life between 50 and 75 percent full.  Until you get comfortable, this may take installing the system twice.  Once to see how big each file system is, and a second time to create them all the right size.
0
 
Los Angeles1Author Commented:
wesly_chen:

[root@ex5b kernel]# pwd
/proc/sys/kernel
[root@ex5b kernel]# cat sem
250     32000   32      128
[root@ex5b kernel]#

Open in new window


haroldpga:

You are quite right, I have gotten plenty of excellent advice, my problem is I do not quite know how to change my file system yet.
0
 
wesly_chenCommented:
The error message you got is
>  Limit for the maximum number of semaphores reached. You can check and set the limits in /proc/sys/kernel/sem.

So increase the sem to resolve the issue as it suggested.
(as root)
# echo 500 512000 64 2048 > /proc/sys/kernel/sem
This is temporary and the value will be reset to original once the system rebooted.
0

Featured Post

Independent Software Vendors: We Want Your Opinion

We value your feedback.

Take our survey and automatically be enter to win anyone of the following:
Yeti Cooler, Amazon eGift Card, and Movie eGift Card!

  • 7
  • 7
  • 2
  • +1
Tackle projects and never again get stuck behind a technical roadblock.
Join Now