c_hockland
asked on
Linux and space issues - Cannot find what is consuming the space
hi all
We have been having space issues since the weekend and something continues to write to /u01, but I can not find what it is.
I noticed you still logged on the server so hopefully you may have an idea of what is consuming the space. See output below
-- there are 154G allocated to the FS and 'df -k' shows that only 26.8G are available, However, the 'du -sh *' do not show what is taking the space.
pwd
/u01
du -sh *
25G app
1.9G home
16K lost+found
# df -k .
Filesystem 1K-blocks Used Available Use% Mounted on
/dev/mapper/VolGroup00-u01 lv
154818540 120154656 26800204 82% /u01
pwd
/u01
any input would be appreciated.
We have been having space issues since the weekend and something continues to write to /u01, but I can not find what it is.
I noticed you still logged on the server so hopefully you may have an idea of what is consuming the space. See output below
-- there are 154G allocated to the FS and 'df -k' shows that only 26.8G are available, However, the 'du -sh *' do not show what is taking the space.
pwd
/u01
du -sh *
25G app
1.9G home
16K lost+found
# df -k .
Filesystem 1K-blocks Used Available Use% Mounted on
/dev/mapper/VolGroup00-u01
154818540 120154656 26800204 82% /u01
pwd
/u01
any input would be appreciated.
ASKER
Hi Gerwin
plz see below
69 zabbix 1w REG 252,5 660 14 /tmp/zabbix_agentd.log (deleted)
zabbix_ag 12269 zabbix 2w REG 252,5 660 14 /tmp/zabbix_agentd.log (deleted)
zabbix_ag 12269 zabbix 3wW REG 252,5 5 17 /tmp/zabbix_agentd.pid (deleted)
zabbix_ag 12271 zabbix txt REG 252,0 166092 917577 /sbin/zabbix_agentd (deleted)
zabbix_ag 12271 zabbix 1w REG 252,5 660 14 /tmp/zabbix_agentd.log (deleted)
zabbix_ag 12271 zabbix 2w REG 252,5 660 14 /tmp/zabbix_agentd.log (deleted)
zabbix_ag 12271 zabbix 3w REG 252,5 5 17 /tmp/zabbix_agentd.pid (deleted)
zabbix_ag 12272 zabbix txt REG 252,0 166092 917577 /sbin/zabbix_agentd (deleted)
zabbix_ag 12272 zabbix 1w REG 252,5 660 14 /tmp/zabbix_agentd.log (deleted)
zabbix_ag 12272 zabbix 2w REG 252,5 660 14 /tmp/zabbix_agentd.log (deleted)
zabbix_ag 12272 zabbix 3w REG 252,5 5 17 /tmp/zabbix_agentd.pid (deleted)
zabbix_ag 12273 zabbix txt REG 252,0 166092 917577 /sbin/zabbix_agentd (deleted)
zabbix_ag 12273 zabbix 1w REG 252,5 660 14 /tmp/zabbix_agentd.log (deleted)
zabbix_ag 12273 zabbix 2w REG 252,5 660 14 /tmp/zabbix_agentd.log (deleted)
zabbix_ag 12273 zabbix 3w REG 252,5 5 17 /tmp/zabbix_agentd.pid (deleted)
zabbix_ag 12274 zabbix txt REG 252,0 166092 917577 /sbin/zabbix_agentd (deleted)
zabbix_ag 12274 zabbix 1w REG 252,5 660 14 /tmp/zabbix_agentd.log (deleted)
zabbix_ag 12274 zabbix 2w REG 252,5 660 14 /tmp/zabbix_agentd.log (deleted)
zabbix_ag 12274 zabbix 3w REG 252,5 5 17 /tmp/zabbix_agentd.pid (deleted)
zabbix_ag 12275 zabbix txt REG 252,0 166092 917577 /sbin/zabbix_agentd (deleted)
zabbix_ag 12275 zabbix 1w REG 252,5 660 14 /tmp/zabbix_agentd.log (deleted)
zabbix_ag 12275 zabbix 2w REG 252,5 660 14 /tmp/zabbix_agent
plz see below
69 zabbix 1w REG 252,5 660 14 /tmp/zabbix_agentd.log (deleted)
zabbix_ag 12269 zabbix 2w REG 252,5 660 14 /tmp/zabbix_agentd.log (deleted)
zabbix_ag 12269 zabbix 3wW REG 252,5 5 17 /tmp/zabbix_agentd.pid (deleted)
zabbix_ag 12271 zabbix txt REG 252,0 166092 917577 /sbin/zabbix_agentd (deleted)
zabbix_ag 12271 zabbix 1w REG 252,5 660 14 /tmp/zabbix_agentd.log (deleted)
zabbix_ag 12271 zabbix 2w REG 252,5 660 14 /tmp/zabbix_agentd.log (deleted)
zabbix_ag 12271 zabbix 3w REG 252,5 5 17 /tmp/zabbix_agentd.pid (deleted)
zabbix_ag 12272 zabbix txt REG 252,0 166092 917577 /sbin/zabbix_agentd (deleted)
zabbix_ag 12272 zabbix 1w REG 252,5 660 14 /tmp/zabbix_agentd.log (deleted)
zabbix_ag 12272 zabbix 2w REG 252,5 660 14 /tmp/zabbix_agentd.log (deleted)
zabbix_ag 12272 zabbix 3w REG 252,5 5 17 /tmp/zabbix_agentd.pid (deleted)
zabbix_ag 12273 zabbix txt REG 252,0 166092 917577 /sbin/zabbix_agentd (deleted)
zabbix_ag 12273 zabbix 1w REG 252,5 660 14 /tmp/zabbix_agentd.log (deleted)
zabbix_ag 12273 zabbix 2w REG 252,5 660 14 /tmp/zabbix_agentd.log (deleted)
zabbix_ag 12273 zabbix 3w REG 252,5 5 17 /tmp/zabbix_agentd.pid (deleted)
zabbix_ag 12274 zabbix txt REG 252,0 166092 917577 /sbin/zabbix_agentd (deleted)
zabbix_ag 12274 zabbix 1w REG 252,5 660 14 /tmp/zabbix_agentd.log (deleted)
zabbix_ag 12274 zabbix 2w REG 252,5 660 14 /tmp/zabbix_agentd.log (deleted)
zabbix_ag 12274 zabbix 3w REG 252,5 5 17 /tmp/zabbix_agentd.pid (deleted)
zabbix_ag 12275 zabbix txt REG 252,0 166092 917577 /sbin/zabbix_agentd (deleted)
zabbix_ag 12275 zabbix 1w REG 252,5 660 14 /tmp/zabbix_agentd.log (deleted)
zabbix_ag 12275 zabbix 2w REG 252,5 660 14 /tmp/zabbix_agent
Those files are from the zabbix monitoring agent. Was there a zabbix update on the system maybe? Can you stop/start the zabbix agent on that system?
ASKER
i am so sorry , here is the complete output
# /usr/sbin/lsof | grep deleted
sh 4485 oracle 1w REG 252,7 95473516846 7599707 /u01/app/oracle/admin/bin/ nohup.out (deleted)
sh 4485 oracle 2w REG 252,7 95473516855 7599707 /u01/app/oracle/admin/bin/ nohup.out (deleted)
expdp 4619 oracle 1w REG 252,7 95473518970 7599707 /u01/app/oracle/admin/bin/ nohup.out (deleted)
expdp 4619 oracle 2w REG 252,7 95473518979 7599707 /u01/app/oracle/admin/bin/ nohup.out (deleted)
hald 8282 haldaemon 13w REG 252,1 0 884771 /var/lock/invcol.tmp (deleted)
hald 8282 haldaemon 14w REG 252,5 384 622829 /tmp/6967_26747/icredir.tx t (deleted)
hald 8282 haldaemon 17w REG 252,5 405 622831 /tmp/6967_26747/PowerSuppl y/inventor y.xml (deleted)
zabbix_ag 12269 zabbix txt REG 252,0 166092 917577 /sbin/zabbix_agentd (deleted)
zabbix_ag 12269 zabbix 1w REG 252,5 660 14 /tmp/zabbix_agentd.log (deleted)
zabbix_ag 12269 zabbix 2w REG 252,5 660 14 /tmp/zabbix_agentd.log (deleted)
zabbix_ag 12269 zabbix 3wW REG 252,5 5 17 /tmp/zabbix_agentd.pid (deleted)
zabbix_ag 12271 zabbix txt REG 252,0 166092 917577 /sbin/zabbix_agentd (deleted)
zabbix_ag 12271 zabbix 1w REG 252,5 660 14 /tmp/zabbix_agentd.log (deleted)
zabbix_ag 12271 zabbix 2w REG 252,5 660 14 /tmp/zabbix_agentd.log (deleted)
zabbix_ag 12271 zabbix 3w REG 252,5 5 17 /tmp/zabbix_agentd.pid (deleted)
zabbix_ag 12272 zabbix txt REG 252,0 166092 917577 /sbin/zabbix_agentd (deleted)
zabbix_ag 12272 zabbix 1w REG 252,5 660 14 /tmp/zabbix_agentd.log (deleted)
zabbix_ag 12272 zabbix 2w REG 252,5 660 14 /tmp/zabbix_agentd.log (deleted)
zabbix_ag 12272 zabbix 3w REG 252,5 5 17 /tmp/zabbix_agentd.pid (deleted)
zabbix_ag 12273 zabbix txt REG 252,0 166092 917577 /sbin/zabbix_agentd (deleted)
zabbix_ag 12273 zabbix 1w REG 252,5 660 14 /tmp/zabbix_agentd.log (deleted)
zabbix_ag 12273 zabbix 2w REG 252,5 660 14 /tmp/zabbix_agentd.log (deleted)
zabbix_ag 12273 zabbix 3w REG 252,5 5 17 /tmp/zabbix_agentd.pid (deleted)
zabbix_ag 12274 zabbix txt REG 252,0 166092 917577 /sbin/zabbix_agentd (deleted)
zabbix_ag 12274 zabbix 1w REG 252,5 660 14 /tmp/zabbix_agentd.log (deleted)
zabbix_ag 12274 zabbix 2w REG 252,5 660 14 /tmp/zabbix_agentd.log (deleted)
zabbix_ag 12274 zabbix 3w REG 252,5 5 17 /tmp/zabbix_agentd.pid (deleted)
zabbix_ag 12275 zabbix txt REG 252,0 166092 917577 /sbin/zabbix_agentd (deleted)
zabbix_ag 12275 zabbix 1w REG 252,5 660 14 /tmp/zabbix_agentd.log (deleted)
zabbix_ag 12275 zabbix 2w REG 252,5 660 14 /tmp/zabbix_agentd.log (deleted)
zabbix_ag 12275 zabbix 3w REG 252,5 5 17 /tmp/zabbix_agentd.pid (deleted)
xfs 12467 xfs txt REG 252,3 113992 1441847 /usr/bin/xfs (deleted)
nsrexecd 12502 root txt REG 252,3 1450200 131501 /usr/sbin/nsrexecd (deleted)
ohasd.bin 12918 oracle 18uw REG 252,1 0 1867780 /var/tmp/.oracle/sprocr_lo cal_conn_0 _PROL_lock (deleted)
ohasd.bin 12918 oracle 23w REG 252,7 108656 7979175 /u01/app/grid/product/11.2 .0/log/beb ma397/aler tbebma397. log (deleted)
yum-updat 13655 root txt REG 252,3 8304 1442624 /usr/bin/python.#prelink# (deleted)
gam_serve 13660 root txt REG 252,3 107912 1048598 /usr/libexec/gam_server (deleted)
mingetty 14553 root txt REG 252,0 16872 917555 /sbin/mingetty (deleted)
mingetty 14554 root txt REG 252,0 16872 917555 /sbin/mingetty (deleted)
mingetty 14555 root txt REG 252,0 16872 917555 /sbin/mingetty (deleted)
mingetty 14556 root txt REG 252,0 16872 917555 /sbin/mingetty (deleted)
mingetty 14557 root txt REG 252,0 16872 917555 /sbin/mingetty (deleted)
mingetty 14558 root txt REG 252,0 16872 917555 /sbin/mingetty (deleted)
init.ohas 14559 root txt REG 252,0 801816 229409 /bin/bash (deleted)
diskmon.b 15959 oracle 4w REG 252,7 34576084 6553608 /u01/app/grid/product/11.2 .0/log/beb ma397/disk mon/diskmo n.log (deleted)
ocssd.bin 16020 oracle 5w REG 252,7 108656 7979175 /u01/app/grid/product/11.2 .0/log/beb ma397/aler tbebma397. log (deleted)
ocssd.bin 16020 oracle 31uw REG 252,1 0 1867795 /var/tmp/.oracle/sOCSSD_LL _bebma397_ _lock (deleted)
ocssd.bin 16020 oracle 33uw REG 252,1 0 1867796 /var/tmp/.oracle/sOracle_C SS_LclLstn r_localhos t_1_lock (deleted)
ocssd.bin 16020 oracle 44uw REG 252,1 0 1867797 /var/tmp/.oracle/sOCSSD_LL _bebma397_ localhost_ lock (deleted)
oraagent. 16792 oracle 12w REG 252,7 108656 7979175 /u01/app/grid/product/11.2 .0/log/beb ma397/aler tbebma397. log (deleted)
evmd.bin 16870 oracle 18w REG 252,7 108656 7979175 /u01/app/grid/product/11.2 .0/log/beb ma397/aler tbebma397. log (deleted)
oracle 17160 oracle 4w REG 252,7 10493324 7864900 /u01/app/grid/product/11.2 .0/log/beb ma397/agen t/ohasd/or aagent_ora cle/oraage nt_oracle. l10 (deleted)
oracle 17160 oracle 12w REG 252,7 108656 7979175 /u01/app/grid/product/11.2 .0/log/beb ma397/aler tbebma397. log (deleted)
oracle 17162 oracle 4w REG 252,7 10493324 7864900 /u01/app/grid/product/11.2 .0/log/beb ma397/agen t/ohasd/or aagent_ora cle/oraage nt_oracle. l10 (deleted)
oracle 17162 oracle 12w REG 252,7 108656 7979175 /u01/app/grid/product/11.2 .0/log/beb ma397/aler tbebma397. log (deleted)
oracle 17185 oracle 4w REG 252,7 10493324 7864900 /u01/app/grid/product/11.2 .0/log/beb ma397/agen t/ohasd/or aagent_ora cle/oraage nt_oracle. l10 (deleted)
oracle 17185 oracle 12w REG 252,7 108656 7979175 /u01/app/grid/product/11.2 .0/log/beb ma397/aler tbebma397. log (deleted)
oracle 17189 oracle 4w REG 252,7 10493324 7864900 /u01/app/grid/product/11.2 .0/log/beb ma397/agen t/ohasd/or aagent_ora cle/oraage nt_oracle. l10 (deleted)
oracle 17189 oracle 12w REG 252,7 108656 7979175 /u01/app/grid/product/11.2 .0/log/beb ma397/aler tbebma397. log (deleted)
oracle 17191 oracle 4w REG 252,7 10493324 7864900 /u01/app/grid/product/11.2 .0/log/beb ma397/agen t/ohasd/or aagent_ora cle/oraage nt_oracle. l10 (deleted)
oracle 17191 oracle 12w REG 252,7 108656 7979175 /u01/app/grid/product/11.2 .0/log/beb ma397/aler tbebma397. log (deleted)
oracle 17193 oracle 4w REG 252,7 10493324 7864900 /u01/app/grid/product/11.2 .0/log/beb ma397/agen t/ohasd/or aagent_ora cle/oraage nt_oracle. l10 (deleted)
oracle 17193 oracle 12w REG 252,7 108656 7979175 /u01/app/grid/product/11.2 .0/log/beb ma397/aler tbebma397. log (deleted)
oracle 17195 oracle 4w REG 252,7 10493324 7864900 /u01/app/grid/product/11.2 .0/log/beb ma397/agen t/ohasd/or aagent_ora cle/oraage nt_oracle. l10 (deleted)
oracle 17195 oracle 12w REG 252,7 108656 7979175 /u01/app/grid/product/11.2 .0/log/beb ma397/aler tbebma397. log (deleted)
oracle 17197 oracle 4w REG 252,7 10493324 7864900 /u01/app/grid/product/11.2 .0/log/beb ma397/agen t/ohasd/or aagent_ora cle/oraage nt_oracle. l10 (deleted)
oracle 17197 oracle 12w REG 252,7 108656 7979175 /u01/app/grid/product/11.2 .0/log/beb ma397/aler tbebma397. log (deleted)
oracle 17199 oracle 4w REG 252,7 10493324 7864900 /u01/app/grid/product/11.2 .0/log/beb ma397/agen t/ohasd/or aagent_ora cle/oraage nt_oracle. l10 (deleted)
oracle 17199 oracle 12w REG 252,7 108656 7979175 /u01/app/grid/product/11.2 .0/log/beb ma397/aler tbebma397. log (deleted)
oracle 17201 oracle 4w REG 252,7 10493324 7864900 /u01/app/grid/product/11.2 .0/log/beb ma397/agen t/ohasd/or aagent_ora cle/oraage nt_oracle. l10 (deleted)
oracle 17201 oracle 12w REG 252,7 108656 7979175 /u01/app/grid/product/11.2 .0/log/beb ma397/aler tbebma397. log (deleted)
oracle 17203 oracle 4w REG 252,7 10493324 7864900 /u01/app/grid/product/11.2 .0/log/beb ma397/agen t/ohasd/or aagent_ora cle/oraage nt_oracle. l10 (deleted)
oracle 17203 oracle 12w REG 252,7 108656 7979175 /u01/app/grid/product/11.2 .0/log/beb ma397/aler tbebma397. log (deleted)
oracle 17205 oracle 4w REG 252,7 10493324 7864900 /u01/app/grid/product/11.2 .0/log/beb ma397/agen t/ohasd/or aagent_ora cle/oraage nt_oracle. l10 (deleted)
oracle 17205 oracle 12w REG 252,7 108656 7979175 /u01/app/grid/product/11.2 .0/log/beb ma397/aler tbebma397. log (deleted)
oracle 17207 oracle 4w REG 252,7 10493324 7864900 /u01/app/grid/product/11.2 .0/log/beb ma397/agen t/ohasd/or aagent_ora cle/oraage nt_oracle. l10 (deleted)
oracle 17207 oracle 12w REG 252,7 108656 7979175 /u01/app/grid/product/11.2 .0/log/beb ma397/aler tbebma397. log (deleted)
oracle 17209 oracle 4w REG 252,7 10493324 7864900 /u01/app/grid/product/11.2 .0/log/beb ma397/agen t/ohasd/or aagent_ora cle/oraage nt_oracle. l10 (deleted)
oracle 17209 oracle 12w REG 252,7 108656 7979175 /u01/app/grid/product/11.2 .0/log/beb ma397/aler tbebma397. log (deleted)
oracle 17211 oracle 4w REG 252,7 10493324 7864900 /u01/app/grid/product/11.2 .0/log/beb ma397/agen t/ohasd/or aagent_ora cle/oraage nt_oracle. l10 (deleted)
oracle 17211 oracle 12w REG 252,7 108656 7979175 /u01/app/grid/product/11.2 .0/log/beb ma397/aler tbebma397. log (deleted)
oracle 17505 oracle 4w REG 252,7 10493324 7864900 /u01/app/grid/product/11.2 .0/log/beb ma397/agen t/ohasd/or aagent_ora cle/oraage nt_oracle. l10 (deleted)
oracle 17505 oracle 12w REG 252,7 108656 7979175 /u01/app/grid/product/11.2 .0/log/beb ma397/aler tbebma397. log (deleted)
oracle 17505 oracle 15u REG 252,7 751 10381178 /u01/app/grid/product/11.2 .0/rdbms/a udit/+asm_ ora_17505_ 4a.aud (deleted)
oracle 17508 oracle 4w REG 252,7 10493324 7864900 /u01/app/grid/product/11.2 .0/log/beb ma397/agen t/ohasd/or aagent_ora cle/oraage nt_oracle. l10 (deleted)
oracle 17508 oracle 12w REG 252,7 108656 7979175 /u01/app/grid/product/11.2 .0/log/beb ma397/aler tbebma397. log (deleted)
oracle 17508 oracle 15u REG 252,7 751 10381181 /u01/app/grid/product/11.2 .0/rdbms/a udit/+asm_ ora_17508_ 4e.aud (deleted)
oracle 17512 oracle 4w REG 252,7 10493324 7864900 /u01/app/grid/product/11.2 .0/log/beb ma397/agen t/ohasd/or aagent_ora cle/oraage nt_oracle. l10 (deleted)
oracle 17512 oracle 12w REG 252,7 108656 7979175 /u01/app/grid/product/11.2 .0/log/beb ma397/aler tbebma397. log (deleted)
oracle 17512 oracle 15u REG 252,7 751 10381180 /u01/app/grid/product/11.2 .0/rdbms/a udit/+asm_ ora_17512_ 4f.aud (deleted)
rman 19907 oracle 0r REG 252,5 0 41 /tmp/sh-thd-1430355787 (deleted)
oracle 20636 oracle 22w REG 252,7 891 4932230 /u01/app/oracle/admin/SMP1 /diagnosti c/diag/rdb ms/smp1/SM P1/trace/S MP1_asmb_2 0636.trc (deleted)
oracle 20636 oracle 23w REG 252,7 60 4932231 /u01/app/oracle/admin/SMP1 /diagnosti c/diag/rdb ms/smp1/SM P1/trace/S MP1_asmb_2 0636.trm (deleted)
oracle 20646 oracle 27u REG 252,7 751 10381200 /u01/app/grid/product/11.2 .0/rdbms/a udit/+asm_ ora_20646_ 59.aud (deleted)
oracle 21064 oracle 22w REG 252,7 896 1950168 /u01/app/oracle/admin/OMPP 1/diagnost ic/diag/rd bms/ompp1/ OMPP1/trac e/OMPP1_as mb_21064.t rc (deleted)
oracle 21064 oracle 23w REG 252,7 61 1950169 /u01/app/oracle/admin/OMPP 1/diagnost ic/diag/rd bms/ompp1/ OMPP1/trac e/OMPP1_as mb_21064.t rm (deleted)
oracle 21068 oracle 27u REG 252,7 751 10381205 /u01/app/grid/product/11.2 .0/rdbms/a udit/+asm_ ora_21068_ 56.aud (deleted)
oracle 21471 oracle 22w REG 252,7 896 7389709 /u01/app/oracle/admin/AVBE 1/diagnost ic/diag/rd bms/avbe1/ AVBE1/trac e/AVBE1_as mb_21471.t rc (deleted)
oracle 21471 oracle 23w REG 252,7 61 7389710 /u01/app/oracle/admin/AVBE 1/diagnost ic/diag/rd bms/avbe1/ AVBE1/trac e/AVBE1_as mb_21471.t rm (deleted)
oracle 21477 oracle 27u REG 252,7 751 10381211 /u01/app/grid/product/11.2 .0/rdbms/a udit/+asm_ ora_21477_ 43.aud (deleted)
oracle 21767 oracle 22w REG 252,7 891 1246279 /u01/app/oracle/admin/MPBE /diagnosti c/diag/rdb ms/mpbe/MP BE/trace/M PBE_asmb_2 1767.trc (deleted)
oracle 21767 oracle 23w REG 252,7 60 1246280 /u01/app/oracle/admin/MPBE /diagnosti c/diag/rdb ms/mpbe/MP BE/trace/M PBE_asmb_2 1767.trm (deleted)
oracle 21771 oracle 27u REG 252,7 751 10381217 /u01/app/grid/product/11.2 .0/rdbms/a udit/+asm_ ora_21771_ 45.aud (deleted)
oracle 22047 oracle 22w REG 252,7 891 557541 /u01/app/oracle/admin/FMBE /diagnosti c/diag/rdb ms/fmbe/FM BE/trace/F MBE_asmb_2 2047.trc (deleted)
oracle 22047 oracle 23w REG 252,7 60 557542 /u01/app/oracle/admin/FMBE /diagnosti c/diag/rdb ms/fmbe/FM BE/trace/F MBE_asmb_2 2047.trm (deleted)
oracle 22051 oracle 27u REG 252,7 751 10381222 /u01/app/grid/product/11.2 .0/rdbms/a udit/+asm_ ora_22051_ 4c.aud (deleted)
tail 22192 root 0u CHR 136,0 0t0 3 /dev/pts/0 (deleted)
tail 22192 root 1u CHR 136,0 0t0 3 /dev/pts/0 (deleted)
tail 22192 root 2u CHR 136,0 0t0 3 /dev/pts/0 (deleted)
tail 22192 root 3r REG 252,1 584218 1278295 /var/log/secure.1 (deleted)
oracle 22282 oracle 17w REG 252,7 1353 5605700 /u01/app/oracle/admin/OBBE /diagnosti c/diag/rdb ms/obbe/OB BE/trace/O BBE_diag_2 2282.trc (deleted)
oracle 22282 oracle 18w REG 252,7 80 5605701 /u01/app/oracle/admin/OBBE /diagnosti c/diag/rdb ms/obbe/OB BE/trace/O BBE_diag_2 2282.trm (deleted)
rman 22283 oracle 0r REG 252,5 0 35 /tmp/sh-thd-1430336644 (deleted)
oracle 22306 oracle 22w REG 252,7 891 5606210 /u01/app/oracle/admin/OBBE /diagnosti c/diag/rdb ms/obbe/OB BE/trace/O BBE_asmb_2 2306.trc (deleted)
oracle 22306 oracle 23w REG 252,7 60 5606217 /u01/app/oracle/admin/OBBE /diagnosti c/diag/rdb ms/obbe/OB BE/trace/O BBE_asmb_2 2306.trm (deleted)
oracle 22310 oracle 27u REG 252,7 751 10381227 /u01/app/grid/product/11.2 .0/rdbms/a udit/+asm_ ora_22310_ 48.aud (deleted)
oracle 22562 oracle 17w REG 252,7 1352 3817604 /u01/app/oracle/admin/VISI /diagnosti c/diag/rdb ms/visi/VI SI/trace/V ISI_diag_2 2562.trc (deleted)
oracle 22562 oracle 18w REG 252,7 90 3817605 /u01/app/oracle/admin/VISI /diagnosti c/diag/rdb ms/visi/VI SI/trace/V ISI_diag_2 2562.trm (deleted)
oracle 22584 oracle 22w REG 252,7 891 12731257 /u01/app/oracle/admin/VISI /diagnosti c/diag/rdb ms/visi/VI SI/trace/V ISI_asmb_2 2584.trc (deleted)
oracle 22584 oracle 23w REG 252,7 60 12731258 /u01/app/oracle/admin/VISI /diagnosti c/diag/rdb ms/visi/VI SI/trace/V ISI_asmb_2 2584.trm (deleted)
oracle 22588 oracle 27u REG 252,7 751 10381232 /u01/app/grid/product/11.2 .0/rdbms/a udit/+asm_ ora_22588_ 54.aud (deleted)
oracle 22936 oracle 22w REG 252,7 896 1246351 /u01/app/oracle/admin/UCMP 1/diagnost ic/diag/rd bms/ucmp1/ UCMP1/trac e/UCMP1_as mb_22936.t rc (deleted)
oracle 22936 oracle 23w REG 252,7 61 1246352 /u01/app/oracle/admin/UCMP 1/diagnost ic/diag/rd bms/ucmp1/ UCMP1/trac e/UCMP1_as mb_22936.t rm (deleted)
oracle 22940 oracle 27u REG 252,7 751 10381237 /u01/app/grid/product/11.2 .0/rdbms/a udit/+asm_ ora_22940_ 5d.aud (deleted)
oracle 23338 oracle 22w REG 252,7 906 5882350 /u01/app/oracle/admin/EREP OP1/diagno stic/diag/ rdbms/erep op1/EREPOP 1/trace/ER EPOP1_asmb _23338.trc (deleted)
oracle 23338 oracle 23w REG 252,7 72 5882351 /u01/app/oracle/admin/EREP OP1/diagno stic/diag/ rdbms/erep op1/EREPOP 1/trace/ER EPOP1_asmb _23338.trm (deleted)
oracle 23342 oracle 27u REG 252,7 751 10381242 /u01/app/grid/product/11.2 .0/rdbms/a udit/+asm_ ora_23342_ 52.aud (deleted)
oracle 24321 oracle 22w REG 252,7 891 4456846 /u01/app/oracle/admin/WBBE /diagnosti c/diag/rdb ms/wbbe/WB BE/trace/W BBE_asmb_2 4321.trc (deleted)
oracle 24321 oracle 23w REG 252,7 69 4456949 /u01/app/oracle/admin/WBBE /diagnosti c/diag/rdb ms/wbbe/WB BE/trace/W BBE_asmb_2 4321.trm (deleted)
oracle 24325 oracle 27u REG 252,7 751 10381253 /u01/app/grid/product/11.2 .0/rdbms/a udit/+asm_ ora_24325_ 57.aud (deleted)
oracle 24920 oracle 22w REG 252,7 901 4538880 /u01/app/oracle/admin/PVEX P1/diagnos tic/diag/r dbms/pvexp 1/PVEXP1/t race/PVEXP 1_asmb_249 20.trc (deleted)
oracle 24920 oracle 23w REG 252,7 62 4538881 /u01/app/oracle/admin/PVEX P1/diagnos tic/diag/r dbms/pvexp 1/PVEXP1/t race/PVEXP 1_asmb_249 20.trm (deleted)
oracle 24926 oracle 27u REG 252,7 751 10381258 /u01/app/grid/product/11.2 .0/rdbms/a udit/+asm_ ora_24926_ 67.aud (deleted)
sudo 31789 root 0u CHR 136,2 0t0 5 /dev/pts/2 (deleted)
sudo 31789 root 1u CHR 136,2 0t0 5 /dev/pts/2 (deleted)
sudo 31789 root 2u CHR 136,2 0t0 5 /dev/pts/2 (deleted)
su 31791 root 0u CHR 136,2 0t0 5 /dev/pts/2 (deleted)
su 31791 root 1u CHR 136,2 0t0 5 /dev/pts/2 (deleted)
su 31791 root 2u CHR 136,2 0t0 5 /dev/pts/2 (deleted)
bash 31792 oracle 0u CHR 136,2 0t0 5 /dev/pts/2 (deleted)
bash 31792 oracle 1u CHR 136,2 0t0 5 /dev/pts/2 (deleted)
bash 31792 oracle 2u CHR 136,2 0t0 5 /dev/pts/2 (deleted)
bash 31792 oracle 255u CHR 136,2 0t0 5 /dev/pts/2 (deleted)
#
shall i still restart zabbix-agent ?
# /usr/sbin/lsof | grep deleted
sh 4485 oracle 1w REG 252,7 95473516846 7599707 /u01/app/oracle/admin/bin/
sh 4485 oracle 2w REG 252,7 95473516855 7599707 /u01/app/oracle/admin/bin/
expdp 4619 oracle 1w REG 252,7 95473518970 7599707 /u01/app/oracle/admin/bin/
expdp 4619 oracle 2w REG 252,7 95473518979 7599707 /u01/app/oracle/admin/bin/
hald 8282 haldaemon 13w REG 252,1 0 884771 /var/lock/invcol.tmp (deleted)
hald 8282 haldaemon 14w REG 252,5 384 622829 /tmp/6967_26747/icredir.tx
hald 8282 haldaemon 17w REG 252,5 405 622831 /tmp/6967_26747/PowerSuppl
zabbix_ag 12269 zabbix txt REG 252,0 166092 917577 /sbin/zabbix_agentd (deleted)
zabbix_ag 12269 zabbix 1w REG 252,5 660 14 /tmp/zabbix_agentd.log (deleted)
zabbix_ag 12269 zabbix 2w REG 252,5 660 14 /tmp/zabbix_agentd.log (deleted)
zabbix_ag 12269 zabbix 3wW REG 252,5 5 17 /tmp/zabbix_agentd.pid (deleted)
zabbix_ag 12271 zabbix txt REG 252,0 166092 917577 /sbin/zabbix_agentd (deleted)
zabbix_ag 12271 zabbix 1w REG 252,5 660 14 /tmp/zabbix_agentd.log (deleted)
zabbix_ag 12271 zabbix 2w REG 252,5 660 14 /tmp/zabbix_agentd.log (deleted)
zabbix_ag 12271 zabbix 3w REG 252,5 5 17 /tmp/zabbix_agentd.pid (deleted)
zabbix_ag 12272 zabbix txt REG 252,0 166092 917577 /sbin/zabbix_agentd (deleted)
zabbix_ag 12272 zabbix 1w REG 252,5 660 14 /tmp/zabbix_agentd.log (deleted)
zabbix_ag 12272 zabbix 2w REG 252,5 660 14 /tmp/zabbix_agentd.log (deleted)
zabbix_ag 12272 zabbix 3w REG 252,5 5 17 /tmp/zabbix_agentd.pid (deleted)
zabbix_ag 12273 zabbix txt REG 252,0 166092 917577 /sbin/zabbix_agentd (deleted)
zabbix_ag 12273 zabbix 1w REG 252,5 660 14 /tmp/zabbix_agentd.log (deleted)
zabbix_ag 12273 zabbix 2w REG 252,5 660 14 /tmp/zabbix_agentd.log (deleted)
zabbix_ag 12273 zabbix 3w REG 252,5 5 17 /tmp/zabbix_agentd.pid (deleted)
zabbix_ag 12274 zabbix txt REG 252,0 166092 917577 /sbin/zabbix_agentd (deleted)
zabbix_ag 12274 zabbix 1w REG 252,5 660 14 /tmp/zabbix_agentd.log (deleted)
zabbix_ag 12274 zabbix 2w REG 252,5 660 14 /tmp/zabbix_agentd.log (deleted)
zabbix_ag 12274 zabbix 3w REG 252,5 5 17 /tmp/zabbix_agentd.pid (deleted)
zabbix_ag 12275 zabbix txt REG 252,0 166092 917577 /sbin/zabbix_agentd (deleted)
zabbix_ag 12275 zabbix 1w REG 252,5 660 14 /tmp/zabbix_agentd.log (deleted)
zabbix_ag 12275 zabbix 2w REG 252,5 660 14 /tmp/zabbix_agentd.log (deleted)
zabbix_ag 12275 zabbix 3w REG 252,5 5 17 /tmp/zabbix_agentd.pid (deleted)
xfs 12467 xfs txt REG 252,3 113992 1441847 /usr/bin/xfs (deleted)
nsrexecd 12502 root txt REG 252,3 1450200 131501 /usr/sbin/nsrexecd (deleted)
ohasd.bin 12918 oracle 18uw REG 252,1 0 1867780 /var/tmp/.oracle/sprocr_lo
ohasd.bin 12918 oracle 23w REG 252,7 108656 7979175 /u01/app/grid/product/11.2
yum-updat 13655 root txt REG 252,3 8304 1442624 /usr/bin/python.#prelink# (deleted)
gam_serve 13660 root txt REG 252,3 107912 1048598 /usr/libexec/gam_server (deleted)
mingetty 14553 root txt REG 252,0 16872 917555 /sbin/mingetty (deleted)
mingetty 14554 root txt REG 252,0 16872 917555 /sbin/mingetty (deleted)
mingetty 14555 root txt REG 252,0 16872 917555 /sbin/mingetty (deleted)
mingetty 14556 root txt REG 252,0 16872 917555 /sbin/mingetty (deleted)
mingetty 14557 root txt REG 252,0 16872 917555 /sbin/mingetty (deleted)
mingetty 14558 root txt REG 252,0 16872 917555 /sbin/mingetty (deleted)
init.ohas 14559 root txt REG 252,0 801816 229409 /bin/bash (deleted)
diskmon.b 15959 oracle 4w REG 252,7 34576084 6553608 /u01/app/grid/product/11.2
ocssd.bin 16020 oracle 5w REG 252,7 108656 7979175 /u01/app/grid/product/11.2
ocssd.bin 16020 oracle 31uw REG 252,1 0 1867795 /var/tmp/.oracle/sOCSSD_LL
ocssd.bin 16020 oracle 33uw REG 252,1 0 1867796 /var/tmp/.oracle/sOracle_C
ocssd.bin 16020 oracle 44uw REG 252,1 0 1867797 /var/tmp/.oracle/sOCSSD_LL
oraagent. 16792 oracle 12w REG 252,7 108656 7979175 /u01/app/grid/product/11.2
evmd.bin 16870 oracle 18w REG 252,7 108656 7979175 /u01/app/grid/product/11.2
oracle 17160 oracle 4w REG 252,7 10493324 7864900 /u01/app/grid/product/11.2
oracle 17160 oracle 12w REG 252,7 108656 7979175 /u01/app/grid/product/11.2
oracle 17162 oracle 4w REG 252,7 10493324 7864900 /u01/app/grid/product/11.2
oracle 17162 oracle 12w REG 252,7 108656 7979175 /u01/app/grid/product/11.2
oracle 17185 oracle 4w REG 252,7 10493324 7864900 /u01/app/grid/product/11.2
oracle 17185 oracle 12w REG 252,7 108656 7979175 /u01/app/grid/product/11.2
oracle 17189 oracle 4w REG 252,7 10493324 7864900 /u01/app/grid/product/11.2
oracle 17189 oracle 12w REG 252,7 108656 7979175 /u01/app/grid/product/11.2
oracle 17191 oracle 4w REG 252,7 10493324 7864900 /u01/app/grid/product/11.2
oracle 17191 oracle 12w REG 252,7 108656 7979175 /u01/app/grid/product/11.2
oracle 17193 oracle 4w REG 252,7 10493324 7864900 /u01/app/grid/product/11.2
oracle 17193 oracle 12w REG 252,7 108656 7979175 /u01/app/grid/product/11.2
oracle 17195 oracle 4w REG 252,7 10493324 7864900 /u01/app/grid/product/11.2
oracle 17195 oracle 12w REG 252,7 108656 7979175 /u01/app/grid/product/11.2
oracle 17197 oracle 4w REG 252,7 10493324 7864900 /u01/app/grid/product/11.2
oracle 17197 oracle 12w REG 252,7 108656 7979175 /u01/app/grid/product/11.2
oracle 17199 oracle 4w REG 252,7 10493324 7864900 /u01/app/grid/product/11.2
oracle 17199 oracle 12w REG 252,7 108656 7979175 /u01/app/grid/product/11.2
oracle 17201 oracle 4w REG 252,7 10493324 7864900 /u01/app/grid/product/11.2
oracle 17201 oracle 12w REG 252,7 108656 7979175 /u01/app/grid/product/11.2
oracle 17203 oracle 4w REG 252,7 10493324 7864900 /u01/app/grid/product/11.2
oracle 17203 oracle 12w REG 252,7 108656 7979175 /u01/app/grid/product/11.2
oracle 17205 oracle 4w REG 252,7 10493324 7864900 /u01/app/grid/product/11.2
oracle 17205 oracle 12w REG 252,7 108656 7979175 /u01/app/grid/product/11.2
oracle 17207 oracle 4w REG 252,7 10493324 7864900 /u01/app/grid/product/11.2
oracle 17207 oracle 12w REG 252,7 108656 7979175 /u01/app/grid/product/11.2
oracle 17209 oracle 4w REG 252,7 10493324 7864900 /u01/app/grid/product/11.2
oracle 17209 oracle 12w REG 252,7 108656 7979175 /u01/app/grid/product/11.2
oracle 17211 oracle 4w REG 252,7 10493324 7864900 /u01/app/grid/product/11.2
oracle 17211 oracle 12w REG 252,7 108656 7979175 /u01/app/grid/product/11.2
oracle 17505 oracle 4w REG 252,7 10493324 7864900 /u01/app/grid/product/11.2
oracle 17505 oracle 12w REG 252,7 108656 7979175 /u01/app/grid/product/11.2
oracle 17505 oracle 15u REG 252,7 751 10381178 /u01/app/grid/product/11.2
oracle 17508 oracle 4w REG 252,7 10493324 7864900 /u01/app/grid/product/11.2
oracle 17508 oracle 12w REG 252,7 108656 7979175 /u01/app/grid/product/11.2
oracle 17508 oracle 15u REG 252,7 751 10381181 /u01/app/grid/product/11.2
oracle 17512 oracle 4w REG 252,7 10493324 7864900 /u01/app/grid/product/11.2
oracle 17512 oracle 12w REG 252,7 108656 7979175 /u01/app/grid/product/11.2
oracle 17512 oracle 15u REG 252,7 751 10381180 /u01/app/grid/product/11.2
rman 19907 oracle 0r REG 252,5 0 41 /tmp/sh-thd-1430355787 (deleted)
oracle 20636 oracle 22w REG 252,7 891 4932230 /u01/app/oracle/admin/SMP1
oracle 20636 oracle 23w REG 252,7 60 4932231 /u01/app/oracle/admin/SMP1
oracle 20646 oracle 27u REG 252,7 751 10381200 /u01/app/grid/product/11.2
oracle 21064 oracle 22w REG 252,7 896 1950168 /u01/app/oracle/admin/OMPP
oracle 21064 oracle 23w REG 252,7 61 1950169 /u01/app/oracle/admin/OMPP
oracle 21068 oracle 27u REG 252,7 751 10381205 /u01/app/grid/product/11.2
oracle 21471 oracle 22w REG 252,7 896 7389709 /u01/app/oracle/admin/AVBE
oracle 21471 oracle 23w REG 252,7 61 7389710 /u01/app/oracle/admin/AVBE
oracle 21477 oracle 27u REG 252,7 751 10381211 /u01/app/grid/product/11.2
oracle 21767 oracle 22w REG 252,7 891 1246279 /u01/app/oracle/admin/MPBE
oracle 21767 oracle 23w REG 252,7 60 1246280 /u01/app/oracle/admin/MPBE
oracle 21771 oracle 27u REG 252,7 751 10381217 /u01/app/grid/product/11.2
oracle 22047 oracle 22w REG 252,7 891 557541 /u01/app/oracle/admin/FMBE
oracle 22047 oracle 23w REG 252,7 60 557542 /u01/app/oracle/admin/FMBE
oracle 22051 oracle 27u REG 252,7 751 10381222 /u01/app/grid/product/11.2
tail 22192 root 0u CHR 136,0 0t0 3 /dev/pts/0 (deleted)
tail 22192 root 1u CHR 136,0 0t0 3 /dev/pts/0 (deleted)
tail 22192 root 2u CHR 136,0 0t0 3 /dev/pts/0 (deleted)
tail 22192 root 3r REG 252,1 584218 1278295 /var/log/secure.1 (deleted)
oracle 22282 oracle 17w REG 252,7 1353 5605700 /u01/app/oracle/admin/OBBE
oracle 22282 oracle 18w REG 252,7 80 5605701 /u01/app/oracle/admin/OBBE
rman 22283 oracle 0r REG 252,5 0 35 /tmp/sh-thd-1430336644 (deleted)
oracle 22306 oracle 22w REG 252,7 891 5606210 /u01/app/oracle/admin/OBBE
oracle 22306 oracle 23w REG 252,7 60 5606217 /u01/app/oracle/admin/OBBE
oracle 22310 oracle 27u REG 252,7 751 10381227 /u01/app/grid/product/11.2
oracle 22562 oracle 17w REG 252,7 1352 3817604 /u01/app/oracle/admin/VISI
oracle 22562 oracle 18w REG 252,7 90 3817605 /u01/app/oracle/admin/VISI
oracle 22584 oracle 22w REG 252,7 891 12731257 /u01/app/oracle/admin/VISI
oracle 22584 oracle 23w REG 252,7 60 12731258 /u01/app/oracle/admin/VISI
oracle 22588 oracle 27u REG 252,7 751 10381232 /u01/app/grid/product/11.2
oracle 22936 oracle 22w REG 252,7 896 1246351 /u01/app/oracle/admin/UCMP
oracle 22936 oracle 23w REG 252,7 61 1246352 /u01/app/oracle/admin/UCMP
oracle 22940 oracle 27u REG 252,7 751 10381237 /u01/app/grid/product/11.2
oracle 23338 oracle 22w REG 252,7 906 5882350 /u01/app/oracle/admin/EREP
oracle 23338 oracle 23w REG 252,7 72 5882351 /u01/app/oracle/admin/EREP
oracle 23342 oracle 27u REG 252,7 751 10381242 /u01/app/grid/product/11.2
oracle 24321 oracle 22w REG 252,7 891 4456846 /u01/app/oracle/admin/WBBE
oracle 24321 oracle 23w REG 252,7 69 4456949 /u01/app/oracle/admin/WBBE
oracle 24325 oracle 27u REG 252,7 751 10381253 /u01/app/grid/product/11.2
oracle 24920 oracle 22w REG 252,7 901 4538880 /u01/app/oracle/admin/PVEX
oracle 24920 oracle 23w REG 252,7 62 4538881 /u01/app/oracle/admin/PVEX
oracle 24926 oracle 27u REG 252,7 751 10381258 /u01/app/grid/product/11.2
sudo 31789 root 0u CHR 136,2 0t0 5 /dev/pts/2 (deleted)
sudo 31789 root 1u CHR 136,2 0t0 5 /dev/pts/2 (deleted)
sudo 31789 root 2u CHR 136,2 0t0 5 /dev/pts/2 (deleted)
su 31791 root 0u CHR 136,2 0t0 5 /dev/pts/2 (deleted)
su 31791 root 1u CHR 136,2 0t0 5 /dev/pts/2 (deleted)
su 31791 root 2u CHR 136,2 0t0 5 /dev/pts/2 (deleted)
bash 31792 oracle 0u CHR 136,2 0t0 5 /dev/pts/2 (deleted)
bash 31792 oracle 1u CHR 136,2 0t0 5 /dev/pts/2 (deleted)
bash 31792 oracle 2u CHR 136,2 0t0 5 /dev/pts/2 (deleted)
bash 31792 oracle 255u CHR 136,2 0t0 5 /dev/pts/2 (deleted)
#
shall i still restart zabbix-agent ?
ASKER CERTIFIED SOLUTION
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
ASKER
i sent him an email. Thank you sir !
lsof +L1
is shorter way to see open deleted files.
is shorter way to see open deleted files.
Try:
/usr/sbin/lsof | grep deleted
(assuming you have lsof in /usr/sbin - if it's not there locate it first)
You should get a list of processes that still have open handles to 'deleted' files. Look for the process id in the 2nd column of the output. Kill that process and you should get your space back.