Solved

Linux system hard drive suddenly fills up.

Posted on 2016-11-03
7
26 Views
1 Endorsement
Last Modified: 2016-11-08
Hello,

I have a Linux box running as a router/firewall/Xeams firewall.  It has 146GB drive.  Up until about 2 weeks ago about 20GB was being used consistently.  All of the sudden the hard drive is getting full.  Xeams told me to run du -h . in the directory where Xeams is installed.  I do not see anything that is unusually large.

Please advise.

Have a great day,

Don
1
Comment
Question by:GEMCC
7 Comments
 
LVL 25

Expert Comment

by:Dr. Klahn
ID: 41872429
Are you in a command line environment, or is a GUI desktop available?
0
 

Author Comment

by:GEMCC
ID: 41872439
Command.  I ran df -h and see /dev/md1 is utilizing 139GB but do not know what to do.
0
 
LVL 25

Expert Comment

by:Dr. Klahn
ID: 41872466
OK.  You will need to su to get reliable results.

Use the command "du -h 1" to print the disk usage for the directory you are in and those one level down.  On the left is the space taken up, on the right is the directory name.  Here is an example from my system.

root@www:/# cd /
root@www:/# du -h -d 1
6.0M    ./bin
4.7M    ./sbin
4.0K    ./media
4.0K    ./srv
92M     ./lib
4.0K    ./opt
310M    ./var
804K    ./home
0       ./dev
8.0K    ./mnt
16K     ./lost+found
1.5G    ./usr
34M     ./boot
48K     ./root
59M     ./www
3.0M    ./etc
1012K   ./run
0       ./tmp
0       ./proc
0       ./sys
2.0G    .

Open in new window


We see that the majority of the space, 1.5 GB, is under /usr.  Now begin a tedious process of going down one level, looking at that level, deciding if it is where the disk went, and repeating until eventually the location of the disk loss is found.

root@www:/# cd /usr
root@www:/usr# du -h -d 1
37M     ./bin
15M     ./sbin
179M    ./lib
28M     ./local
190M    ./share
4.0K    ./games
984M    ./src
22M     ./include
30M     ./libexec
1.5G    .

Open in new window


This is mostly system stuff with the exception of /local.  Back up one level, look at /var.  And so on.

I would start by looking in /var.  Logfiles can become enormous very rapidly.
0
Master Your Team's Linux and Cloud Stack!

The average business loses $13.5M per year to ineffective training (per 1,000 employees). Keep ahead of the competition and combine in-person quality with online cost and flexibility by training with Linux Academy.

 
LVL 88

Expert Comment

by:rindi
ID: 41872557
What distro is it based on? are you running updates and not removing old kernel versions?

Xeams firewall seems to dump junk mails into the Quarantine, so if you get plenty of junk that would fill that up. So I'd suggest you check the Xeams firewall Quarantine settings.
1
 
LVL 77

Expert Comment

by:arnold
ID: 41872570
df -k

How your partitioned, the likely issue is that your logrotate did not sever the connection between a process and a LogFile into which it was writing, so while du does not reflect the space used, the file handle being active still reflects that space as in use.

You need to use lsof to locate a process attached to a large file. Restarting that process will release the space.

We're any changes made two to three weeks prior I.e. Logging firewall events, implementation of fail2ban?
0
 

Accepted Solution

by:
GEMCC earned 0 total points
ID: 41873285
1
 

Author Closing Comment

by:GEMCC
ID: 41878491
Issue resolved
0

Featured Post

PRTG Network Monitor: Intuitive Network Monitoring

Network Monitoring is essential to ensure that computer systems and network devices are running. Use PRTG to monitor LANs, servers, websites, applications and devices, bandwidth, virtual environments, remote systems, IoT, and many more. PRTG is easy to set up & use.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Suggested Solutions

Create your own, high-performance VM backup appliance by installing NAKIVO Backup & Replication directly onto a Synology NAS!
Google Drive is extremely cheap offsite storage, and it's even possible to get extra storage for free for two years.  You can use the free account 15GB, and if you have an Android device..when you install Google Drive for the first time it will give…
Learn how to get help with Linux/Unix bash shell commands. Use help to read help documents for built in bash shell commands.: Use man to interface with the online reference manuals for shell commands.: Use man to search man pages for unknown command…
This video Micro Tutorial explains how to clone a hard drive using a commercial software product for Windows systems called Casper from Future Systems Solutions (FSS). Cloning makes an exact, complete copy of one hard disk drive (HDD) onto another d…

803 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question