Solved

Linux system hard drive suddenly fills up.

Posted on 2016-11-03
7
28 Views
1 Endorsement
Last Modified: 2016-11-08
Hello,

I have a Linux box running as a router/firewall/Xeams firewall.  It has 146GB drive.  Up until about 2 weeks ago about 20GB was being used consistently.  All of the sudden the hard drive is getting full.  Xeams told me to run du -h . in the directory where Xeams is installed.  I do not see anything that is unusually large.

Please advise.

Have a great day,

Don
1
Comment
Question by:GEMCC
[X]
Welcome to Experts Exchange

Add your voice to the tech community where 5M+ people just like you are talking about what matters.

  • Help others & share knowledge
  • Earn cash & points
  • Learn & ask questions
7 Comments
 
LVL 27

Expert Comment

by:Dr. Klahn
ID: 41872429
Are you in a command line environment, or is a GUI desktop available?
0
 

Author Comment

by:GEMCC
ID: 41872439
Command.  I ran df -h and see /dev/md1 is utilizing 139GB but do not know what to do.
0
 
LVL 27

Expert Comment

by:Dr. Klahn
ID: 41872466
OK.  You will need to su to get reliable results.

Use the command "du -h 1" to print the disk usage for the directory you are in and those one level down.  On the left is the space taken up, on the right is the directory name.  Here is an example from my system.

root@www:/# cd /
root@www:/# du -h -d 1
6.0M    ./bin
4.7M    ./sbin
4.0K    ./media
4.0K    ./srv
92M     ./lib
4.0K    ./opt
310M    ./var
804K    ./home
0       ./dev
8.0K    ./mnt
16K     ./lost+found
1.5G    ./usr
34M     ./boot
48K     ./root
59M     ./www
3.0M    ./etc
1012K   ./run
0       ./tmp
0       ./proc
0       ./sys
2.0G    .

Open in new window


We see that the majority of the space, 1.5 GB, is under /usr.  Now begin a tedious process of going down one level, looking at that level, deciding if it is where the disk went, and repeating until eventually the location of the disk loss is found.

root@www:/# cd /usr
root@www:/usr# du -h -d 1
37M     ./bin
15M     ./sbin
179M    ./lib
28M     ./local
190M    ./share
4.0K    ./games
984M    ./src
22M     ./include
30M     ./libexec
1.5G    .

Open in new window


This is mostly system stuff with the exception of /local.  Back up one level, look at /var.  And so on.

I would start by looking in /var.  Logfiles can become enormous very rapidly.
0
Comprehensive Backup Solutions for Microsoft

Acronis protects the complete Microsoft technology stack: Windows Server, Windows PC, laptop and Surface data; Microsoft business applications; Microsoft Hyper-V; Azure VMs; Microsoft Windows Server 2016; Microsoft Exchange 2016 and SQL Server 2016.

 
LVL 88

Expert Comment

by:rindi
ID: 41872557
What distro is it based on? are you running updates and not removing old kernel versions?

Xeams firewall seems to dump junk mails into the Quarantine, so if you get plenty of junk that would fill that up. So I'd suggest you check the Xeams firewall Quarantine settings.
1
 
LVL 78

Expert Comment

by:arnold
ID: 41872570
df -k

How your partitioned, the likely issue is that your logrotate did not sever the connection between a process and a LogFile into which it was writing, so while du does not reflect the space used, the file handle being active still reflects that space as in use.

You need to use lsof to locate a process attached to a large file. Restarting that process will release the space.

We're any changes made two to three weeks prior I.e. Logging firewall events, implementation of fail2ban?
0
 

Accepted Solution

by:
GEMCC earned 0 total points
ID: 41873285
1
 

Author Closing Comment

by:GEMCC
ID: 41878491
Issue resolved
0

Featured Post

Back Up Your Microsoft Windows Server®

Back up all your Microsoft Windows Server – on-premises, in remote locations, in private and hybrid clouds. Your entire Windows Server will be backed up in one easy step with patented, block-level disk imaging. We achieve RTOs (recovery time objectives) as low as 15 seconds.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Fine Tune your automatic Updates for Ubuntu / Debian
This article aims to explain the working of CircularLogArchiver. This tool was designed to solve the buildup of log file in cases where systems do not support circular logging or where circular logging is not enabled
Learn how to find files with the shell using the find and locate commands. Use locate to find a needle in a haystack.: With locate, check if the file still exists.: Use find to get the actual location of the file.:
This demo shows you how to set up the containerized NetScaler CPX with NetScaler Management and Analytics System in a non-routable Mesos/Marathon environment for use with Micro-Services applications.

756 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question