Solved

Can I delete all the Linux log files or some of them? They are consuming a lot of disk space

Posted on 2014-09-24
16
670 Views
Last Modified: 2014-10-26
Hi,

Just wondering if it's safe to delete some or all of the log files? Maybe back them up? They are consuming a lot of disk space on my VPS.

Should I be running some script to only save the latest logs from the server and limit the log files? If so, how do I do that?

I'm on Centos 5.9.

I ran this:

 du -a /var | sort -n -r | head -n 10
... (meaning there were a bunch of permission denied directories)
du: cannot read directory `/var/empty/sshd': Permission denied
du: cannot read directory `/var/lib/mysql': Permission denied
du: cannot read directory `/var/lib/dav': Permission denied
du: cannot read directory `/var/lib/dovecot': Permission denied
du: cannot read directory `/var/lib/mlocate': Permission denied
du: cannot read directory `/var/lib/pgsql': Permission denied
5990632 /var
2745120 /var/log
2579472 /var/spool
2579428 /var/spool/repackage
1711924 /var/log/brcm-iscsi.log
808244  /var/log/btmp
417972  /var/lib
200108  /var/log/chkservd.log
182264  /var/cpanel
118796  /var/cpanel/perl

Open in new window



Thank you<><
Victor
0
Comment
Question by:Victor Kimura
[X]
Welcome to Experts Exchange

Add your voice to the tech community where 5M+ people just like you are talking about what matters.

  • Help others & share knowledge
  • Earn cash & points
  • Learn & ask questions
  • 4
  • 4
  • 4
  • +2
16 Comments
 
LVL 19

Accepted Solution

by:
NickUpson earned 166 total points
ID: 40343167
you should have logrotate running, that will rotate the logs and remove them depending upon its configuration. that way you have the most recent logs only
0
 
LVL 5

Assisted Solution

by:NARANTHIRAN
NARANTHIRAN earned 42 total points
ID: 40343375
The user does not have full rights to access the directory or the file.

Run the command with administrative privilege(root) it should work fine..
0
 
LVL 29

Assisted Solution

by:serialband
serialband earned 167 total points
ID: 40344364
Your logrotate should also have been set to compress the files.  I would suggest you peruse them to see why they're so big.  Unless you run a very active server, your logs should not be   You might have a problem that's filling the logs with information that you should be aware of.

Also, your /var/spool/repackage is quite large.  Try yum clean all  to remove them.  You don't need to keep yum packages around forever after you've updated.
0
U.S. Department of Agriculture and Acronis Access

With the new era of mobile computing, smartphones and tablets, wireless communications and cloud services, the USDA sought to take advantage of a mobilized workforce and the blurring lines between personal and corporate computing resources.

 
LVL 62

Assisted Solution

by:gheist
gheist earned 125 total points
ID: 40344473
I would be more worried about what hides in /var/spool
If you have *ANY* log rotation package installed, it is fairly likely it chops logs weekly and keeps some old logs around, so they dont grow to the sky.

Before deleting run lsof /vat/log/whatever.log - you will need to restart process that keeps log open before deleting.
It is much safer to truncate logs:
:>whatever.log
0
 
LVL 29

Assisted Solution

by:serialband
serialband earned 167 total points
ID: 40344919
Your /var/log/brcm-iscsi.log seem big too.  Do you have lots of iSCSI error messages.  Maybe it isn't be rotated and you have too many old entries.  I suggest clearing that out.
0
 

Author Comment

by:Victor Kimura
ID: 40344975
Hi all,

Thanks for all the feedback. Can you tell me what you fellows have for your log configuration? What commands do I use or scripts to rotate logs properly?

@gheist, you mentioned a log rotation pacakge. Which one? Command(s) to run? Still new to the linux admin stuff a bit.
@gheist, you mentioned before I delete any logs then I would need to restart a process? What process? What command(s) before deleting?

@serilalband mentioned run yum clean all. Any other commands suggested?

Thank you.
0
 
LVL 19

Assisted Solution

by:NickUpson
NickUpson earned 166 total points
ID: 40345002
check you have a package called logrotate installed, if you do check its called from cron
0
 
LVL 29

Assisted Solution

by:serialband
serialband earned 167 total points
ID: 40345117
Have you run yum clean all?  Did /var/spool/repackage empty out?  If not, you could just remove the rpms in there if you aren't actively running an update.  You don't need to keep them after the update.  Have you cleared out your /var/log/brcm-iscsi.log file yet?    Those 2 combined are more than 2/3 of the space in /var

You can worry about the logrotate after you clear those out first.  Logrotate should be on Linux/BSD/OSX now and enable by default.  You can easily tell that you have it if you have logs followed by numbers and/or gz.

For example:
ls syslog*
syslog      syslog.1  syslog.2.gz  syslog.3.gz  syslog.4.gz  syslog.5.gz  syslog.6.gz  syslog.7.gz
0
 

Author Comment

by:Victor Kimura
ID: 40345219
@serialband, ok. I just deleted all the *.rpm files in the /var/spool/repackage. Also deleted /var/log/brcm-iscsi.log.

I have WHM/cPanel. Are there any know problems/issues with running yum clean all?

---
Ok, thanks, NickUpson. Just reading this on the logrotate:
http://articles.slicehost.com/2010/6/30/understanding-logrotate-on-centos-part-1

I have this in the /var folder re. disk usage:
root@ip-184-168-116-73 [/var/log]# du -a /var | sort -n -r | head -n 10         28511228        /var
27194204        /var/lib
26739776        /var/lib/mysql
1049608 /var/lib/mysql/mysql-bin.000024
1049608 /var/lib/mysql/mysql-bin.000022
1049608 /var/lib/mysql/mysql-bin.000021
1049608 /var/lib/mysql/mysql-bin.000020
1049608 /var/lib/mysql/mysql-bin.000019
1049608 /var/lib/mysql/mysql-bin.000018
1049608 /var/lib/mysql/mysql-bin.000017

Open in new window


Should I delete the files like /var/lib/mysql/mysql-bin.000024, etc?

Looks like I have logrotate.
cd /etc/cron.daily
root@ip-184-168-116-73 [/etc/cron.daily]# ls -al
total 36
drwxr-xr-x  2 root root  4096 Oct 11  2013 ./
drwxr-xr-x 80 root root 12288 Sep 25 17:45 ../
-rwxr-xr-x  1 root root   180 Jun  4  2012 logrotate*
-rwxr-xr-x  1 root root   418 May 30  2012 makewhatis.cron*
-rw-r--r--  1 root root   137 Jul  6  2011 mlocate.cron
-rwxr-xr-x  1 root root   296 Feb 25  2013 rpm*
-rwxr-xr-x  1 root root   354 Aug 11  2010 tmpwatch*
root@ip-184-168-116-73 [/etc/cron.daily]#

Open in new window

0
 

Author Comment

by:Victor Kimura
ID: 40345252
My logrotate config settings. Does it look ok? Should I change or add anything in this config file? Considering I had massive logs I would think so but not quite sure what to add.

cd /etc/cron.daily
root@ip-184-168-116-73 [/etc/cron.daily]# ls -al
total 36
drwxr-xr-x  2 root root  4096 Oct 11  2013 ./
drwxr-xr-x 80 root root 12288 Sep 25 17:45 ../
-rwxr-xr-x  1 root root   180 Jun  4  2012 logrotate*
-rwxr-xr-x  1 root root   418 May 30  2012 makewhatis.cron*
-rw-r--r--  1 root root   137 Jul  6  2011 mlocate.cron
-rwxr-xr-x  1 root root   296 Feb 25  2013 rpm*
-rwxr-xr-x  1 root root   354 Aug 11  2010 tmpwatch*
root@ip-184-168-116-73 [/etc/cron.daily]# cat logrotate
#!/bin/sh

/usr/sbin/logrotate /etc/logrotate.conf
EXITVALUE=$?
if [ $EXITVALUE != 0 ]; then
    /usr/bin/logger -t logrotate "ALERT exited abnormally with [$EXITVALUE]"
fi
exit 0
root@ip-184-168-116-73 [/etc/cron.daily]# cat /etc/logrotate.conf
# see "man logrotate" for details
# rotate log files daily
daily

# keep 14 days worth of backlogs
rotate 14

# create new (empty) log files after rotating old ones
create

# uncomment this if you want your log files compressed
compress

# RPM packages drop log rotation information into this directory
include /etc/logrotate.d

# no packages own wtmp -- we'll rotate them here
/var/log/wtmp {
    monthly
    minsize 1M
    create 0664 root utmp
    rotate 1
}

# system-specific logs may be also be configured here.
root@ip-184-168-116-73 [/etc/cron.daily]#

Open in new window

0
 
LVL 62

Assisted Solution

by:gheist
gheist earned 125 total points
ID: 40345435
No, it seems rotating logs.
What is written in that bcrm-iscsi log ?
Is it mentioned anywhere in /etc/logrotate.conf /etc/logrotate.d/*.conf
If not - truncate it.
0
 
LVL 19

Assisted Solution

by:NickUpson
NickUpson earned 166 total points
ID: 40345745
if you don't want them reducing this might suit you

# keep 14 days worth of backlogs
rotate 14
0
 

Author Comment

by:Victor Kimura
ID: 40345972
@gheist, I deleted the bcrm-iscsi log. Don't know what it was. I backed it up though in case I need it. It's not mentioned in the the /etc/logrotate.conf nor the /etc/logrotate.d/*.conf

@NickUpson, the rotate 14 seems to be set already.

I have WHM/cPanel. Are there any know problems/issues with running yum clean all?

Should I delete the files like /var/lib/mysql/mysql-bin.000024, etc?
0
 
LVL 62

Assisted Solution

by:gheist
gheist earned 125 total points
ID: 40345985
Mistake Mistake Mistake
You deleted it and now some process continues to write deleted inode.
check with lsof +L1 and restart it correctly.

No you should not delete any database data files.

There are no problems running "yum clean all". Only effect is that it will download updates next time you run it.
0
 
LVL 29

Assisted Solution

by:serialband
serialband earned 167 total points
ID: 40346064
iscsi is SCSI connection over ethernet.  It's probably a log of your RAID.  You should read your logs first before attempting to delete them.  As gheist said, the process is still writing to the iNode.  You need to stop the process that's holding the file open before you can delete it properly.  You now have an "invisible" file that's still being written to.

Don't just delete log files.  At least find out what's in them to make them so large.  Otherwise, why even bother having them.


yum clean all cleans out the cruft left behind by yum when it updates or installs files.  There's no problem running that.  It's the only "safe" thing you can do.  yum downloads updates each time it runs.

DO NOT DELETE your MySQL files.  You'll corrupt your database.
0
 
LVL 19

Assisted Solution

by:NickUpson
NickUpson earned 166 total points
ID: 40346441
I'm suggesting you consider changing the 14 to something less
0

Featured Post

Back Up Your Microsoft Windows Server®

Back up all your Microsoft Windows Server – on-premises, in remote locations, in private and hybrid clouds. Your entire Windows Server will be backed up in one easy step with patented, block-level disk imaging. We achieve RTOs (recovery time objectives) as low as 15 seconds.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Suggested Solutions

Title # Comments Views Activity
Linux Hosting 16 108
postfix start error! 9 86
Linux Server mapping drive using SSH key 9 53
need a help to fix one if else shell script issue 6 33
SSH (Secure Shell) - Tips and Tricks As you all know SSH(Secure Shell) is a network protocol, which we use to access/transfer files securely between two networked devices. SSH was actually designed as a replacement for insecure protocols that sen…
Linux users are sometimes dumbfounded by the severe lack of documentation on a topic. Sometimes, the documentation is copious, but other times, you end up with some obscure "it varies depending on your distribution" over and over when searching for …
Learn several ways to interact with files and get file information from the bash shell. ls lists the contents of a directory: Using the -a flag displays hidden files: Using the -l flag formats the output in a long list: The file command gives us mor…
Get a first impression of how PRTG looks and learn how it works.   This video is a short introduction to PRTG, as an initial overview or as a quick start for new PRTG users.
Suggested Courses

710 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question