When compressing a file on a unix server

Posted on 2005-03-09
Medium Priority
Last Modified: 2013-12-27
When compressing a log file using the compress command in solaris8 how much space do I need to leave on the file system. For example I want to compress a 1gig file and only have 100 megs free space on the drive could I do it?
Question by:tuckcorp
Welcome to Experts Exchange

Add your voice to the tech community where 5M+ people just like you are talking about what matters.

  • Help others & share knowledge
  • Earn cash & points
  • Learn & ask questions

Accepted Solution

garypen earned 1000 total points
ID: 13499426
A neat trick is to use the -v option to work out how well the compression will be performed whilst writing the output to /dev/null. That way you can work out how much space is required before doing the compress for real.


I want to compress a large file called large.file

-bash-3.00$ ls -l large.file
-rw-r--r--   1 garypen  staff    294625280 Feb 26 14:11 large.file
-bash-3.00$ compress -cv large.file > /dev/null
large.file: Compression: 29.87%

In this case we know when we compress the file for real it will reduce by 29.87% of 294625280, i.e. 88004571, bytes. So as long as we have 294625280 -  88004571, i.e. 20662079 bytes, space free, we are in business.

An alternative solution is to do the following:

1) Identify an alternative filesystem which has at least [logfile size] free space.
2) Use compress to compress the file to standard output and redirect standard output to this other filesystem.


Imagine you have a 1G log file which you wish to compress and you have 2G of space available in /tmp.

You could do the following command

compress -c log.file > /tmp/log.file.Z

The above will definitely work on Solaris 10. I don't have access to Solaris 8 to test this, but I think the -v option existed back in Solaris 8. Also, gzip and bzip2 both compress better than compress if you are really keen to save space. Both are widely available as freeware for Solaris 8 and gzip may even have shipped as standard in Solaris 8, can't remember. They both ship as standard in Solaris 10.

LVL 48

Expert Comment

ID: 13500996
When you compress a file using any of the compression utilities (you should be using gzip or bzip2 instead of compress), the amount of space needed is:

original size + compressed size

Say your 1GB file compresses to 100MB, then you will need 1110MB free in order to compress it.

However, as garypen has pointed out, you can simply compress it to another filesystem.  I would do it like:

gzip -c /filesystem/largefile >/tmp/largefile.gz

if [ $? -eq 0 ]
     echo "gzip succeeded"
     rm /filesystem/largefile
     mv /tmp/filefile.gz /filesystem/
    echo "gzip failed.  Leaving original file intact"
    rm -f /tmp/largefile.gz

Featured Post

Get 15 Days FREE Full-Featured Trial

Benefit from a mission critical IT monitoring with Monitis Premium or get it FREE for your entry level monitoring needs.
-Over 200,000 users
-More than 300,000 websites monitored
-Used in 197 countries
-Recommended by 98% of users

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

I have been running these systems for a few years now and I am just very happy with them.   I just wanted to share the manual that I have created for upgrades and other things.  Oooh yes! FreeBSD makes me happy (as a server), no maintenance and I al…
Introduction Regular patching is part of a system administrator's tasks. However, many patches require that the system be in single-user mode before they can be installed. A cluster patch in particular can take quite a while to apply if the machine…
Learn several ways to interact with files and get file information from the bash shell. ls lists the contents of a directory: Using the -a flag displays hidden files: Using the -l flag formats the output in a long list: The file command gives us mor…
Learn how to find files with the shell using the find and locate commands. Use locate to find a needle in a haystack.: With locate, check if the file still exists.: Use find to get the actual location of the file.:
Suggested Courses

765 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question