Solved

7zip zip file size growth

Posted on 2011-03-10
5
433 Views
Last Modified: 2012-05-11
Ok i have 140,000+ exchange log files that i need to get rid of, but they are not to be deleted. so instead of keeping 140GB on my server, i thought i would zip them. since it is all text and should zip quite nicely. so i wrote a script to seperate all 140,000+ logs into groups of 500 to be zipped. This is my result of the first 7 groups.

1) 1.754MB     <----What i expected
2) 129.455MB <----A little large, and as you can see it gets bigger
3) 181.573MB
4) 236.767MB
5) 249.138MB
6) 73.349MB   <----Now small again?...
7) 209.941MB

So my question to you all is, am i doing something wrong? nothing should be running on the pc in the background but i thought that it would just take longer to zip the files, not lower the quality.

This is what i am zipping with and the file specs
7za.exe                  <----7zip command line exe
Ultra                       <----7zip highest compression
EachFile is 1024KB <----All are the same size.

Any Ideas? i would like them to be 1MB if possible like the 1st group. :)

Thanks!
0
Comment
Question by:FnGizzardBrain
  • 3
  • 2
5 Comments
 
LVL 25

Accepted Solution

by:
Dr. Klahn earned 500 total points
ID: 35098734
The fact that the log files are the same size is misleading in this case.  They won't all compress to a standard size, even though the source files are the same size.

A log file that was used very little will compress very well, because most of the contents are unused.  A log file filled with dissimilar transactions is not going to compress well, because there is little redundancy, while a log file filled with similar transactions and similar timestamps will compress well due to considerable redundancy.  There may be considerable variation in compression efficiency.

I would, however, look very carefully at that first one that compressed 500 1 MB filed down to 1.7 MB.  The only way I can see that happening is if the files were filled with zeroes.
0
 

Author Comment

by:FnGizzardBrain
ID: 35098930
I see, i had a feeling it would be the contents not just the size. ok thank you for your input i will explore this and comment again tomorrow. Is there a way to compress that type of file more so than what i am getting? Any other program or type of compression i should look into?

Thank you
0
 
LVL 25

Assisted Solution

by:Dr. Klahn
Dr. Klahn earned 500 total points
ID: 35098995
You could look into bzip2, but I'm not sure whethere it's going to provide significantly better results.  Still, a byte saved is a byte earned.
0
 

Author Comment

by:FnGizzardBrain
ID: 35099007
I actually had some free time, i opened up several files from the first group and the second group to compare. And you are right, the first group was not blank however it is filled up with the same characters over and over again. Unlike the second that is a random assortment of characters and symbols. So that answered my question now i have a new one.

Is there a compression program that will do what i want?
0
 

Author Comment

by:FnGizzardBrain
ID: 35099018
Ok sounds good, Thank you for your help Drklahn!
0

Featured Post

Three Reasons Why Backup is Strategic

Backup is strategic to your business because your data is strategic to your business. Without backup, your business will fail. This white paper explains why it is vital for you to design and immediately execute a backup strategy to protect 100 percent of your data.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

This article is an update and follow-up of my previous article:   Storage 101: common concepts in the IT enterprise storage This time, I expand on more frequently used storage concepts.
Finding original email is quite difficult due to their duplicates. From this article, you will come to know why multiple duplicates of same emails appear and how to delete duplicate emails from Outlook securely and instantly while vital emails remai…
This video teaches viewers how to encrypt an external drive that requires a password to read and edit the drive. All tasks are done in Disk Utility. Plug in the external drive you wish to encrypt: Make sure all previous data on the drive has been …
This tutorial will walk an individual through the process of installing the necessary services and then configuring a Windows Server 2012 system as an iSCSI target. To install the necessary roles, go to Server Manager, and select Add Roles and Featu…

786 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question