Solved

7zip zip file size growth

Posted on 2011-03-10
5
432 Views
Last Modified: 2012-05-11
Ok i have 140,000+ exchange log files that i need to get rid of, but they are not to be deleted. so instead of keeping 140GB on my server, i thought i would zip them. since it is all text and should zip quite nicely. so i wrote a script to seperate all 140,000+ logs into groups of 500 to be zipped. This is my result of the first 7 groups.

1) 1.754MB     <----What i expected
2) 129.455MB <----A little large, and as you can see it gets bigger
3) 181.573MB
4) 236.767MB
5) 249.138MB
6) 73.349MB   <----Now small again?...
7) 209.941MB

So my question to you all is, am i doing something wrong? nothing should be running on the pc in the background but i thought that it would just take longer to zip the files, not lower the quality.

This is what i am zipping with and the file specs
7za.exe                  <----7zip command line exe
Ultra                       <----7zip highest compression
EachFile is 1024KB <----All are the same size.

Any Ideas? i would like them to be 1MB if possible like the 1st group. :)

Thanks!
0
Comment
Question by:FnGizzardBrain
  • 3
  • 2
5 Comments
 
LVL 24

Accepted Solution

by:
Dr. Klahn earned 500 total points
ID: 35098734
The fact that the log files are the same size is misleading in this case.  They won't all compress to a standard size, even though the source files are the same size.

A log file that was used very little will compress very well, because most of the contents are unused.  A log file filled with dissimilar transactions is not going to compress well, because there is little redundancy, while a log file filled with similar transactions and similar timestamps will compress well due to considerable redundancy.  There may be considerable variation in compression efficiency.

I would, however, look very carefully at that first one that compressed 500 1 MB filed down to 1.7 MB.  The only way I can see that happening is if the files were filled with zeroes.
0
 

Author Comment

by:FnGizzardBrain
ID: 35098930
I see, i had a feeling it would be the contents not just the size. ok thank you for your input i will explore this and comment again tomorrow. Is there a way to compress that type of file more so than what i am getting? Any other program or type of compression i should look into?

Thank you
0
 
LVL 24

Assisted Solution

by:Dr. Klahn
Dr. Klahn earned 500 total points
ID: 35098995
You could look into bzip2, but I'm not sure whethere it's going to provide significantly better results.  Still, a byte saved is a byte earned.
0
 

Author Comment

by:FnGizzardBrain
ID: 35099007
I actually had some free time, i opened up several files from the first group and the second group to compare. And you are right, the first group was not blank however it is filled up with the same characters over and over again. Unlike the second that is a random assortment of characters and symbols. So that answered my question now i have a new one.

Is there a compression program that will do what i want?
0
 

Author Comment

by:FnGizzardBrain
ID: 35099018
Ok sounds good, Thank you for your help Drklahn!
0

Featured Post

Ransomware-A Revenue Bonanza for Service Providers

Ransomware – malware that gets on your customers’ computers, encrypts their data, and extorts a hefty ransom for the decryption keys – is a surging new threat.  The purpose of this eBook is to educate the reader about ransomware attacks.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Suggested Solutions

Title # Comments Views Activity
make a "clean" image windows 7 with paragon 6 44
VMware 6 increase datastore size 8 84
data center consolidation 8 88
Hot Spare Testing on Non-HotSwap server 6 24
A bootable USB key can be very handy now-a-days. My favorite USB key consists of our Windows 7 image, network card drivers (to connect up to a Ghost server), the latest BIOS updates for all of our PCs and CopyWipe (to erase a retired PC) Creating…
The article will include the best Data Recovery Tools along with their Features, Capabilities, and their Download Links. Hope you’ll enjoy it and will choose the one as required by you.
This tutorial will walk an individual through the process of installing the necessary services and then configuring a Windows Server 2012 system as an iSCSI target. To install the necessary roles, go to Server Manager, and select Add Roles and Featu…
This Micro Tutorial will teach you how to reformat your flash drive. Sometimes your flash drive may have issues carrying files so this will completely restore it to manufacturing settings. Make sure to backup all files before reformatting. This w…

932 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

9 Experts available now in Live!

Get 1:1 Help Now