Go Premium for a chance to win a PS4. Enter to Win

x
?
Solved

7zip zip file size growth

Posted on 2011-03-10
5
Medium Priority
?
439 Views
Last Modified: 2012-05-11
Ok i have 140,000+ exchange log files that i need to get rid of, but they are not to be deleted. so instead of keeping 140GB on my server, i thought i would zip them. since it is all text and should zip quite nicely. so i wrote a script to seperate all 140,000+ logs into groups of 500 to be zipped. This is my result of the first 7 groups.

1) 1.754MB     <----What i expected
2) 129.455MB <----A little large, and as you can see it gets bigger
3) 181.573MB
4) 236.767MB
5) 249.138MB
6) 73.349MB   <----Now small again?...
7) 209.941MB

So my question to you all is, am i doing something wrong? nothing should be running on the pc in the background but i thought that it would just take longer to zip the files, not lower the quality.

This is what i am zipping with and the file specs
7za.exe                  <----7zip command line exe
Ultra                       <----7zip highest compression
EachFile is 1024KB <----All are the same size.

Any Ideas? i would like them to be 1MB if possible like the 1st group. :)

Thanks!
0
Comment
Question by:FnGizzardBrain
  • 3
  • 2
5 Comments
 
LVL 30

Accepted Solution

by:
Dr. Klahn earned 2000 total points
ID: 35098734
The fact that the log files are the same size is misleading in this case.  They won't all compress to a standard size, even though the source files are the same size.

A log file that was used very little will compress very well, because most of the contents are unused.  A log file filled with dissimilar transactions is not going to compress well, because there is little redundancy, while a log file filled with similar transactions and similar timestamps will compress well due to considerable redundancy.  There may be considerable variation in compression efficiency.

I would, however, look very carefully at that first one that compressed 500 1 MB filed down to 1.7 MB.  The only way I can see that happening is if the files were filled with zeroes.
0
 

Author Comment

by:FnGizzardBrain
ID: 35098930
I see, i had a feeling it would be the contents not just the size. ok thank you for your input i will explore this and comment again tomorrow. Is there a way to compress that type of file more so than what i am getting? Any other program or type of compression i should look into?

Thank you
0
 
LVL 30

Assisted Solution

by:Dr. Klahn
Dr. Klahn earned 2000 total points
ID: 35098995
You could look into bzip2, but I'm not sure whethere it's going to provide significantly better results.  Still, a byte saved is a byte earned.
0
 

Author Comment

by:FnGizzardBrain
ID: 35099007
I actually had some free time, i opened up several files from the first group and the second group to compare. And you are right, the first group was not blank however it is filled up with the same characters over and over again. Unlike the second that is a random assortment of characters and symbols. So that answered my question now i have a new one.

Is there a compression program that will do what i want?
0
 

Author Comment

by:FnGizzardBrain
ID: 35099018
Ok sounds good, Thank you for your help Drklahn!
0

Featured Post

Industry Leaders: We Want Your Opinion!

We value your feedback.

Take our survey and automatically be enter to win anyone of the following:
Yeti Cooler, Amazon eGift Card, and Movie eGift Card!

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

This article is an update and follow-up of my previous article:   Storage 101: common concepts in the IT enterprise storage This time, I expand on more frequently used storage concepts.
Windows Server 2003 introduced persistent Volume Shadow Copies and made 2003 a must-do upgrade.  Since then, it's been a must-implement feature for all servers doing any kind of file sharing.
This Micro Tutorial will teach you how to reformat your flash drive. Sometimes your flash drive may have issues carrying files so this will completely restore it to manufacturing settings. Make sure to backup all files before reformatting. This w…
Despite its rising prevalence in the business world, "the cloud" is still misunderstood. Some companies still believe common misconceptions about lack of security in cloud solutions and many misuses of cloud storage options still occur every day. …
Suggested Courses

916 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question