file archives in VB.NET

What are some of the proven methods of archiving large quantities of files (e.g. millions of files, terabytes in size)? I'm looking for a way to retrieve/add files to an archive format VB.NET in such a way that it is still robust even if the archive is huge.

I have explored using ZIP files for archives, with Chilkat's libraries. There's a 4gb limit on zip archives though. I've thought about ISOs, TARs, maybe CAB files?

What other standardized formats exist out there specifically for archiving large quanties of files? Is there anything microsoft made, or reliable VB.NET classes that can manipulate them?
LVL 31
Who is Participating?

[Webinar] Streamline your web hosting managementRegister Today

Jorge PaulinoConnect With a Mentor IT Pro/DeveloperCommented:
Another option ... SharpZipLib (

To be honest I don't know if it works with 4gb but you can try it!
May I ask why do you compress files that result in such a large compressed file? If you have thousand/milions of small files thar result in one large compressed file, then damage to this file will make all files in it useless. Anyway, one of the options is


Frosty555Author Commented:
>May I ask why do you compress files that result in such a large compressed file?
I'll be using the archives to store backups of files from the system (as well as from other sources). There can be many revisions of any single file in the archive, and file servers often have half a million files. Typically, each archive should be no larger than 4gb, small enough to store on a DVD, but I'm writing for scaleability.

Does not hav a programmable interface... that I could see. I love RAR files but I don't see how I can use this.

>Another option ... SharpZipLib (
I see a lot of promise in this - supports zip, bzip, encryption, even tar files! But where's their documentation? Code examples? Can't seem to find much in the way of help.
Jorge PaulinoIT Pro/DeveloperCommented:
I dont know how long will you keep archived files, but if you intend to keep them longer than 2-3 years, DVDs shouldn't be an option, since they are not made to last long. There was a story about some company that had a climatized room, and they kept in there archiving files on DVDs, and after a couple of years (not remember exactly how many years) most of them were unreadable.

As for the winrar, it accepts command line parameters
All Courses

From novice to tech pro — start learning today.