At my work we have massive data in gzip files with corrupted headers.
gzip -l ewifi.log-pb-inet2111.rly2.company-20140116-anclogging2102.txt.gz
compressed uncompressed ratio uncompressed_name
210974992 83353 -253010.2% ewifi.log-pb-inet2111.rly2.company-20140116-anclogging2102.txt
This data has to be read line by line using both C++ and Java.
I have tried this in the past and asked this in the past as well. Any library that is based on header information wouldn't work.
Linux utility gunzip works but for other libraries, it would only read first few hundred KB and then quit.
In the past I always did it in C by opening a pipe (popen) and using gunzip -c command. I wonder if you could use streams to read it line by line or something.