Problem reading .csv file from zip archive using PHP

I'm having a problem reading the contents of a .csv file from a zip archive in PHP. The zip archive contains several .csv files and is approximately 5K in size. My script works when looping through echoing the file names, but as soon as I add the line to read the contents of a file, the page hangs up and eventually results in a "Network Error (tcp_error)" "Operation timed out."

Here is my code, it's pretty basic and practically copied from the documentation.
if (is_file($rpt) ) {
	if ($zip = zip_open($rpt) ) {
		while ($zip_entry = zip_read($zip) ) {
			if (zip_entry_name($zip_entry) == 'config.csv') {
				if (zip_entry_open($zip,$zip_entry) ) {
					if ($buff =  zip_entry_read($zip_entry) ) {
						echo $buff;
					}
				}
			} else {
				echo zip_entry_name($zip_entry)."\n";
			}
		}
		zip_close($zip);
	}
}

Open in new window

Here are the contents of the zip file I'm trying to read:
Key,Value
AccountID,494
AccountName,XXXXXXXXXXXXXXXXX
ScheduleID,45
EndPoint,http://xxxxxxxxxxxxx.xxx/REST/programservice/GetOutboundReportData
Email,xxxxxxxxxx@xxxxxxxxxxx.com
FtpUrl,ftp://XXXXXXXXXXXXX.com
FtpUsername,XXXXXXXXXX
FtpPassword,XXXXXXXXX

Open in new window

LVL 22
Kim WalkerWeb Programmer/TechnicianAsked:
Who is Participating?
I wear a lot of hats...

"The solutions and answers provided on Experts Exchange have been extremely helpful to me over the last few years. I wear a lot of hats - Developer, Database Administrator, Help Desk, etc., so I know a lot of things but not a lot about one thing. Experts Exchange gives me answers from people who do know a lot about one thing, in a easy to use platform." -Todd S.

rinfoCommented:
you can try this in stead
<?php
{
	if (is_file($rpt)
	{
		$fp = fopen('zip://'.$rpt.'#config.csv', 'r'); 
        if (!$fp) 
		{
			exit("cannot open\n");
		}	
    }
	print "<table>\n";
    while (($data = fgetcsv($fp, 0)) !== false)
	{
		 print '<tr>';
		 for ($i = 0, $j = count($data); $i < $j; $i++)
		 {
			print '<td>'.$data[$i].'</td>';		 
		 }
		 print "</tr>\n";
		 print '</table>\n';
    }

    fclose($fp);   		
	
	
}
<?

Open in new window

0
Ray PaseurCommented:
Please post a link to your test data, and I'll see if I can give you a tested and working code sample.  Thanks, ~Ray
0
Kim WalkerWeb Programmer/TechnicianAuthor Commented:
rinfo, I copied your solution, removed lines 1, 2, 24-27, and added a closing parenthesis to line 3, and inserted in place of the 16 lines I posted in my question. It produced the same results.

Thanks to Ray_Paseur, however, I've discovered the problem is in the archive I'm trying to unzip. I redacted the files in the archive and re-uploaded them to the server and both our scripts generated the appropriate output from the redacted files.

The archive is produced by a service provider and uploaded automatically by them every thirty minutes. I need to write a script to access those files and append the contents to a database that is used to generated a dynamic report. The data is incremental so if I miss one archive, the dynamic report is inaccurate.

Can you suggest a reason that PHP might not be able to unzip the archive but I can unzip it on my local computer? I can even decompress the archive and re-compress it without modification on my local computer, upload it to the server and execute the script without error. But of course, I can't do that every thirty minutes.

Ray_Paseur, is there a way to upload the original archive and delete it after you've looked at it? It contains personally identifiable information, so I'm reluctant to upload it permanently.
0
Cloud Class® Course: Ruby Fundamentals

This course will introduce you to Ruby, as well as teach you about classes, methods, variables, data structures, loops, enumerable methods, and finishing touches.

Ray PaseurCommented:
I am pretty sure I can delete the file for you, but I hope you can give us test data instead of live data -- well-crafted test data is a requirement for successful programming.
0
Kim WalkerWeb Programmer/TechnicianAuthor Commented:
This is data from test subjects all of whom are adults and none of the data is confidential. But I would appreciate it if we can delete it when the question is closed.
0
Kim WalkerWeb Programmer/TechnicianAuthor Commented:
It appears that I can't upload the file to EE. I gave up when it hadn't finished after several minutes. So I've posted it where I can delete myself when the time comes. I should have thought of that earlier.
0
Ray PaseurCommented:
This ZIP archive does not appear to be usable with PHP.  Please see http://www.laprbass.com/RAY_temp_xmediaman.php
<?php // RAY_temp_xmediaman.php
error_reporting(E_ALL);

// SEE http://www.experts-exchange.com/Web_Development/Web_Languages-Standards/PHP/Q_28228007.html#a39457014

// LOCATION OF THE TEST DATA SET
$url = 'http://stats.ha31.com/report.zip';

// TRY...
$za = new ZipArchive;
$za->open($url);

// SHOW THE OBJECT AND ANY STATUS / ERROR MESSAGES
var_dump($za);
echo ZipStatusString($za->status); // NO ERRORS, NO DATA

// CAN WE GET THE INNARDS?
$za->extractTo('RAY_junk');


function ZipStatusString( $status )
{
    switch( (int) $status )
    {
        case ZipArchive::ER_OK           : return 'N No error';
        case ZipArchive::ER_MULTIDISK    : return 'N Multi-disk zip archives not supported';
        case ZipArchive::ER_RENAME       : return 'S Renaming temporary file failed';
        case ZipArchive::ER_CLOSE        : return 'S Closing zip archive failed';
        case ZipArchive::ER_SEEK         : return 'S Seek error';
        case ZipArchive::ER_READ         : return 'S Read error';
        case ZipArchive::ER_WRITE        : return 'S Write error';
        case ZipArchive::ER_CRC          : return 'N CRC error';
        case ZipArchive::ER_ZIPCLOSED    : return 'N Containing zip archive was closed';
        case ZipArchive::ER_NOENT        : return 'N No such file';
        case ZipArchive::ER_EXISTS       : return 'N File already exists';
        case ZipArchive::ER_OPEN         : return 'S Can\'t open file';
        case ZipArchive::ER_TMPOPEN      : return 'S Failure to create temporary file';
        case ZipArchive::ER_ZLIB         : return 'Z Zlib error';
        case ZipArchive::ER_MEMORY       : return 'N Malloc failure';
        case ZipArchive::ER_CHANGED      : return 'N Entry has been changed';
        case ZipArchive::ER_COMPNOTSUPP  : return 'N Compression method not supported';
        case ZipArchive::ER_EOF          : return 'N Premature EOF';
        case ZipArchive::ER_INVAL        : return 'N Invalid argument';
        case ZipArchive::ER_NOZIP        : return 'N Not a zip archive';
        case ZipArchive::ER_INTERNAL     : return 'N Internal error';
        case ZipArchive::ER_INCONS       : return 'N Zip archive inconsistent';
        case ZipArchive::ER_REMOVE       : return 'S Can\'t remove file';
        case ZipArchive::ER_DELETED      : return 'N Entry has been deleted';

        default: return sprintf('Unknown status %s', $status );
    }
}

Open in new window

Some information that may be of value:
http://php.net/manual/en/class.ziparchive.php#100268

Does your service provider offer any other way of getting the data, except by ZIP?
0
Kim WalkerWeb Programmer/TechnicianAuthor Commented:
Thanks, Ray_Paseur. Miraculously, the script is working now -- though it does push the 60-second timeout limit and the re-zipped archive processes almost instantaneously. I'm going to increase the time limit and proceed as is. But I will contact our service provider to see if they have any alternatives to offer. I'll post an update if anything changes in the next couple of days. But I expect to close the question with my own resolution and, since it's a new month, award some points for effort.

Thanks,
XMediaMan
0
Ray PaseurCommented:
Glad you've got a working solution, but I still sense a disconnect here.  You can make PHP scripts survive and run as long as you want with set_time_limit(). I wonder why there is a difference in time between two ZIP archives of essentially the same data.  This sounds like a bug in the ZIP extension!
0
Kim WalkerWeb Programmer/TechnicianAuthor Commented:
You're absolutely right, Ray_Paseur. I felt the same way. Now it appears that the server has been serving the last rendered page instead of the error. It seemed that no matter what changes I made to my script, I was getting the exact same results. I just looked at my error logs and this is what I found repeated over and over.
[Sun Sep 01 16:41:41 2013] [warn] [client 67.213.33.1] mod_fcgid: read data timeout in 45 seconds
[Sun Sep 01 16:41:41 2013] [error] [client 67.213.33.1] Premature end of script headers: process_report.php

Open in new window

Only after stopping and restarting Apache am I again getting the timeout errors in my browser even after increasing the timeout to 5 minutes.

This would not have been a viable solution anyway when I start to get these reports every 30 minutes for 10-15 different clients.

Now it's time for my service provide to start providing better service!
0
Slick812Commented:
This may not help or matter for your purposes, but just because a file has a .zip extention OR it is called a "Zip" file by the creator, does not mean that it has a "Standard" zip compression, and file organization. Even in a "Standard" Zip file , there are several compression levels options  and there may be several options for the Kind or Type of compression used (you can even make a Zip file without any compression, like a linux TAR file). The computer program you use to decompress and recompress the wayward Zip file, has more "options" for a Zip file and can deal with the wayward Zip file, but the PHP ZIP you use for zip_open( ) has less options OR is meant to use with with a ZIP format that your wayward zip file IS NOT.  You may can see if the program that you successfully use to decompress the zip file has a "File properties or File analysis to use on this wayward zip file that may tell you that it is a Bzip2, lzf, or gz file or have some info about the properties for that zip, and maybe this will help you figure out what is the problem with this file. But the php Zip and ZipArchive do have less options than many OS Archive programs.
Also, You can produce a .zip file, that is really a program (executable) that will automatically decompress the enclosed archive file, and some Archive programs can still read this and get the archive with out using the "program" of that file. .
0

Experts Exchange Solution brought to you by

Your issues matter to us.

Facing a tech roadblock? Get the help and guidance you need from experienced professionals who care. Ask your question anytime, anywhere, with no hassle.

Start your 7-day free trial
Kim WalkerWeb Programmer/TechnicianAuthor Commented:
Good information, Slick812. Unfortunately, my 7z program doesn't give me any details of this nature. Can you suggest a free app that would?
0
Slick812Commented:
I have the 7z program and I know that it will do many, many ZIP file configurations, but I do not see any file analysys in that program. Sorry I can not recomend any programs at this time, and just can not take the time now, However you seem like a capable developer, and web searches are my greatest programming resource and "savior" in doing code work, , , some search for zip file analysis may turn up something.
the producer of this file may have some info about it, but I have seen some Bzip2 with a .zip extention, and I know that PHP has a separate decompression for Bzip2, worth a shot maybe?
0
Kim WalkerWeb Programmer/TechnicianAuthor Commented:
My service provider has responded that they use "7za" and included a file named 7za in their response. They also included a php file they described as "code that we have used to unzip." The php file contains nearly 750 lines of code. The only comments appear to be lines of code they've disabled. On line 713 is this function definition:
function unZipOnLinux($domain,$sourceFileName,$destinationPath){
  $destinationPath = $destinationPath.'/';
  $directoryPos = strrpos($sourceFileName,'/');
  $directory = substr($sourceFileName,0,$directoryPos+1);
  $dir = opendir( $directory );
  $info = pathinfo($sourceFileName);
  if ( strtolower($info['extension']) == 'zip' ) {
echo '7za e '.$sourceFileName .'  -o'. $destinationPath.'<br>';
   //system('unzip -q '.$sourceFileName .'  -d '. $destinationPath);
   system('/var/www/vhosts/'.$domain.'/httpdocs/dashboard/upload/7za e '.$sourceFileName .'  -o'. $destinationPath);
  }
  closedir( $dir );
}

Open in new window

Do I upload the 7za file to my working folder?

They appear to extract a path from the $sourceFileName variable (see lines 3 and 4 in the code above). So do I submit the entire path to the zipped file or just the relative path from the 7za file?

I've searched for examples or commentary of using 7za in php on linux and have come up empty. What little I have found uses shell_exec.

Any advise would be helpful.
0
Slick812Commented:
I read your last post and to me the "Important" line of code is this -
7za e '.$sourceFileName .'  -o'. $destinationPath

which seems to correspond to -
echo '7za e '.$sourceFileName .'  -o'. $destinationPath.'<br>';

this to me looks like it calls  a LINUX executable (7za) with  input-output parameters e and -o , , , by using system( )  and this was substituted for this line -
system('unzip -q '.$sourceFileName .'  -d '. $destinationPath);
  which uses the "Standard" unzip method.

I would think that the 7za is the 7z archive program for linux, and instead of using the .7z  file extension, they use the .zip extension.
I do know that the 7z archive file format is NOT compatible with the standard .zip archive file format. But you can set the 7z to do a DEFAULT "standard" zip file format, but apparently they did not do this.

for you to use this line  -
system('/var/www/vhosts/'.$domain.'/httpdocs/dashboard/upload/7za e '.$sourceFileName .'  -o'. $destinationPath);

on your server, you would need to have the 7za archive program installed , , and on a file path that your PHP can use with one of the PHP linux system functions calls, , like -
shell_exec('7za e '.$sourceFileName .'  -o'. $destinationPath);
exec('7za e '.$sourceFileName .'  -o'. $destinationPath);
system('7za e '.$sourceFileName .'  -o'. $destinationPath);
    Not all of these system functions are available (maybe none) in various PHP-LINUX setups, and they do vary somewhat in what is returned from the function.

In this system call it looks like the  7za executable is installed on this directory -
/var/www/vhosts/thisDomain/httpdocs/dashboard/upload/

I could be wrong about that, since this seems like a highly unusual place to to have an archive program in linux?

anyhow, if u is not so familiar wid de PHP LINUX system stuff, you might start out with the easy -
$output = shell_exec('ls -lart');
echo "<pre>$output</pre>"; // from manual , uses shell script  BASH "ls" to list files in a directory

just to see if the shell_exec( ) thing works. you can use the various linux install methods to get and or install 7z, But you may find better help for this than me in the EE Linux section. .

I just found this, which may shed some light -
https://www.ibm.com/developerworks/community/blogs/6e6f6d1b-95c3-46df-8a26-b7efd8ee4b57/entry/how_to_use_7zip_on_linux_command_line144?lang=en
http://www.dotnetperls.com/7-zip-examples
0
Kim WalkerWeb Programmer/TechnicianAuthor Commented:
I finally found a site with instructions for installing the p7zip package properly for my CentOS linux installation. With proper installation, I don't need to include a path to the 7za bin file. I can expand the archive with the following php command:
system('7za e /var/www/vhosts/domain.com/reports/report-*.zip');

Open in new window

This expands the files to the same folder as the archive where I can process them and delete them.
0
Kim WalkerWeb Programmer/TechnicianAuthor Commented:
I've split the points according to how much your comment contributed to my own resolution. I doubt if I'd have solved this without your comments. Thanks.
0
It's more than this solution.Get answers and train to solve all your tech problems - anytime, anywhere.Try it for free Edge Out The Competitionfor your dream job with proven skills and certifications.Get started today Stand Outas the employee with proven skills.Start learning today for free Move Your Career Forwardwith certification training in the latest technologies.Start your trial today
PHP

From novice to tech pro — start learning today.