Welcome to Experts Exchange

Add your voice to the tech community where 5M+ people, just like you, are talking about what matters.

  • Help others & share knowledge
  • Earn cash & points
  • Learn & ask questions
Solved

How to get the output of 7zip piped to another program

Posted on 2009-04-05
3
2,489 Views
Last Modified: 2013-12-26
Hello Community,

I'm using Gentoo Linux and try to write a bash script that compresses a directory using 7zip. While compressing the script should show the progress using the 'dialog' tool.

First of all I count the number of files in that directory using 'find' and 'wc'. That's easy. After that I start 7zip to put the files in that directory into an archive. Until now, there is now problem. 7zip writes every file it processes to stdout and I can see it compressing file after file. The problem is that the shell seems to store the output when I try to pipe it to another process.

Simple Example:
7z a -bd archive.7z <dir-to-backup> | while read line ; do echo "${line}" ; done

When I execute this little script, I see nothing at first... but after 7zip has compressed some files all output is done at one go. After that there is no output for a certain time. 7zip works and consumes a lot of cpu time, but the output seems to be buffered in the pipe or somewhere else. After some time, I guess when the pipe is running full, 7zip's output becomes visible.

Neither zip nor tar does behave in that strange way. Their output can be seen line after line...

Does anyone know how I can solve that problem?
0
Comment
Question by:Metacrawler
3 Comments
 
LVL 7

Accepted Solution

by:
Morne Lategan earned 250 total points
ID: 24070912
Got the same on my side on both debian and Ubuntu. Tried to run it through pv which is a utility to do exactly what you are doing with progress:

7z a /tmp/m.7z <backupdir> | pv -s `find <backupdir> | wc -l` -l -n

pv also just gets to a point and then returns the same progress percentage for a long period of time before resuming.

Same happens with ksh, so my guess is that you'll have to contact the 7z developers and ask them to change the buffering behavior.

0
 
LVL 43

Expert Comment

by:ravenpl
ID: 24070918
Correct, 7z seems buffering stdout if it's not a terminal (man isatty).
0
 

Author Closing Comment

by:Metacrawler
ID: 31566721
Thank you very much. That was exactly what I was looking for. It works like a charm :-)
0

Featured Post

Efficient way to get backups off site to Azure

This user guide provides instructions on how to deploy and configure both a StoneFly Scale Out NAS Enterprise Cloud Drive virtual machine and Veeam Cloud Connect in the Microsoft Azure Cloud.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Suggested Solutions

It’s 2016. Password authentication should be dead — or at least close to dying. But, unfortunately, it has not traversed Quagga stage yet. Using password authentication is like laundering hotel guest linens with a washboard — it’s Passé.
Active Directory replication delay is the cause to many problems.  Here is a super easy script to force Active Directory replication to all sites with by using an elevated PowerShell command prompt, and a tool to verify your changes.
Learn how to find files with the shell using the find and locate commands. Use locate to find a needle in a haystack.: With locate, check if the file still exists.: Use find to get the actual location of the file.:
Learn how to navigate the file tree with the shell. Use pwd to print the current working directory: Use ls to list a directory's contents: Use cd to change to a new directory: Use wildcards instead of typing out long directory names: Use ../ to move…

790 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question