How to display a progress bar of remote linux process

Posted on 2011-09-30
Last Modified: 2012-05-12
I am looking for a way to generate a progress bar for a process executing on a remote linux machine (through ssh). The process on the remote machine is a g++ compilation of a big C++ project.

A lot of output text is produced by the compilation process. I imagine to generate the progress bar by displaying (in the shell on the local machine) a "#" each time a line starting with "Making install in" comes out from the compilation process on the remote machine (I know how many such lines it will be in total, so I know how long the progress bar will be).

The remote process is started from a python script on the local machine, and I have currently come so far:

cmd="ssh "+rhost+" \"cd "+r_build_dir+" ; make -j install | grep 'Making install in'\""
# perhaps I could also have used: commands.getoutput(cmd) instead of os.system(cmd)

This kind of does what I want, but the whole point is lost because all the lines come out at once when the remote compilation process is done!, e.g.:

Making install in ProjectRoot
Making install in ProjectRoot/Project1
Making install in ProjectRoot/Project2
Making install in ProjectRoot/Project3

Of course the lines should be displayed one by one as the compilation progresses. Ideally I don't even want see those lines coming, but a growing progress bar.

The solution is probably related to flushing stdout and ssh. You experts probably have a "one-liner" to do the job!

Question by:Phazz

Expert Comment

ID: 36894867
You may want not to pipe the output into grep, but redirect it into the file, and fork a parallel process that would access the remote machine, read the redirected file and output progress as it goes.

LVL 25

Accepted Solution

clockwatcher earned 500 total points
ID: 36900037
You probably want to use the subprocess module and popen to catch the output as it comes in.  The os.system is just going to execute the command.

import re
from subprocess import Popen,PIPE

lookfor = re.compile(r'Making install in')

rhost = "somehost"
cmd = "ssh "+rhost+" \"cd "+r_build_dir+" ; make -j install"

build = Popen(cmd, stdout=PIPE, shell=True, universal_newlines = True, bufsize=1024 )

count = 0
while build.poll() == None:
        line = build.stdout.readline(1024).strip()
        if (lookfor.match(line)):
                count = count + 1
                print "%04d" % count

Open in new window


Author Closing Comment

ID: 36908296
Excellent, brilliant! Thanks a lot!

Featured Post

Master Your Team's Linux and Cloud Stack!

The average business loses $13.5M per year to ineffective training (per 1,000 employees). Keep ahead of the competition and combine in-person quality with online cost and flexibility by training with Linux Academy.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Suggested Solutions

Title # Comments Views Activity
Convert grep lines to perl 6 59
Linux boot cd to do hardware report on PC? 3 64 default error log 1 40
Slow computer- outside access? 14 52
Strings in Python are the set of characters that, once defined, cannot be changed by any other method like replace. Even if we use the replace method it still does not modify the original string that we use, but just copies the string and then modif…
I. Introduction There's an interesting discussion going on now in an Experts Exchange Group — Attachments with no extension ( This reminded me of questions tha…
Learn the basics of lists in Python. Lists, as their name suggests, are a means for ordering and storing values. : Lists are declared using brackets; for example: t = [1, 2, 3]: Lists may contain a mix of data types; for example: t = ['string', 1, T…
Learn the basics of while and for loops in Python.  while loops are used for testing while, or until, a condition is met: The structure of a while loop is as follows:     while <condition>:         do something         repeate: The break statement m…

820 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question