• Status: Solved
  • Priority: Medium
  • Security: Public
  • Views: 535
  • Last Modified:

How to display a progress bar of remote linux process

I am looking for a way to generate a progress bar for a process executing on a remote linux machine (through ssh). The process on the remote machine is a g++ compilation of a big C++ project.

A lot of output text is produced by the compilation process. I imagine to generate the progress bar by displaying (in the shell on the local machine) a "#" each time a line starting with "Making install in" comes out from the compilation process on the remote machine (I know how many such lines it will be in total, so I know how long the progress bar will be).

The remote process is started from a python script on the local machine, and I have currently come so far:

cmd="ssh "+rhost+" \"cd "+r_build_dir+" ; make -j install | grep 'Making install in'\""
# perhaps I could also have used: commands.getoutput(cmd) instead of os.system(cmd)

This kind of does what I want, but the whole point is lost because all the lines come out at once when the remote compilation process is done!, e.g.:

Making install in ProjectRoot
Making install in ProjectRoot/Project1
Making install in ProjectRoot/Project2
Making install in ProjectRoot/Project3

Of course the lines should be displayed one by one as the compilation progresses. Ideally I don't even want see those lines coming, but a growing progress bar.

The solution is probably related to flushing stdout and ssh. You experts probably have a "one-liner" to do the job!

1 Solution
You may want not to pipe the output into grep, but redirect it into the file, and fork a parallel process that would access the remote machine, read the redirected file and output progress as it goes.

You probably want to use the subprocess module and popen to catch the output as it comes in.  The os.system is just going to execute the command.

import re
from subprocess import Popen,PIPE

lookfor = re.compile(r'Making install in')

rhost = "somehost"
cmd = "ssh "+rhost+" \"cd "+r_build_dir+" ; make -j install"

build = Popen(cmd, stdout=PIPE, shell=True, universal_newlines = True, bufsize=1024 )

count = 0
while build.poll() == None:
        line = build.stdout.readline(1024).strip()
        if (lookfor.match(line)):
                count = count + 1
                print "%04d" % count

Open in new window

PhazzAuthor Commented:
Excellent, brilliant! Thanks a lot!
Question has a verified solution.

Are you are experiencing a similar issue? Get a personalized answer when you ask a related question.

Have a better answer? Share it in a comment.

Join & Write a Comment

Featured Post

A proven path to a career in data science

At Springboard, we know how to get you a job in data science. With Springboard’s Data Science Career Track, you’ll master data science  with a curriculum built by industry experts. You’ll work on real projects, and get 1-on-1 mentorship from a data scientist.

Tackle projects and never again get stuck behind a technical roadblock.
Join Now