Solved

How to start background processes in Python?

Posted on 2014-12-24
3
305 Views
Last Modified: 2014-12-28
Hello,

I'm trying to
  * read a file
  * write some data into a folder
  * start another process to process the data
  * resume reading the file while the process works on the data

Is there a good way to go about doing this? Would it be better than starting a bunch of threads?

The reason I'm thinking about it this way, is that I don't really need the processes to communicate with each other, once started, the processes will be working on their own set of data; and I'm hoping Unix would manage running multiple scripts better than Python would manage threads.

Appreciate any advice on this!
0
Comment
Question by:Errang Genevre
3 Comments
 
LVL 74

Accepted Solution

by:
käµfm³d   👽 earned 350 total points
ID: 40517414
You spawn a process by invoking the subprocess function.

You have a potential race condition in that the second process won't know exactly when the Python process is done writing the file (unless these are fixed-size files), so you might want to use a convention like writing a zero-byte sentinel file to alert the subprocess as to when the file is done being written. Unless, of course, you won't be kicking off the 2nd process until the Python process finishes writing the file.
0
 
LVL 25

Assisted Solution

by:clockwatcher
clockwatcher earned 150 total points
ID: 40518156
Might be a little overkill for what you're after but it sounds like a good candidate for a message queuing system:  https://www.rabbitmq.com/tutorials/tutorial-two-python.html
0
 

Author Closing Comment

by:Errang Genevre
ID: 40520677
Thanks!
0

Featured Post

How your wiki can always stay up-to-date

Quip doubles as a “living” wiki and a project management tool that evolves with your organization. As you finish projects in Quip, the work remains, easily accessible to all team members, new and old.
- Increase transparency
- Onboard new hires faster
- Access from mobile/offline

Join & Write a Comment

Sequence is something that used to store data in it in very simple words. Let us just create a list first. To create a list first of all we need to give a name to our list which I have taken as “COURSE” followed by equals sign and finally enclosed …
Utilizing an array to gracefully append to a list of EmailAddresses
Learn the basics of if, else, and elif statements in Python 2.7. Use "if" statements to test a specified condition.: The structure of an if statement is as follows: (CODE) Use "else" statements to allow the execution of an alternative, if the …
Learn the basics of modules and packages in Python. Every Python file is a module, ending in the suffix: .py: Modules are a collection of functions and variables.: Packages are a collection of modules.: Module functions and variables are accessed us…

707 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

17 Experts available now in Live!

Get 1:1 Help Now