Solved

Is it possible to build an ffmpeg based transcoding/render farm?

Posted on 2008-10-12
1
2,032 Views
Last Modified: 2013-11-13
Hello,

We transcode a lot of video (AVIs exported from Adobe Premiere CS3) to MPEG. Currently we are doing this using Adobe Encore on individual machines.

I'm wondering if it is possible to create a transcoding farm out of multiple machines using ffmpeg or any other similar software. Basically something similar to Dr. Queue but for video transcoding instead of 3D rendering.

Does anyone have such experience or thoughts on where to start?

Thanks in advance!
0
Comment
Question by:alexkink
1 Comment
 
LVL 39

Accepted Solution

by:
noci earned 250 total points
Comment Utility
If you can break down a job into a set of file conversions.... then you only need a way to manage these files (distribute them, collect the conversions).

Some facility like: a set of spool directories to process a file move it to a local directory (same filesystem preferably) and convert it. And move the output to anothe spool directory.
You can use rsync (run from cron) or such tool to move the files around.

If you need a more event driven way you can create printer queues with a very special printing engine (ffmpeg -> whatever conversion) the post the result into another printqueue/mail etc.

mail can be used (if jobs are small enough) and if security is no issue. Jobs need to be <10MB ~ 6-7MB raw file size). to distribute.

You might want to investigate tools like:
Task spooler: (simple, single node)
http://vicerveza.homeunix.net/~viric/soft/ts/
OpenPBS: (more complex, network/cluster)
http://www-unix.mcs.anl.gov/openpbs/
PBSPro:
http://www.pbsgridworks.com/PBSTemp1.3.aspx?top_nav_name=Products&item_name=OpenPBS&top_nav_str=1

Generaly this is the type of software you are looking for: batch job scheduling preferably in a network.


0

Featured Post

What Should I Do With This Threat Intelligence?

Are you wondering if you actually need threat intelligence? The answer is yes. We explain the basics for creating useful threat intelligence.

Join & Write a Comment

Using 'screen' for session sharing, The Simple Edition Step 1: user starts session with command: screen Step 2: other user (logged in with same user account) connects with command: screen -x Done. Both users are connected to the same CLI sessio…
Keep your audience engaged and get the most out of your next presentation with these quick Prezi tips.
Viewers will learn how to turn a Live Set into a compressed Live Pack file, and how to install Live Packs. Make: File > Collect All And Save: File > Manage Files: Click Manage Project: Click Create Pack: Select save location: Install: Doub…
Viewers will learn the basics of creating custom device Racks in Ableton Live. Place instrument(s) and effects onto a track, and select them all by holding the Shift key and clicking on the device title bars: Group them by typing Command-G (Ctrl-G…

763 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

12 Experts available now in Live!

Get 1:1 Help Now