Solved

Is it possible to build an ffmpeg based transcoding/render farm?

Posted on 2008-10-12
1
2,126 Views
Last Modified: 2013-11-13
Hello,

We transcode a lot of video (AVIs exported from Adobe Premiere CS3) to MPEG. Currently we are doing this using Adobe Encore on individual machines.

I'm wondering if it is possible to create a transcoding farm out of multiple machines using ffmpeg or any other similar software. Basically something similar to Dr. Queue but for video transcoding instead of 3D rendering.

Does anyone have such experience or thoughts on where to start?

Thanks in advance!
0
Comment
Question by:alexkink
1 Comment
 
LVL 40

Accepted Solution

by:
noci earned 250 total points
ID: 22705390
If you can break down a job into a set of file conversions.... then you only need a way to manage these files (distribute them, collect the conversions).

Some facility like: a set of spool directories to process a file move it to a local directory (same filesystem preferably) and convert it. And move the output to anothe spool directory.
You can use rsync (run from cron) or such tool to move the files around.

If you need a more event driven way you can create printer queues with a very special printing engine (ffmpeg -> whatever conversion) the post the result into another printqueue/mail etc.

mail can be used (if jobs are small enough) and if security is no issue. Jobs need to be <10MB ~ 6-7MB raw file size). to distribute.

You might want to investigate tools like:
Task spooler: (simple, single node)
http://vicerveza.homeunix.net/~viric/soft/ts/ 
OpenPBS: (more complex, network/cluster)
http://www-unix.mcs.anl.gov/openpbs/
PBSPro:
http://www.pbsgridworks.com/PBSTemp1.3.aspx?top_nav_name=Products&item_name=OpenPBS&top_nav_str=1

Generaly this is the type of software you are looking for: batch job scheduling preferably in a network.


0

Featured Post

Free Tool: Path Explorer

An intuitive utility to help find the CSS path to UI elements on a webpage. These paths are used frequently in a variety of front-end development and QA automation tasks.

One of a set of tools we're offering as a way of saying thank you for being a part of the community.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Suggested Solutions

Introduction We as admins face situation where we need to redirect websites to another. This may be required as a part of an upgrade keeping the old URL but website should be served from new URL. This document would brief you on different ways ca…
In this article, I'll explain how to setup a Plex Media Server (https://plex.tv/) on a Redhat (Centos) 7 based NAS with screenshots to help those looking for assistance.  What is Plex? If you aren't familiar with Plex, it’s a DLNA media serv…
This video will give a brief orientation and organization of Logic Pro X, and how to access different work spaces quickly with keyboard shortcuts.
Viewers will get an overview of how to make and use Drum Racks in Ableton Live. Load new Drum Rack into empty MIDI track: Fill rack with audio samples: Re-arrange sample slots as necessary: Adjust parameters of each slot to tailor each sound a…

860 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question