?
Solved

Is it possible to build an ffmpeg based transcoding/render farm?

Posted on 2008-10-12
1
Medium Priority
?
2,280 Views
Last Modified: 2013-11-13
Hello,

We transcode a lot of video (AVIs exported from Adobe Premiere CS3) to MPEG. Currently we are doing this using Adobe Encore on individual machines.

I'm wondering if it is possible to create a transcoding farm out of multiple machines using ffmpeg or any other similar software. Basically something similar to Dr. Queue but for video transcoding instead of 3D rendering.

Does anyone have such experience or thoughts on where to start?

Thanks in advance!
0
Comment
Question by:alexkink
[X]
Welcome to Experts Exchange

Add your voice to the tech community where 5M+ people just like you are talking about what matters.

  • Help others & share knowledge
  • Earn cash & points
  • Learn & ask questions
1 Comment
 
LVL 40

Accepted Solution

by:
noci earned 1000 total points
ID: 22705390
If you can break down a job into a set of file conversions.... then you only need a way to manage these files (distribute them, collect the conversions).

Some facility like: a set of spool directories to process a file move it to a local directory (same filesystem preferably) and convert it. And move the output to anothe spool directory.
You can use rsync (run from cron) or such tool to move the files around.

If you need a more event driven way you can create printer queues with a very special printing engine (ffmpeg -> whatever conversion) the post the result into another printqueue/mail etc.

mail can be used (if jobs are small enough) and if security is no issue. Jobs need to be <10MB ~ 6-7MB raw file size). to distribute.

You might want to investigate tools like:
Task spooler: (simple, single node)
http://vicerveza.homeunix.net/~viric/soft/ts/ 
OpenPBS: (more complex, network/cluster)
http://www-unix.mcs.anl.gov/openpbs/
PBSPro:
http://www.pbsgridworks.com/PBSTemp1.3.aspx?top_nav_name=Products&item_name=OpenPBS&top_nav_str=1

Generaly this is the type of software you are looking for: batch job scheduling preferably in a network.


0

Featured Post

On Demand Webinar - Networking for the Cloud Era

This webinar discusses:
-Common barriers companies experience when moving to the cloud
-How SD-WAN changes the way we look at networks
-Best practices customers should employ moving forward with cloud migration
-What happens behind the scenes of SteelConnect’s one-click button

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

You may or may not be familiar with Kodi (http://www.kodi.tv) the open source software media player and entertainment hub that can be installed on Linux, OSX, Windows, iOS, and Android.  Kodi which started off as X-Box Media Center (XBMC) many years…
The advancements in today's technology are unparalleled. Much of the technology that we have could not have been imagined twenty years ago. One of the latest additions to the list of technological advances is virtual reality. Virtual reality has an …
Connecting to an Amazon Linux EC2 Instance from Windows Using PuTTY.
Get a first impression of how PRTG looks and learn how it works.   This video is a short introduction to PRTG, as an initial overview or as a quick start for new PRTG users.
Suggested Courses
Course of the Month15 days, 2 hours left to enroll

771 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question