How to create an embedded video player

Hello, I've got an at91sam9261 up and running, and it is working (more or less) with an LCD (640x480), and I can draw pixels and pictures to it just fine.  (Currently we are not using an OS, just writing C code off a main loop.)  Now we're thinking about what it would take to play video.  Say I started with a small file of AVI, MPG, WMV format, what do I need to do to play this on the LCD.  

Aren’t there any OS-agnostic (or OS-free) video libraries out there?   Making pixels appear on an LCD is fairly simple; once you’ve set the LCD CPU registers, it’s just writing numbers to memory and DMA takes care of the rest (i.e., copies to the LCD at the specified burst rate and pixels magically appear).   Hasn’t someone published a generic video library that boils down to

render_video_data_to_framebuffer(*input_avi_data, time_sec, *output_frame_buffer_origin);

which simply decompresses/decodes/scales a frame from the video data (you’d have to tell it the format too – oh, and maybe width and height?) and copies pixel data to the specified buffer.  And you’d call this repeatedly at the desired frame rate?  

The only solutions readily available are using an embedded OS and their video player.   Something like the function exists, obviously, buried in their video player.  Has anyone published raw C functions that would be OS-independent (and in fact could be used without an OS)?

Thanks for any insight.

Who is Participating?
I wear a lot of hats...

"The solutions and answers provided on Experts Exchange have been extremely helpful to me over the last few years. I wear a lot of hats - Developer, Database Administrator, Help Desk, etc., so I know a lot of things but not a lot about one thing. Experts Exchange gives me answers from people who do know a lot about one thing, in a easy to use platform." -Todd S.

I would be very suprised to find cross-platform solutions in C. I think you may be "luckier" with Java.
you may still try to run gstreamer:

Which probably is the nearest you get to a cross-platform API

satsumoSoftware DeveloperCommented:
You could try libmpeg2
First thing - i would recommend you to think over your overall idea of putting the whole load of managing data i/o, user interactions, (other actions like power management, etc ?), demuxing, decoding and outputing video to the µController. Maybe it could be a better idea to use some external hardware decoder solution taking into account that at91sam9261migt not be able to deliver a smooth performance - and by smooth i mean at least 23 fps unless you are targeting certain video formats. (please take some time and do reading here and here
If you still are determined to stick to your original solution(at91sam9261 based software decoding) then please proceed reading :)
Although unfortunately i was unable to find such library that would directly include the interface you were looking for, i still found some information that might help you in your project.
Frankly saying, i was not too sure what you are trying to get, so i developed two versions of what you would want to achieve:
A device that can be plugged/connected to computer/server via cable/wifi/other wireless means and would receive streaming video data (not any of the standard formats but some kind of raw display data) and display it on a lcd
A standalone video player
If i were to implement standalone video player based on arm architecture, i would first set some minor goals - like
outputting a picture to a lcd (which you have already done :))),
setting up and running some external hardware storage,
geting device to play some audio data,
getting device to play video and audio data,
playlist making, favorites, etc

As far as the audio i was able to find a project that have successefully implemented the  MPEG-1 Layer III : mp3 hardware play, but uses external decoder though

Refering to codecs as such - the video file itself holds sets of images in a color space format (like for example MPEG holds images that could be called "JPEG's in  Y'CbCr format"), and codec is basically something like a zip or rar standard - algorithm that tells how to compress and decompress those files. So basically to achieve your goal - you would have to display a set of images and play a sound while extracting them from an archive file. Sounds simple, right :P ?
Hope i gave you some points to start from :)

Experts Exchange Solution brought to you by

Your issues matter to us.

Facing a tech roadblock? Get the help and guidance you need from experienced professionals who care. Ask your question anytime, anywhere, with no hassle.

Start your 7-day free trial
Ultimate Tool Kit for Technology Solution Provider

Broken down into practical pointers and step-by-step instructions, the IT Service Excellence Tool Kit delivers expert advice for technology solution providers. Get your free copy now.

riceman0Author Commented:

Thank you very much.  Good pointers and thoughts.

wvuser1, you are correct about the 9261 and I actually gave you some outdated data, we usually use the 9261 but in this case we are using the 9M10, in part for the performance reason you refer to.

And looking at the 9M10 datasheet (sorry cant find link strangely but sure its there), apparently it has a "multi format" video decoder *peripheral*.   I don't know what this is about yet but its exciting -- maybe this is optimistic but perhaps they have put the codec in hardware, and all I have to do is (a) point it to source data (b) point it to LCD memory (c) tell it the format (d) painstakingly tune/configure it I'm sure, and presto!  Video appears on my LCD?

I'll have to read up on this -- but if this works anything like that maybe this is even better I was hoping for... I just have to write a software wrapper for this peripheral, and don't need third party software at all, nor do I have to work through an OS (provided I have access to the needed memories).

Am I dreaming?
satsumoSoftware DeveloperCommented:
I suspect you're dreaming.  Decoder hardware is usually something that assists the decoder rather than processes a whole data stream.  The clue in this case is 'multi format', it can do some of the work for any format.  It probably helps with DCT or something like that.
riceman0Author Commented:
It was a nice dream.

Although if it does something like, given a pointer to AVI data (and perhaps a frame number or time?), it decompresses (is that what DCT means?  not clear from wikipedia) and extracts and places raw pixel data at a location of my choosing, that is still a LOT of the work.  That's a nice dream too.  Or do you think it does even less -- what would you envision by "assists"?

Will read up on this soon, hopefully tonight; just trying to calibrate my expectations, it will probably take me a while to get my head around it and see if it all relates to my original question.

satsumoSoftware DeveloperCommented:
DCT is discrete cosine transform, a significant part of the number crunching in many compression schemes.  My guess would be that the hardware helps with DCT, doing a lot of the math in parallel with the CPU.  There may be an interface around the hardware that can process AVI data.

Mid you, I am guessing here, based on other chips made for optimising particular activities.  That chip could do many things, DCT just seems the obvious candidate.  It would be costly, difficult and restrictive to make a chip that processes AVI data specifically.  More useful and cheaper to create a general purpose chip that takes a chunk of the workload.
riceman0Author Commented:
Hm, video peripheral for the 9M10 seems pretty sophisticated.  It seems to do more than DCT -- there are specific formats it supports.  And you do provide references to memory (inputs outputs), the output seems to be pixel data, and you provide a frame index.  There are of course a zillion settings I don't understand yet, and haven't found a proper use case yet, will report back when I know what I'm talking about:

Hopefully we find the state of the art has advanced.  If you're curious,
satsumoSoftware DeveloperCommented:
That does looks hopeful, there a lot more than one chip involved though.  I'd be interested to know what you find out.  The last chip I used that had anything to do with video was basically a DCT decompressor.  However, that was quite a few years back.
riceman0Author Commented:
"there a lot more than one chip involved though"

Could you clarify what you mean by that?  I'm very much on the lookout for pitfalls.
satsumoSoftware DeveloperCommented:
I couldn't read the entire datasheet, as far as I could gather, there are many parts to the micro controller.  Chips for LCD display, USB, Ethernet, Cryptography, DMA etc.  It's says that the video decoder supports many formats, I couldn't find a page which explains how that support works.

The system I used (many years ago) supported video through a chip that was billed as a sound chip, it decoded DCT as part of MP3.  That meant it could be used for video too.  It had DMA paths to CPU RAM and video memory.  To get video decoding to work, the CPU had to schedule DMA from system memory into the chip and then from the chip to video RAM.

However, the system you're using is a different beast.  I'm interested to see what you find out.
riceman0Author Commented:
I will report back on this thread.  Thanks for the help so far.
riceman0Author Commented:
Looking like it might be a few days... will follow up with another thread.  Thanks all.
It's more than this solution.Get answers and train to solve all your tech problems - anytime, anywhere.Try it for free Edge Out The Competitionfor your dream job with proven skills and certifications.Get started today Stand Outas the employee with proven skills.Start learning today for free Move Your Career Forwardwith certification training in the latest technologies.Start your trial today

From novice to tech pro — start learning today.

Question has a verified solution.

Are you are experiencing a similar issue? Get a personalized answer when you ask a related question.

Have a better answer? Share it in a comment.