Solved

Performance Vidoe Image Processing

Posted on 2001-06-30
13
277 Views
Last Modified: 2013-11-13
Hi,

Id like to perform image processing on a video sequence (e.g. avi) on a per frame basis. Id like to achieve results of around 25 fps output with an 16/24 bit colour 320x240 input sequence. Processing would include calculating differences between frames thresholding, averaging frames, performing custom mathematical operations on a per pixel basis. All this and more for each frame of hte sequence. The goal in the end is to do this in real time by feeding a live sequence from a capture card.

Has anyone done this sort of thing before? How should I go about implementing this? (language, APIs, SDKs etc...)

Note: I tried to implement this using Java with JMF and JAI, but the results were very poor.

Regards,
Hussan

Regards,
Hussan
0
Comment
Question by:hussan
  • 5
  • 3
  • 2
  • +2
13 Comments
 
LVL 22

Expert Comment

by:CJ_S
Comment Utility
I only have experience with image processing itself, writing my own filters which worked just like the ones Photoshop uses. The thing however is that it is pretty slow if you do it on a pixel-per-pixel basis. Depending on what kind of image processing you'd like to use it will execute fast or slow. The images I used were 800x600 and up, and it took 1 second to 2 seconds before it was finished.

If all you want to do is apply another image on the video sequence (as a filter laying above it) it will execute fast, (assuming you use RGBA).

You can use any language you want to accomplish this. My personal favorite is C++ for image processing, but any other language will do.
API's, it doesn't really matter, as long as you load the video sequence into memory and are able to retrieve the seperate frames. If you then just apply the filter or calculations ( the latter on a pixel-per-pixel basis) you are on a good way!!

You cannot optimize the program since any video sequence can be sent to your application...so no idea what kind of video it is.

Regards,
CJ
0
 
LVL 4

Expert Comment

by:ct.smith
Comment Utility
1 - per pixel operations should be avoided, too many high-cost calculations.  Try to rework your calculations into a convolution or interpolation or something.

2 - Take advantage of video hardware.  Some graphics APIs can move some of the manipulation work onto hardware.  Check out the OpenGL API for example.  This will probably give a major increase in speed if done well.

3 - The sort of goal you have set will likely require some heavy optimizations.  In my own imaging work,  I've seen max performance on a PC  for this sort of video at a whopping 1-2 frames per second without heavily optimized code.  To be sure, Java , while a nice language, is not upto this task, you shoud switch to C++, or even assembly.  There was a good article recently on www.ddj.com about this sort of thing, go check it out.
0
 

Author Comment

by:hussan
Comment Utility
Hi,

Thanks for the replies. I dont have a choice but to perform per pixel processing.

For example I need to apply the given formula on the current image:

global = current*(1-e^(-1/t)) + global*(e^(-1/t))

where global is an image. Surely the only way to do this would be per pixel.

Also surely using assembly would mean that my code would be platform specific and also processor specific eg MMX instructions differ to instructions on other procesors eg athlon. Or am i wrong thinking this.

Finally - if Im expecting a live feed then the frames will probably be just raw rgb data - therefore I could optimise.

Yours
Hussan
0
 
LVL 22

Expert Comment

by:CJ_S
Comment Utility
Well, the only way you could "optimize" is by doing memcpy for the whole block (if the life video sequence allows that). And that's not optimizing in image processing...

Regards,
CJ
0
 
LVL 4

Expert Comment

by:prashant_n_mhatre
Comment Utility
In my opinion...MATLAB is the best tool to do these things than other programming languages...
0
 
LVL 3

Accepted Solution

by:
ygal02 earned 200 total points
Comment Utility
From my experience in video processing:
MATLAB is the best tool for algorithm developing (my favourite) but absolutly not the fastest !
The method of developing such programs is usually like that:
1. Develop and check your algorithm on matlab.
2. Implement (or convert) your matlab code in C/C++
3. Optimize your C/C++ code.

About phase 3 of the process:
Java is absolutly not the right choice, the biggest known problem of Java is speed ! (otherwise it would probably be the main language by now...). With C or C++ code (and a good compiler) you can achieve results which are pretty close (in speed) to machine code. Don't forget to use all the compiler speed optimization options, and, ofcourse, the C/C++ optimization options (like templates, static variables, etc.).
If you want to achieve better results by restricting your self to Intel processors you can use the SSE library which allows you to perform mathematical operations in parallel (up to 4 variables at one CPU tick on single CPU). You can also consider using parallel machine...
I guess that the problem you introduced here can be solved easily using the above methods (and a fast CPU, memory & Board).
I don't know the purpose of your project, but if you can give a description of the images you are working with, I might have an idea for reducing the amount of calculations performed. For example, operations on images that contains a lot of zeros, can be reduces using blocks representation.
Let us know if you need more info.
Good Luck !
0
Highfive + Dolby Voice = No More Audio Complaints!

Poor audio quality is one of the top reasons people don’t use video conferencing. Get the crispest, clearest audio powered by Dolby Voice in every meeting. Highfive and Dolby Voice deliver the best video conferencing and audio experience for every meeting and every room.

 
LVL 4

Expert Comment

by:prashant_n_mhatre
Comment Utility
Converted MATLAB code is much faster than JAVA. Earlier MATCOM used to provide a nice library and compiler which was much better than MATLAB. The company has been merged with Mathworks now.

By the way, Matlab with its toolboxes and compiler is very expensive.
0
 
LVL 4

Expert Comment

by:ct.smith
Comment Utility
In case you don't want to fork out maoney for matlab, you can use the opensource 'Octave' (www.octave.org).  Similar syntax to matlab; howerver, it's not as fast or flexible, but it's a free download, so the price is right.  I've always used it for imaging where performance isn't an issue.
0
 
LVL 4

Expert Comment

by:prashant_n_mhatre
Comment Utility
Cheaper things like O-Matrix accepts MATLAB code and even produces dll or exe and performance is also good.

Octave...I never tried out but heard that it is good.
0
 

Author Comment

by:hussan
Comment Utility
Hi,

just a few clarifications - are there tools that can convert matlab code to c code autmatically? If yes which are the best.

Also which c compiler would you advise for performance.

I know nothing about optimisation - how do I get started?? Are there any books or artcles on the net.

Thanks once more for your help.
0
 
LVL 4

Expert Comment

by:prashant_n_mhatre
Comment Utility
Yes..Matlab C/C++ compiler does that.

MATLAB 'C/C++' compiler is very good and produces optimized code so you need not to worry about optimization.
It works well with Visual C++.

Matcom is no longer available. I guess Octave doesn't support code conversion to C/C++.
0
 
LVL 4

Expert Comment

by:ct.smith
Comment Utility
if you're new to optimization, then you should go by Dr. Knuth's famous adage: "Premature optimization is the root of all evil."
0
 
LVL 4

Expert Comment

by:prashant_n_mhatre
Comment Utility
well said ct.smith !!!

Generally compiler does that job better than us.
0

Featured Post

Highfive Gives IT Their Time Back

Highfive is so simple that setting up every meeting room takes just minutes and every employee will be able to start or join a call from any room with ease. Never be called into a meeting just to get it started again. This is how video conferencing should work!

Join & Write a Comment

A short article about problems I had with the new location API and permissions in Marshmallow
Although it can be difficult to imagine, someday your child will have a career of his or her own. He or she will likely start a family, buy a home and start having their own children. So, while being a kid is still extremely important, it’s also …
Viewers will learn how to properly install Eclipse with the necessary JDK, and will take a look at an introductory Java program. Download Eclipse installation zip file: Extract files from zip file: Download and install JDK 8: Open Eclipse and …
In this seventh video of the Xpdf series, we discuss and demonstrate the PDFfonts utility, which lists all the fonts used in a PDF file. It does this via a command line interface, making it suitable for use in programs, scripts, batch files — any pl…

728 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

9 Experts available now in Live!

Get 1:1 Help Now