?
Solved

Performance Vidoe Image Processing

Posted on 2001-06-30
13
Medium Priority
?
282 Views
Last Modified: 2013-11-13
Hi,

Id like to perform image processing on a video sequence (e.g. avi) on a per frame basis. Id like to achieve results of around 25 fps output with an 16/24 bit colour 320x240 input sequence. Processing would include calculating differences between frames thresholding, averaging frames, performing custom mathematical operations on a per pixel basis. All this and more for each frame of hte sequence. The goal in the end is to do this in real time by feeding a live sequence from a capture card.

Has anyone done this sort of thing before? How should I go about implementing this? (language, APIs, SDKs etc...)

Note: I tried to implement this using Java with JMF and JAI, but the results were very poor.

Regards,
Hussan

Regards,
Hussan
0
Comment
Question by:hussan
[X]
Welcome to Experts Exchange

Add your voice to the tech community where 5M+ people just like you are talking about what matters.

  • Help others & share knowledge
  • Earn cash & points
  • Learn & ask questions
  • 5
  • 3
  • 2
  • +2
13 Comments
 
LVL 22

Expert Comment

by:CJ_S
ID: 6240934
I only have experience with image processing itself, writing my own filters which worked just like the ones Photoshop uses. The thing however is that it is pretty slow if you do it on a pixel-per-pixel basis. Depending on what kind of image processing you'd like to use it will execute fast or slow. The images I used were 800x600 and up, and it took 1 second to 2 seconds before it was finished.

If all you want to do is apply another image on the video sequence (as a filter laying above it) it will execute fast, (assuming you use RGBA).

You can use any language you want to accomplish this. My personal favorite is C++ for image processing, but any other language will do.
API's, it doesn't really matter, as long as you load the video sequence into memory and are able to retrieve the seperate frames. If you then just apply the filter or calculations ( the latter on a pixel-per-pixel basis) you are on a good way!!

You cannot optimize the program since any video sequence can be sent to your application...so no idea what kind of video it is.

Regards,
CJ
0
 
LVL 4

Expert Comment

by:ct.smith
ID: 6241106
1 - per pixel operations should be avoided, too many high-cost calculations.  Try to rework your calculations into a convolution or interpolation or something.

2 - Take advantage of video hardware.  Some graphics APIs can move some of the manipulation work onto hardware.  Check out the OpenGL API for example.  This will probably give a major increase in speed if done well.

3 - The sort of goal you have set will likely require some heavy optimizations.  In my own imaging work,  I've seen max performance on a PC  for this sort of video at a whopping 1-2 frames per second without heavily optimized code.  To be sure, Java , while a nice language, is not upto this task, you shoud switch to C++, or even assembly.  There was a good article recently on www.ddj.com about this sort of thing, go check it out.
0
 

Author Comment

by:hussan
ID: 6241568
Hi,

Thanks for the replies. I dont have a choice but to perform per pixel processing.

For example I need to apply the given formula on the current image:

global = current*(1-e^(-1/t)) + global*(e^(-1/t))

where global is an image. Surely the only way to do this would be per pixel.

Also surely using assembly would mean that my code would be platform specific and also processor specific eg MMX instructions differ to instructions on other procesors eg athlon. Or am i wrong thinking this.

Finally - if Im expecting a live feed then the frames will probably be just raw rgb data - therefore I could optimise.

Yours
Hussan
0
Get real performance insights from real users

Key features:
- Total Pages Views and Load times
- Top Pages Viewed and Load Times
- Real Time Site Page Build Performance
- Users’ Browser and Platform Performance
- Geographic User Breakdown
- And more

 
LVL 22

Expert Comment

by:CJ_S
ID: 6241995
Well, the only way you could "optimize" is by doing memcpy for the whole block (if the life video sequence allows that). And that's not optimizing in image processing...

Regards,
CJ
0
 
LVL 4

Expert Comment

by:prashant_n_mhatre
ID: 6244818
In my opinion...MATLAB is the best tool to do these things than other programming languages...
0
 
LVL 3

Accepted Solution

by:
ygal02 earned 600 total points
ID: 6245388
From my experience in video processing:
MATLAB is the best tool for algorithm developing (my favourite) but absolutly not the fastest !
The method of developing such programs is usually like that:
1. Develop and check your algorithm on matlab.
2. Implement (or convert) your matlab code in C/C++
3. Optimize your C/C++ code.

About phase 3 of the process:
Java is absolutly not the right choice, the biggest known problem of Java is speed ! (otherwise it would probably be the main language by now...). With C or C++ code (and a good compiler) you can achieve results which are pretty close (in speed) to machine code. Don't forget to use all the compiler speed optimization options, and, ofcourse, the C/C++ optimization options (like templates, static variables, etc.).
If you want to achieve better results by restricting your self to Intel processors you can use the SSE library which allows you to perform mathematical operations in parallel (up to 4 variables at one CPU tick on single CPU). You can also consider using parallel machine...
I guess that the problem you introduced here can be solved easily using the above methods (and a fast CPU, memory & Board).
I don't know the purpose of your project, but if you can give a description of the images you are working with, I might have an idea for reducing the amount of calculations performed. For example, operations on images that contains a lot of zeros, can be reduces using blocks representation.
Let us know if you need more info.
Good Luck !
0
 
LVL 4

Expert Comment

by:prashant_n_mhatre
ID: 6245455
Converted MATLAB code is much faster than JAVA. Earlier MATCOM used to provide a nice library and compiler which was much better than MATLAB. The company has been merged with Mathworks now.

By the way, Matlab with its toolboxes and compiler is very expensive.
0
 
LVL 4

Expert Comment

by:ct.smith
ID: 6245560
In case you don't want to fork out maoney for matlab, you can use the opensource 'Octave' (www.octave.org).  Similar syntax to matlab; howerver, it's not as fast or flexible, but it's a free download, so the price is right.  I've always used it for imaging where performance isn't an issue.
0
 
LVL 4

Expert Comment

by:prashant_n_mhatre
ID: 6245688
Cheaper things like O-Matrix accepts MATLAB code and even produces dll or exe and performance is also good.

Octave...I never tried out but heard that it is good.
0
 

Author Comment

by:hussan
ID: 6255608
Hi,

just a few clarifications - are there tools that can convert matlab code to c code autmatically? If yes which are the best.

Also which c compiler would you advise for performance.

I know nothing about optimisation - how do I get started?? Are there any books or artcles on the net.

Thanks once more for your help.
0
 
LVL 4

Expert Comment

by:prashant_n_mhatre
ID: 6255631
Yes..Matlab C/C++ compiler does that.

MATLAB 'C/C++' compiler is very good and produces optimized code so you need not to worry about optimization.
It works well with Visual C++.

Matcom is no longer available. I guess Octave doesn't support code conversion to C/C++.
0
 
LVL 4

Expert Comment

by:ct.smith
ID: 6256167
if you're new to optimization, then you should go by Dr. Knuth's famous adage: "Premature optimization is the root of all evil."
0
 
LVL 4

Expert Comment

by:prashant_n_mhatre
ID: 6256185
well said ct.smith !!!

Generally compiler does that job better than us.
0

Featured Post

Optimize your web performance

What's in the eBook?
- Full list of reasons for poor performance
- Ultimate measures to speed things up
- Primary web monitoring types
- KPIs you should be monitoring in order to increase your ROI

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

A short article about problems I had with the new location API and permissions in Marshmallow
Today, the web development industry is booming, and many people consider it to be their vocation. The question you may be asking yourself is – how do I become a web developer?
In this fifth video of the Xpdf series, we discuss and demonstrate the PDFdetach utility, which is able to list and, more importantly, extract attachments that are embedded in PDF files. It does this via a command line interface, making it suitable …
In this seventh video of the Xpdf series, we discuss and demonstrate the PDFfonts utility, which lists all the fonts used in a PDF file. It does this via a command line interface, making it suitable for use in programs, scripts, batch files — any pl…

800 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question