testing the performance of a decoder

Given many choices of decoder to use, it is very hard to determine the performance of each decoder in an objective way.

Currently I am writing an application using directshow which will capture frame rates output by the decoder. Its use is to measure the performance of the decoder, since a bad performance decoder will overload the cpu, therefore will drop frames producing low frame rates.

Is there any other good/popular way to test the performance of a decoder, and what conclusion can be drawn from that particular result?

Who is Participating?
I wear a lot of hats...

"The solutions and answers provided on Experts Exchange have been extremely helpful to me over the last few years. I wear a lot of hats - Developer, Database Administrator, Help Desk, etc., so I know a lot of things but not a lot about one thing. Experts Exchange gives me answers from people who do know a lot about one thing, in a easy to use platform." -Todd S.

Use QueryPerformanceCounter and set up a test to decode a number of frames of a movie.
>>  Its use is to measure the performance of the decoder,

Are you talking about the performance of the decoding process ? Or of the encoded video ?
fungi8210Author Commented:
I'm talking about the performance of the decoding process, in particular, the performance of the decoder
Learn SQL Server Core 2016

This course will introduce you to SQL Server Core 2016, as well as teach you about SSMS, data tools, installation, server configuration, using Management Studio, and writing and executing queries.

If you use QPC and set up an adequate test environment then you should be able to yield very accurate results.
If you are purely looking at runtime performance, and not at the quality of the generated video, then you'll have to either :

1) ensure that the generated videos are (almost) exactly the same for all decoders
2) or take differences in quality of the generated video into account by adding a "quality factor" to the "performance value"

If the output of the different encoders is the same (ie. none of them have dropped frames, and all are encoded with the same quality settings, and the visual quality is similar for all of them), then it's simply a matter of measuring the time the encoders take.

If the output is different (ie. some encoders drop frames, different quality settings were used and/or the visual quality is visibly different), then you'll have to quantize the different quality parameters, and add those in the calculation. This quantization will mostly depend on what importance you attribute to them. For example, a few dropped frames might not be bad, as long as the visual quality isn't affected, etc.

btw, are we talking about lossless or lossy encodings here ?

Experts Exchange Solution brought to you by

Your issues matter to us.

Facing a tech roadblock? Get the help and guidance you need from experienced professionals who care. Ask your question anytime, anywhere, with no hassle.

Start your 7-day free trial
note : halfway through my last mail, I switched from the word "decoder" to "encoder". Just read "decoder" everywhere :)
fungi8210Author Commented:
by the way, my decoder is running on windows mobile platform
fungi8210Author Commented:
to Infinity08:

it is used in video telephony application so i guess it is a lossy encoding
>> it is used in video telephony application so i guess it is a lossy encoding
Not necessarily - it depends how the image is acquired (ie. the resolution of the camera I imagine).

What you could do to prevent that loss of frames is a factor, is to run the decoder in a "laboratory" environment. Meaning that you don't decode in real-time. You just let the decoder do its word without any loss of frames, and you time how long it takes.

A lot depends (as I already mentioned) on what you think is more important when it comes to quality.

Preferrably, you'd do several tests for each decoder :

1) pure performance (frames per second with similar quality settings)
2) quality : crispness of image
3) quality : fluidness of image (not too much dropped frames)
4) quality : color correctness of image

and maybe other tests. Then you attribute a weight to each of these tests, and the final decoder "performance" can then be calculated like this for example :

    weight1 * result1 + weight2 * result2 + weight3 * result3 + weight4 * result4
It's more than this solution.Get answers and train to solve all your tech problems - anytime, anywhere.Try it for free Edge Out The Competitionfor your dream job with proven skills and certifications.Get started today Stand Outas the employee with proven skills.Start learning today for free Move Your Career Forwardwith certification training in the latest technologies.Start your trial today
Game Programming

From novice to tech pro — start learning today.