RonMexico
asked on
Computer vision -- quantifying "similarity" of colors
Hi, I'm experimenting a little with computer vision, image analysis, feature detection, etc. This question is fairly involved and I've done all the googling/web research I can do, and it probably requires someone with at least little experience in this area -- maybe I'll get lucky (wish I could offer more than 500 points).
I'm trying to analyze a photo and pick out the shapes/objects within the photo. I've mixed and matched with the standard algorithms that are typically used for this (sobol, canny, others), and now I'm stepping back and crunching some basic numbers and just trying to apply some common sense.
I took a photo of a ball on a table (attached) and tried to imagine an algorithm to find the ball. It is the big region of orange in the middle, but the first problem is that it is not perfectly uniform orange, it has a shadow across it so that it is brightest at the upper left and darker at the lower right. It's all the same color but varying brightness.
So I thought, how do I normalize on brightness so that the computer sees a blob of uniform orange in the middle? Well, if I take the RGB data and transform it to HSL, well then perhaps I should see the "luminance" (L) change across the ball and the H+S will stay the same. Then the computer could just modify all the colors to have the same "L" of the HSL and then we will see a relatively uniform orange.
So I sampled some pixels and did the RGB->HSL transform by hand (tabulated and color coded, also attached), and found that it does *not* work like that. I thought hue and saturation, or at LEAST the hue, would be uniform across the ball. It is not, the saturation and hue both drop significantly as the shadow increases.
Out of curiosity I sampled some points from the regions of the picture from the back wall (gray-ish) and the table it's sitting on (beige). I started to realize that I couldn't pick out anything numerically similar between the groups of colors (orange ball, gray wall, beige table). I would have thought that the hue, or hue+saturation stayed the same while maybe the luminance changes with the brightness.
So... if that is not the rule, what IS the rule? What is the math that would group these sets of points together? Maybe there is some other transform other than RGB-HSL that makes it apparent? Obviously there is something, since it is IMMEDIATELY obvious to our eyes: those are orange, those are gray, and those are beige (of varying brightnesses).
Thanks very much for reading this far, and any thoughts. Like I said, I have already tried googling and web research, I probably need to hear from someone who has actually gone a little further down this road I'm *just* starting on.
ball.png
tabulated-RGB--HSL.png
I'm trying to analyze a photo and pick out the shapes/objects within the photo. I've mixed and matched with the standard algorithms that are typically used for this (sobol, canny, others), and now I'm stepping back and crunching some basic numbers and just trying to apply some common sense.
I took a photo of a ball on a table (attached) and tried to imagine an algorithm to find the ball. It is the big region of orange in the middle, but the first problem is that it is not perfectly uniform orange, it has a shadow across it so that it is brightest at the upper left and darker at the lower right. It's all the same color but varying brightness.
So I thought, how do I normalize on brightness so that the computer sees a blob of uniform orange in the middle? Well, if I take the RGB data and transform it to HSL, well then perhaps I should see the "luminance" (L) change across the ball and the H+S will stay the same. Then the computer could just modify all the colors to have the same "L" of the HSL and then we will see a relatively uniform orange.
So I sampled some pixels and did the RGB->HSL transform by hand (tabulated and color coded, also attached), and found that it does *not* work like that. I thought hue and saturation, or at LEAST the hue, would be uniform across the ball. It is not, the saturation and hue both drop significantly as the shadow increases.
Out of curiosity I sampled some points from the regions of the picture from the back wall (gray-ish) and the table it's sitting on (beige). I started to realize that I couldn't pick out anything numerically similar between the groups of colors (orange ball, gray wall, beige table). I would have thought that the hue, or hue+saturation stayed the same while maybe the luminance changes with the brightness.
So... if that is not the rule, what IS the rule? What is the math that would group these sets of points together? Maybe there is some other transform other than RGB-HSL that makes it apparent? Obviously there is something, since it is IMMEDIATELY obvious to our eyes: those are orange, those are gray, and those are beige (of varying brightnesses).
Thanks very much for reading this far, and any thoughts. Like I said, I have already tried googling and web research, I probably need to hear from someone who has actually gone a little further down this road I'm *just* starting on.
ball.png
tabulated-RGB--HSL.png
Here are some commonly used metrics: http://en.wikipedia.org/wiki/Color_difference
But it's actually much more complicated: http://people.virginia.edu/~smb3u/ColorVision2/ColorVision2.html
You may find this to be a practical approximation: http://en.wikipedia.org/wiki/Lab_color_space
But it's actually much more complicated: http://people.virginia.edu/~smb3u/ColorVision2/ColorVision2.html
You may find this to be a practical approximation: http://en.wikipedia.org/wiki/Lab_color_space
ASKER
Yep, seen those links. Played with most of the math it describes. What do you mean a practical approximation?
To summarize, there must be a transform that groups my data in the desired fashion (orange vs beige vs white) regardless of brightness. I thought it was hue or hue+saturation but apparently it is not.
And frankly the transform can't be much more complicated than hsv or hsi or hsl transforms because there's not that much input data. There are some simple equations I could plug into the next excel cells that would make orange and dark orange numerically similar.
There are a million things I could try, could really benefit from some experience here.
To summarize, there must be a transform that groups my data in the desired fashion (orange vs beige vs white) regardless of brightness. I thought it was hue or hue+saturation but apparently it is not.
And frankly the transform can't be much more complicated than hsv or hsi or hsl transforms because there's not that much input data. There are some simple equations I could plug into the next excel cells that would make orange and dark orange numerically similar.
There are a million things I could try, could really benefit from some experience here.
SOLUTION
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
ASKER
Okay, I'll try that.
Can't you just do a threshold on the Saturation channel? Pick an appropriate threshold level, say 100 maybe and then any pixel with saturation over that is the ball.
SOLUTION
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
ASKER
@satsumo
"The hue will change in the shade because it gets more colour from the diffuse light."
Thanks, this is the exact thing I didn't know, I thought "hue" was a property of the ball. If two pixels are from the same object but are shaded differently, is there some calculable color property that does *not* change? (I haven't tried the L in L*a*b lab space yet, but it's something to try... however you sound like you know something about this, would you suggest something different or do you know that there is no such property that is unaffected by the shade?)
I can't subtract a known reference object because my actual problem is to find any object. So all I'm trying to do at the moment as step 0 is to find regions of similar color that might represent an object in space. And my immediate problem is simply that "similar" should disregard shading/shadows.
"The hue will change in the shade because it gets more colour from the diffuse light."
Thanks, this is the exact thing I didn't know, I thought "hue" was a property of the ball. If two pixels are from the same object but are shaded differently, is there some calculable color property that does *not* change? (I haven't tried the L in L*a*b lab space yet, but it's something to try... however you sound like you know something about this, would you suggest something different or do you know that there is no such property that is unaffected by the shade?)
I can't subtract a known reference object because my actual problem is to find any object. So all I'm trying to do at the moment as step 0 is to find regions of similar color that might represent an object in space. And my immediate problem is simply that "similar" should disregard shading/shadows.
Object recognition involves a lot more than quantifying "similarity" of colors.
One way to reduce the effect of differences in illumination is to suppress the low frequencies in the log domain
http://www.cs.sfu.ca/~stella/papers/blairthesis/main/node35.html
One way to reduce the effect of differences in illumination is to suppress the low frequencies in the log domain
http://www.cs.sfu.ca/~stella/papers/blairthesis/main/node35.html
ASKER CERTIFIED SOLUTION
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
ASKER
"Object recognition involves a lot more than quantifying "similarity" of colors."
Obviously.
Obviously.
ASKER
Thanks mccarl, that helps a lot -- I understand (belatedly, perhaps) from what you're saying that the color of light bouncing off the ball and towards the camera is a function of both the ball and -- significantly -- the light source(s), so it is quite possible the shaded area appears quite different in these numbers available to me. I won't waste time looking for a magic discriminator then, and fall back to fuzzier/tuned techniques.
Different backgrounds too, but otherwise that's a nice suggestion. I'm biting off more than I can chew, but I like to do that sometimes.
Different backgrounds too, but otherwise that's a nice suggestion. I'm biting off more than I can chew, but I like to do that sometimes.
Glad to help & good luck! :)
Something you might try something like the PlayStation EyeToy does. It assumes the camera is not moving. So it can compare one image to the previous image, any difference is movement. In your case, the background may be different but it does not change between frames. As long as the target objects move enough to give you a reference you should be able to detect the difference.
However, identifying non-specific objects without a reference is a very difficult problem. The best solution I've seen for that was by Google. They used their image search data to match objects against. To locate a static object against static background is similarly tricky, because the program has no idea what the object is. This is why early computer vision research used things like colored balls.
Thanks for the points.
However, identifying non-specific objects without a reference is a very difficult problem. The best solution I've seen for that was by Google. They used their image search data to match objects against. To locate a static object against static background is similarly tricky, because the program has no idea what the object is. This is why early computer vision research used things like colored balls.
Thanks for the points.
http://web.mit.edu/persci/people/adelson/checkershadow_illusion.html