?
Solved

OpenGL picking (hit list)

Posted on 2009-04-28
19
Medium Priority
?
1,052 Views
Last Modified: 2013-12-21
I'm trying to create a hit list for selecting objects in OpenGL.  Problem is, if an object is on the viewport, it's being recognized as a hit, whether I'm clicking the object or not.  Any help is appreciated.
void startPicking(int x, int y) {
	GLint viewport[4];
 
	glSelectBuffer(512, selectBuf);
	glRenderMode(GL_SELECT);
 
	glMatrixMode(GL_PROJECTION);
	glPushMatrix();
	glLoadIdentity();
 
	glGetIntegerv(GL_VIEWPORT, viewport);
	gluPickMatrix(x, viewport[3]-y, 5, 5, viewport);
	gluPerspective(45, xy_aspect, 0.1, 1000);
	glMatrixMode(GL_MODELVIEW);
	glInitNames();
}
 
void processHits(GLint hits, GLuint buffer[])
{
   int i, j;
   int names, *ptr, minZ, *ptrNames, numberOfNames;
 
   printf ("hits = %d\n", hits);
   ptr = (int *) buffer;
   minZ = 0xffffffff;
   for (i = 0; i < hits; i++) {	
      names = *ptr;
	  ptr++;
	  if (*ptr < minZ) {
		  numberOfNames = names;
		  minZ = *ptr;
		  ptrNames = ptr+2;
	  }
	  
	  ptr += names+2;
	}
  printf ("The closest hit names are ");
  ptr = ptrNames;
  for (j = 0; j < numberOfNames; j++,ptr++) {
     printf ("%d ", *ptr);
  }
  printf ("\n");
   
}
 
void stopPicking(int x, int y) {
	int hits;
	
	// restoring the original projection matrix
	glMatrixMode(GL_PROJECTION);
	glPopMatrix();
	glMatrixMode(GL_MODELVIEW);
	glFlush();
	
	// returning to normal rendering mode
	hits = glRenderMode(GL_RENDER);
	
	// if there are hits process them
	if (hits != 0)
		processHits(hits,selectBuf);
}
 
void mouse(int button, int state, int x, int y) {
	if(button==GLUT_LEFT_BUTTON) {
		switch(state) {
			case GLUT_DOWN:
				startPicking(x, y);
				break;
			case GLUT_UP:
				stopPicking(x, y);
				break;
		}
	}
}
 
void display(void) {
	glClearColor( .9f, .9f, .9f, 1.0f );
	glClear(GL_COLOR_BUFFER_BIT | GL_DEPTH_BUFFER_BIT);
 
	glMatrixMode(GL_PROJECTION);
	glLoadIdentity();
	gluPerspective(45, xy_aspect, 0.1, 1000);
 
	glMatrixMode(GL_MODELVIEW);
	...
	glPushName(0);
	glutSolidSphere(...);
	glPopName();
	...
}

Open in new window

0
Comment
Question by:palehose
  • 10
  • 7
17 Comments
 

Author Comment

by:palehose
ID: 24257513
I've made some progress on this, and actually have somewhat of a solution.  The only problem is the coordinates somehow appear to be off.  It says I'm picking an object outside of the actual object.  Anyone know why?
0
 
LVL 20

Expert Comment

by:ikework
ID: 24257657
hi palehose,

can you post the entire code?

ike
0
 

Author Comment

by:palehose
ID: 24257710
Hi ikework,
I uploaded it here: http://www.tomsisk.com/Modeler.cpp if that's okay.

Thanks for the help.
0
Concerto's Cloud Advisory Services

Want to avoid the missteps to gaining all the benefits of the cloud? Learn more about the different assessment options from our Cloud Advisory team.

 
LVL 20

Expert Comment

by:ikework
ID: 24257714
you dont pass the mouse-x and -y position to gluPickMatrix here:

  gluPickMatrix(x, viewport[3]-y, 5, 5, viewport);

should be something like:

  gluPickMatrix((GLdouble) mouse_x, (GLdouble) (viewport[3]-mouse_y), 1.0f, 1.0f, viewport);

have a look at:

  http://nehe.gamedev.net/data/lessons/lesson.asp?lesson=32

for details
0
 
LVL 20

Expert Comment

by:ikework
ID: 24257728
this code, you call twice, once in startPicking and once in stopPicking

      glMatrixMode(GL_PROJECTION);
      glPopMatrix();
      glMatrixMode(GL_MODELVIEW);
      glFlush();

         int hits = glRenderMode(GL_RENDER);
      if(hits != 0) processHits(hits, selectBuf);

it should only be called once per picking-loop
0
 

Author Comment

by:palehose
ID: 24257740
I am passing the mouse information, that's what x and y is.  It's passed from the mouse click.

And my mistake on the second issue.  stopPicking isn't even being used, I just haven't deleted it.
0
 

Author Comment

by:palehose
ID: 24257753
Again, my problem is what is considered clicking the object.  It's a little off (clicking higher than the object is considered picking, and just below the middle of the object, or so, is not considered picking.
0
 
LVL 20

Expert Comment

by:ikework
ID: 24257773
>> I am passing the mouse information, that's what x and y is.  It's passed from the mouse click.

right, sorry .. too early .. need some coffee .. ;)

>> It's a little off (clicking higher than the object is considered picking, and just below the middle of the object, or so, is not considered picking.

ok .. i see .. gonna check that
0
 

Author Comment

by:palehose
ID: 24258083
Any luck finding what's wrong?  I'm getting frustrated here.
0
 
LVL 20

Expert Comment

by:ikework
ID: 24258209
im at work right now, gonna have a look at it later.. can you make a smaller sample-app without gui, which has the same issue? if you can upload it, i can have a deeper look at it
0
 

Author Comment

by:palehose
ID: 24263203
Haven't had time to do that, but I'm still troubleshooting this and it appears that the object is in a different place then it is rendered.  I currently have it set up so it returns the color of the pixel that I click, and it is off, just as it is when I try using the hit list.  Would that have to do with where my camera is located?  I thought I knew how gluPerspective worked, but now I'm not so sure.
0
 

Author Comment

by:palehose
ID: 24263488
Alright, I did what you wanted me to do and removed all the GLUI stuff, and it works correctly.  I don't know what was throwing it off, but I guess I'm going to have to re-do all that (ugh).  I'll let you know if I have any other issues.
0
 

Author Comment

by:palehose
ID: 24265439
So I'm convinced the problem is when the GLUI has elements added (and thus making the viewport smaller, to make room for the new elements).  When the viewport is shifted upwards (I have a glui subwindow on the bottom), the selection buffer (I'm guessing) does not shrink, but simply shifts up.  Is this not the correct thing to do with reshape?

void reshape(int w, int h) {
	width = w;
	height = h;
 
	int vx, vy, vw, vh;
	GLUI_Master.get_viewport_area(&vx, &vy, &vw, &vh);
	glViewport(vx, vy, vw, vh);
 
	xy_aspect = (float)width / (float)height;
	glutPostRedisplay();
}

Open in new window

0
 

Accepted Solution

by:
palehose earned 0 total points
ID: 24266095
Okay, I found the problem and a solution.  Here it is to possibly save someone some time:

It appears this is probably a bug with GLUI, but using a BOTTOM (or TOP) subwindow screws up the PROJECTION matrix.  That's why my program thought the object was above where it actually renders.  My fix is listed, but it requires a static size, which is fine for what I'm trying to implement.  I'm translating the SIZE*(height of the GLUI subwindow) in the negative direction.  You need to make sure you acquire the viewport again after doing this translation, like my code shows.

void startPicking(int x, int y) {
	GLint viewport[4];
 
	glSelectBuffer(512, selectBuf);
	glRenderMode(GL_SELECT);
 
	glInitNames();
   	glPushName(0);
 
	glMatrixMode(GL_PROJECTION);
	glPushMatrix();
	glLoadIdentity();
 
	glGetIntegerv(GL_VIEWPORT, viewport);
	glTranslatef(0, (float)(-SIZE*viewport[1]), 0);
 
	glGetIntegerv(GL_VIEWPORT, viewport);
	gluPickMatrix(x, viewport[3]-y, 5, 5, viewport);
 
	gluPerspective(45, xy_aspect, 0.1, 1000);
 
	display();
 
	glMatrixMode(GL_PROJECTION);
	glPopMatrix();
	glMatrixMode(GL_MODELVIEW);
	glFlush();
 
   	int hits = glRenderMode(GL_RENDER);
	processHits(hits, selectBuf);
}

Open in new window

0
 
LVL 20

Assisted Solution

by:ikework
ikework earned 800 total points
ID: 24268266
hey palehose, glad to hear that .. :) lets not delete it, this can be useful to others ..
0
 
LVL 20

Expert Comment

by:ikework
ID: 24268281
and btw. im not quite done with it, are you? we should debug it better, to understand what happens and why.
normally the projection-matrix *cant* be off, because it is used by the renderer, then it wouldnt be rendered in the right way, would it?
0
 

Author Comment

by:palehose
ID: 24276703
I was not deleting it, just closing it.

Personally, yes, I am done with it.  This is a school project, and I don't have the time to get into the details of WHY it works the way it does.   I will give you some points for helping me move in the right direction.
0

Featured Post

[Webinar] Cloud and Mobile-First Strategy

Maybe you’ve fully adopted the cloud since the beginning. Or maybe you started with on-prem resources but are pursuing a “cloud and mobile first” strategy. Getting to that end state has its challenges. Discover how to build out a 100% cloud and mobile IT strategy in this webinar.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Recently, in one of the tech-blogs I usually read, I saw a post about the best-selling video games through history. The first place in the list is for the classic, extremely addictive Tetris. Well, a long time ago, in a galaxy far far away, I was…
Performance in games development is paramount: every microsecond counts to be able to do everything in less than 33ms (aiming at 16ms). C# foreach statement is one of the worst performance killers, and here I explain why.
This video shows how to quickly and easily deploy an email signature for all users in Office 365 and prevent it from being added to replies and forwards. (the resulting signature is applied on the server level in Exchange Online) The email signat…
With just a little bit of  SQL and VBA, many doors open to cool things like synchronize a list box to display data relevant to other information on a form.  If you have never written code or looked at an SQL statement before, no problem! ...  give i…
Suggested Courses

755 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question