How to pan the camera in perspective projection

ikework:
I think you have misunderstood the question.
I already have the camera rotation (commonly known as orbit) , I already have zoom and move forward.
I also have the move along up vector and right vector (commonly known as pan) , however the panning is not intuitive enough. The panning is performed by clicking a mouse button , then dragging the mouse then releasing.
When first clicked, the user essentially grabs a point in 3d scene, by dragging the mouse the user moves the camera along up and right vectors in such a way that the grabbed point stays under the cursor at all times. The latter is what I find difficult to achieve.

What I understand is, e.g. there's a TV set in the right part of the room and screen, when you grab it and pan the view to the left, the mouse is still on the TV set, whereever the TV set is on the screen (in the middle or left part of the screen).

It seems the problem generates from the ratio of the mouse displacement / the view and objects displacement (which involves the zoom factor and the perspective nature). This can be the way you want, only if the sw knows which object you clicked on. Because of the perspective, a TV set in front of the wall will move a less displacement than the wall behind it. I'm not sure if I'm answering the right question, but I hope this helps. The idea is making the 3D application aware on the object or the rendered pixel which you click on.

kbac:
The idea is correct,

I am aware of both the 2d sceen coordinates of the picked pixel and the 3d coordinates in the scene.

Problem is correcting the movement of the camera in such a way that the 3d point I am moving is still under the cursor when I finish the movement.

I can summarize the question in this way: how much do I need to move the camera target and position so that a certain 3d point in the scene moves from position a to position b ...

You need not to move the camera based on the screenmovement, but based on the world movement. When you click your mouse, find out out how you camera is positioned relative to the object you click on in the World. Then pan that position according to you movement. When done, reposition your camera relative to the new point. Then your camera should be panned accordingly.

Xcone:
Your suggestion is good, however I have already tried it and it's not that simple. You are suggesting that the relation between the camera and the picked point should stay the same, when in reality it is not...

Here is the approach I tried:
1 Pick a point in 3d,
2 calculate a vector from camera to that point ,
3 move the point in 3d according to the mouse movement,
4 calculate the camera by adding the same vector (calculated in step 2) to the new position of the 3d point.

Didn't work. The point lost relation the mouse 2d position...

I've never created such thing myself, but it sounded so logical. Now that I think of it, it'll only work if your cursor is allways centered (like a crosshair) and only movement is tracked. I know some other theoretical steps. I have no project at hand to verify it, so try them yourself first. If you need help it'll cost me some time to configure a project to recreate it myself.

First: You need to get the mouse->world position twice. One where the mouse was, and one where the mouse is.

Second: Cross the direction of the camera to world-startposition with the direction of the camera to the world-endposition. This would result in the 'up' vector. Crossing the 'Up' vector with either the direction to the world start or end positions, will provide you with a 'sideways' vector. These 2 vectors define a plane on which you can movement relative to your current camera position.

Thirdly: Subtracting the world-endposition from the world-startposition, will give you a vector with a magnitude to go from start to end. Taking into account that the end position will be further or closer then the start, you'll need to scale that magnitude relative to the orientation of the plane. My knowledge on 'projecting' is limited, but I guess using some dotproduct on the vectors you allready have, or perhaps build a matrix would do the trick.

Fourth and finally: Normalize the directional vector from start to end, and multiply it with the scaled magnitude of the third step. Add that result to your camera position.

Maybe it can be done simpler. As I mentioned, I've not done this before, but these are the steps I'd take should I create it myself.

Xcone :
I'll try your suggestion and let you know.

Hi! I have the exact same problem.. Sunnydark, have you found a solution that works?

Xcone, I don't quite understand the general concept behind your last post, but it sounds like something similar to ray casting?

This post on CodeProject is helping me a lot: http://69.10.233.10/KB/openGL/TranslationController.aspx
I think what we're aiming at here is the same as described under "The Design" section in that article, but with the "virtual plane" always parallel to the "viewing plane/screen"..

Thanks,
g

Xcone, thanks very much, I'll see what I can make of this in the next few days!
I'm actually trying to do this in WPF so it might take a while longer :)

SunnyDark - have you found a solution yet?