pigouvian
asked on
Match, transform, and drape 2D images to 3D mesh
I am starting a new project, but find I am out of my league and need some insight from the gurus. I am happy to learn what needs to be done on my own, but time is scarce and rather than spend days working on something that may not work, I would appreciate if the community would mind pointing me in the right direction (i.e., best practices, relevant papers, or even just your thoughts).
Situation: I have an m x 2 array of images (relative order is known) and high-resolution (1mm x 1mm) elevation data for an area covering L x W meters. Objects in the images have fine detail, are heterogeneous, and for all intents and purposes, randomly distributed.
Objective:
1. Create a mesh from elevation data
2. Find homographies between mesh and image (SIFT in 3D?)
3. Position camera in space to display desired position and scale
4. Transform images as needed and drape over mesh
5. Blend images (multiresolution spline?) to completely eliminate seams
6. Apply bump map (presumably from original photos?) to give a more natural appearance
7. Create red/cyan anaglyph on current view and export as TIF (will be large, but dims > supported by JPEG)
I am most comfortable programming in VB .NET and C#, but will take a crash course in C++ if I need to (I probably should anyway).
Thanks for any help!
Situation: I have an m x 2 array of images (relative order is known) and high-resolution (1mm x 1mm) elevation data for an area covering L x W meters. Objects in the images have fine detail, are heterogeneous, and for all intents and purposes, randomly distributed.
Objective:
1. Create a mesh from elevation data
2. Find homographies between mesh and image (SIFT in 3D?)
3. Position camera in space to display desired position and scale
4. Transform images as needed and drape over mesh
5. Blend images (multiresolution spline?) to completely eliminate seams
6. Apply bump map (presumably from original photos?) to give a more natural appearance
7. Create red/cyan anaglyph on current view and export as TIF (will be large, but dims > supported by JPEG)
I am most comfortable programming in VB .NET and C#, but will take a crash course in C++ if I need to (I probably should anyway).
Thanks for any help!
ASKER
Ozo -- thanks for the quick response! Unfortunately, the images are not stereo pairs. It just so happened that it only took two images to capture the width of the area under study with ~20% overlap.
ASKER CERTIFIED SOLUTION
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
So is your question about projection mapping?
http://www3.interscience.wiley.com/cgi-bin/abstract/77502997/ABSTRACT
http://www3.interscience.wiley.com/cgi-bin/abstract/77502997/ABSTRACT
ASKER
From a cursory look into the concepts of stereopsis and projection mapping, it seems like we are on the right track.
W.r.t. projection mapping, I would agree this seems to be an integral method to the construction of the image. However, there are two additional requirements I must find a solution for. First, there are lots of images and lots of runs. As such, I must find a way to automate the process of finding (at least I assume I need to find) point-wise correspondence between features on the generated mesh with features of the image* -- something analogous to what SIFT and other matching algorithms do for finding homographies between two 2D images to create panoramas. Second, since there is more than one image being projected onto the surface, I need to be able blend the overlap in the images so they are seamless. My original thought was I could create the panorama and then drape over the mesh, but I really only want to deform the images once.
W.r.t stereopsis algorithms, this seems the ticket to complete the last step. Thanks!
* note: each image is named such that the relative position is known, e.g., file001x_10000y_00600.jpg.
W.r.t. projection mapping, I would agree this seems to be an integral method to the construction of the image. However, there are two additional requirements I must find a solution for. First, there are lots of images and lots of runs. As such, I must find a way to automate the process of finding (at least I assume I need to find) point-wise correspondence between features on the generated mesh with features of the image* -- something analogous to what SIFT and other matching algorithms do for finding homographies between two 2D images to create panoramas. Second, since there is more than one image being projected onto the surface, I need to be able blend the overlap in the images so they are seamless. My original thought was I could create the panorama and then drape over the mesh, but I really only want to deform the images once.
W.r.t stereopsis algorithms, this seems the ticket to complete the last step. Thanks!
* note: each image is named such that the relative position is known, e.g., file001x_10000y_00600.jpg.
SOLUTION
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
Forced accept.
Computer101
EE Admin
Computer101
EE Admin
http://vision.stanford.edu/~birch/p2p/alg/