Last update: Aug. 13, 2001

BACK

With SkiP, our intent is to provide a user with a system allowing gesture-based interactions for rapid 3D reconstruction of complex models/shapes/buildings from images showing strong perspective cues.

The emphasis in 3D reconstruction from multiple images has been either:

• In Photogrammetry: high precision of measurements (3D loci) with extensive and cumbersome human interactions (mostly through point measures and correspondences).
• In Computer Vision: full automation with less precision, but still through the matching of point-based measurements (mostly), and some primitive-based measures (recent work based on conics and similar regular shapes).

3D reconstruction from a single view has also been considered to a lesser extent, in both fields, with similar paradigms. Working with a single view has the disadvantage of providing less constraints on object point loci, and less cues for 3D computations. However, there are some practical advantages when working with a single view:

1. Sometimes it is all we have available: the scene does not exist anymore or is unreachable and all we have is this single snapshot (think of an old photo).
2. The scene of interest may be a drawing or painting of a scene that may have only existed in the imagination of the artist.
3. It is arguably easier for a human to work on one image at a time.

Hence, our first ambition with SkiP was to explore interactive ways of doing 3D reconstruction from single views. We have had some level of success with this approach (cf. the first 2 versions of SkiP, below). Our approach is based on three steps:

1. Interactive extraction of Perspective cues, an object being seen in 1, 2 or 3 Point Perspective, and of a Ground plane, in overlay to the original single view (picture), to specify camera parameters (Central projection, with no lens distortion, is assumed).
2. Interactive placement of primitives from/onto the Ground plane, in overlay of the picture, allowing for an eventual refinement of the camera parameters (if the primitives do not fit well enough the outlines in the picture).
3. Continuation of the placement of 3D primitives & CSGs to obtain as much geometric information as can be from the single view; followed by texture mapping of some selected primitives using again the picture.

Examples of reconstruction and use of Skip (v2):

• Ames room (in 1 PtP)
• Archeological site of Petra in Jordan (1 or 2 PtP).

The extensions to SkiP shall tackle the following problems:

1. Importing degrees of automation in the extraction of:
   • calibration data, such as vanishing points;
   • extraction of outlines of objects.
2. Process multiple images.
3. Permit the fusion with other useful sources of data, i.e. previous "sketches" (seen now as more complex sets of "primitives"), maps and floor plans.
4. Free-form modeling in 2D & 3D.
5. Use of texture gradients in natural scenes.

First version of the project: May 1998 - Starts as a final course project for CS224.

Second version: Oct. 1998 - Improvements on the user interface & some rendering tools.

• Mathematical background
• Camera Calibration :
  • Perspective Based I
    http://iris.usc.edu/Vision-Notes/bibliography/active700.html
  • Perspective Based II
  • Dimensional Analysis
• Shape & Structure from Perspective Effects
  http://iris.usc.edu/Vision-Notes/bibliography/shapefrom458.html
• Applications to road modeling
• Perception theories' perspective
• Drawing in perspective

BACK

Page created & maintained by Frederic Leymarie, 1998-2001.
Comments, suggestions, etc., mail to: leymarie@lems.brown.edu