3D-2D Registration of Medical Images

3D-2D projective registration of free-form curves and surfaces
Jacques Feldmar, Nicholas Ayache, Fabienne Betting
Computer Vision and Image Understanding, 403-424, 1997

 
Outline:



2D-3D registration

3D-2D registration (as well as 2D-3D) is the process of trying to align a 2D view of an object with the 3D data of an object (same or different) i.e., to determine the pose of the 3D object, from which the 2D view came 

The goal of 2D-3D registration is to determine the optimal transformations (rotation, translation, and projection)  needed to bring the 2D image into alignment with the 3D data, i.e., find (, ).

Back to Top


"3D-2D projective registration of free-form curves and surfaces"
Feldmar, Ayache, Betting; CVIU, 403-424, 1997
    Organized into two steps: 
  1. Initial Estimate 
  2. Optimization 
1.  Initial Estimate 
The goal is to find: (R0, t0), an initial estimate of the transformation parameters. 

a.  Choose 3 points (m1, m2, m3) on the 2D curve (the projection)  such that m1 and m2 have the same tangent and the normal of m1 and the normal of m3 are as close to orthogonal as possible (the actual angle is referred to as ). 

b. For each triplet of points on the 3D surface (M1, M2, M3) that M1 and M2 share the same tangent plane and the angle between the normal of M1 and the normal of M3 is  . 
Compute the projective transformations which maps (M1, M2, M3) onto (m1, m2, m3). 

c.  Find the number of points that map from the surface to the curve, and if the ratio of this number and the total number of points on the curve is over some threshold, stop. 


2.  Optimization

Use an extension of the Iterative Closest Point (ICP) algorithm to optimize the registration. 
Remember that ICP operates by iterating over the set of points and minimizing the distance between the two sets using gradient descent. 
ICP in general works well but will generally fall into a local minimum if a good initial estimate is not found.  That is why it is important to have an initial estimator function.
d(m, M) = ( 1 (x - X/Z)2 + 2 (y - Y/Z)2  + (Normal2D(m) - Normal3D(M))2)1/2

The distance function used, where m=(x, y) is a point on the 2D Curve and M=(X,Y,Z) is a point on the 3D surface. 
1 = 1/(maxX-minX)   [on the 2D curve] 
2 = 1/(maxY-minY)   [on the 2D curve] 


Back to Top

Initial Results
Completed Tasks:
  • Found set of three points (m1, m2, m3) on 2D contour
  • Found all sets of three points (M1, M2, M3) on 3D surface
  • Started implementing the iterating over 3D points to attempt to find optimal set of (M1, M2, M3) which match to fixed (m1, m2, m3)
Result Data:
3D Model
2D Unknown Contour
2D Triplets:  (m1, m2, m3) Examples of 3D Triplets:  (M1, M2, M3)
3D Model
2D Unknown Contour
2D Triplets:  (m1, m2, m3) Examples of 3D Triplets:  (M1, M2, M3)
3D Model
2D Unknown Contour
2D Triplets:  (m1, m2, m3) Examples of 3D Triplets:  (M1, M2, M3)

Sample of set of triplets:

Animation of Alignment:



Back to Top

Future Plans/Directions
  • Finish implementation of matching the sets of triplets against each other to obtain R-1 
  • Implement ICP to optimize the matches.
  • Run experiments using CT-Xray-MRI data.
Back to Top

Supplemental Sources:
 
A Survey of Medical Image Registration
J.B. Antoine Maintz, Max A. Viergever
 Medical Image Analysis, 1998, 2(1):1-36
Alignment by Maximization of Mutual Information
Paul Viola and William Wells III
 ICCV, 1996, 15-23

 
An approach to 2D/3D registration of a vertebra in 2D X-ray fluoroscopies with 3D CT images, 
 J. Weese, Th. M. Buzug, C. Lorenz and C. Fassnacht, 
Proc. of CVRMed/ MRCAS’97, p. 119 
Rendering Methods for Voxel-based 2D/3D Registration - A Comparative Study. 
R. Goecke, J. Weese, and H. Schumann. Fast Volume Proceedings of International Workshop on Biomedical Image Registration '99, pages 89-102