3D-2D Registration of Medical Images

 
3D-2D projective registration of free-form curves and surfaces
Jacques Feldmar, Nicholas Ayache, Fabienne Betting
Computer Vision and Image Understanding, 403-424, 1997
[Main Paper]

A Survey of Medical Image Registration
J.B. Antoine Maintz, Max A. Viergever
 Medical Image Analysis,1998, 2(1):1-36
[Supplemental Paper]

Alignment by Maximization of Mutual Information
Paul Viola and William Wells III
 In Proc. of the IEEE Intl. Conf. on Computer Vision, 1996, 15-23
[Supplemental Paper]
 
 

[OUTLINE]


 
Outline
  • Introduction
    • The necessity of registration in medical imaging
    • Registration overview
  • 3D-2D Registration
    • Overview of 3D-2D registration
    • 3D-2D projective registration of free-form curves and surfaces
 
back to outline

 
Introduction

"Registration is the determination of a geometrical transformation that aligns points in one view of an object with corresponding points in another view of that object or another object"
-Fitzpatrick, Hill, Maurer: Handbook of Medical Imaging. 

  • The necessity of registration in medical imaging
      • It is obvious from the above quote why registration plays a vital role in the usefulness of medical imaging in clinical settings:  without a common coordinate system, comparison is meaningless
       

       
      ?
      		  
       
      =
      MR Scan
      CT Scan

      You can't compare these meaningfully without registering them first

    • Registration overview


      • Click for Movie  - From the Scientific Movie Library: [http://www.crd.ge.com/esl/cgsp/projects/video/medical/index.html]

      What differentiates registration methods?

      1. Dimensionally:  3D/3D, 2D/3D, 2D/2D and spatial, time-spatial
      2. Nature of registration basis: extrinsic or intrinsic
      3. Nature of transformation: rigid, affine, projective, curved
      4. Domain of transformation: global, local
      5. Interaction: interactive, semi-automatic, automatic
      6. Modalities: monomodal, multimodal, modality to model, patient to modality
      7. Subject: intrasubject, intersubject, atlas
      8. Object:  head, abdomen, thorax, etc.
      You can decompose any registration procedure into 3 major fundamental parts which determine the above list:
      1. Problem statement - influences 1, 3 and determines 6, 7, 8
      2. Registration paradigm - influences 2, 3, 4, 5
      3. Optimization procedure - influences 5
     
    back to outline

    3D-2D Registration
    • Overview of 3D-2D registration


      • 3D-2D registration (as well as 2D-3D) is the process of trying to align a 2D view of an object with the 3D data of an object (same or different) i.e., to determine the pose of the 3D object, from which the 2D view came

      A problem encountered in radiotherapy is to rely on MRI with respect to the actual position of the patient to guide the radiation source.  The current practice is to employ a stereotactic frame.1

      If reliable registration could be provided between a known model of the anatomy and a current 2D scan there would be no need for the extrinsic marker (more comfortable on the patient and more cost-effective for the hospital).


      Generating the projection of a 3D object.  This give rise to the occluding contour.

      The goal of 2D-3D registration is to determine the optimal transformations (rotation, translation, and projection)  needed to bring the 2D image into alignment with the 3D data.

    • 3D-2D projective registration of free-form surfaces
      Organized into two steps:
      1. Initial Estimate
      2. Optimization


      1.  Initial Estimate

        The goal is to find: (R0, t0), an initial estimate of the transformation parameters.
        a.  Choose 3 points (m1, m2, m3) on the 2D curve (the projection)  such that m1 and m2 have the same tangent and the normal of m1 and the normal of m3 are as close to orthogonal as possible (the actual angle is referred to as ).

        b. For each triplet of points on the 3D surface (M1, M2, M3) that M1 and M2 share the same tangent plane and the angle between the normal of M1 and the normal of M3 is .
        Compute the projective transformations which maps (M1, M2, M3) onto  (m1, m2, m3).
        c.  Find the number of points that map from the surface to the curve, and if the ratio of this number and the total number of points on the curve is over some threshold, stop.
         

      2.  Optimization
         
        Use an extension of the Iterative Closest Point (ICP) algorithm to optimize the registration.

        Remember that ICP operated by iterating over the set of points and minimizing the distance between the two sets using gradient descent.

        ICP in general works well but will generally fall into a local minimum if a good initial estimate is not found.  That is why it is important to have an initial estimator function.

    The distance function used, where m=(x, y) is a point on the 2D Curve and M=(X,Y,Z) is a point on the 3D surface.  
    1 = 1/(maxX-minX)   [on the 2D curve]
    2 = 1/(maxY-minY)   [on the 2D curve]

    Minimize this energy function:

    Match is a function which associates  a point on the 3D surface with each point on the 2D curve.

    Keep iterating until the min is hit!

     1Shweikard, et al. Proc. IEEE Int. Conference on Robotics and Automation 1994
     

    back to outline

    Results and Conclusions


    These are the results that were presented in the paper (and hopefully i will arrive at as well):
     

    3D CT-scan of a mannequin head Video picture of the mannequin head The result of the initial estimate, the white line is the contour of the 2D image. The result of the  final optimization procedure.

     
    Extracted CT-scan skull  X-Ray of the skull. Results of the initial transformation found. Final results of the algorithm.
    • Future Direction:
      • For next presentation (4/24 - 5/3):
          • Obtain data and implement the data handling
          Get the initial estimate functionality working
           
      • For final presentation (5/3 - 5/?):
          • Implement the optimization function
          Get registration results
     
     
    back to outline