Laboratory for Engineering Man/Machine Systems

Benjamin Kimia

We have developed a computational paradigm for vision that relies on

• Wave Propagation (Eikonal Equation)
• Interference Patterns or Shock Waves
• Transformation of these patterns
• Detection of optimal paths of transformation


to approach a number of vision problems,

 
• Figure-ground segregation / perceptual grouping / segmentation
• Object recognition / indexing into image databases by context
• Prototype formation and categorization

 


Example
 
 



 
 
 
 
 
 
 



 
 
 
 
 
 
 



 
 










A central question in computer and human vision is how to organize local evidence of structure such that globally coherent structures emerge.  In other words, how to organize pixels into objects?
 
 









 
 








The need to organize "geometrically" related structure is not unique to vision, but also applies to other domains such as touch and sound, as well as motor maps.
 

More generally, it applies in a more abstract context where topology and geometry can play a role, e.g.,
 

• Organization of objects in the shape space:  prototype formation via a similarity-induced distance and neighborhood topology.






Abstract setting applicable to a metric space with transformations defining a move from each point to a neighboring point.
 

A recent proposal in categorization theory shows that transformation based approach unifies Tversky's feature space and Shepard's geometric space theories. [Chater and Hahn, SimCat '97]
 

A computational paradigm based on wave-propagation and shock waves
 
 
 
 
 

• A representation of the neighborhood relationship of the data is constructed via wave-propagation from the data, resulting in interference patterns or shockwaves

 

 
 





Single Edge
3 Edges
Simple Shape
Complex Shape




• The resulting interference pattern is iteratively transformed to construct the object neighborhood structure

 

 
 





Rectangle with Gaps
Rectangle with Noisy Edges
Occluded Rectangle
Rectangle with a Part
Dot-grouping
L Shape
Noisy Corner










• Construct optimal paths consisting of candidate transform sequences

 

 
 
 
 
 
 
 
 
 
 
 

• Minimal Paths for Object Recognition
• Paths of Least Action to simplify Perceptual Grouping
• Recognize high traffic "Hubs" as Prototypes
• Shock Matching
• Image Indexing

 

 
 









 
 
 
 


 


 




 
 
 
 


 
 
 
 
 
 
 

The Neural Connection





• The proposed paradigm requires a combination of intra-layer "horizontal" activity as well as inter-layer "vertical" activity.

 
• Intra-layer wave propagation proposed here is intriguingly consistent with exisiting psychophysical evidence and neurophysiological recordings.

 

 
 
 
 
 
 
 
 
 
 
 

• Traditional Receptive Field (RF) models is equivalent to "convolution".

 

 
 














• Effect of repeated convolutions is to construct a large RF.

 

 
 
 
 
 
 
 
 
 
 
 

• Example in Computer Vision, partly motivated by the biological model

 

 
 
 
 
 
 
 
 
 

        - Repeated Gaussian Kernel Convolution:
            Gausssian pyramid (used in image compression)

        - Scale space models to allow visual operators ( e.g., edge -
             detection) to act in different scales.
            (Kimia, etal 1991,1993,1994,1997)

        - Kernel is the green function of a parabolic PDE
 

• Departure: hyperbolic PDE's

 

 
 
 
 
 
 
 
 
 

            - Propagation in the same plane: dynamic element
                provides the additional dimension.
 
 

• Psychophysics

        - Paradiso's proposal on the propagation of
            brightness and simultaneous contrast experiment
        - Kovacz and Julez
 
• Neuro Evidence

        - Conners etal
                 * invitro wave-propagation
        - Fregnac etal
                 * horizontal propagation
        - Nelson and Koutz
                 *  ferret's development of horizontal activity
                     dependant on the refinement of orientation map.
        - others

Goals
 

• Parallel implentations and experiments

        - take into account neuronal circuit constraint
 
• Provide a model to seek dynamic activity in large scale recording

        - evidence of interference pattern
        - shock wave propagation
        - role of scale
 
• Extend model to spatially variant Eikonal equation

        - account for propagation velocity variations
        - dependance on local context
        - region based segmentation
 
• Interactions between the "shockwave"

and the "network of networks" models.