Query by Sketch in Image Databases

The database used here consisted of 137 typical images drawn from the web. The geometric search engine was developed by Princeton University and Z. Lei, of our laboratory. We desire to find an image containing a car pointing at roughly 40 degrees with respect to the viewer of the image. This is a difficult shape for most people to draw, and is very difficult to draw for someone devoid of artistic skill. The idea is to proceed interactively with the database system, letting the system help the user in making an appropriate sketch. The user begins by sketching an easy shape, e.g., a side view of a car, the first Figure, and indexes into the database. The 8 highest ranking returns are shown: the 5th is a car pointing 60 degrees to 70 degrees with respect to the viewer. With that outline as a guide, the user modifies the outline into the sketch in the second Figure, which is the users attempt at sketching the desired shape. Upon indexing into the database again, The 5th image in the highest ranking 8 returned is an example of what is desired. This is an example of interacting with the database by letting the database help based on what is in the database rather than interaction by using a set of generic primitives for querying the database. Using generic primitives seems to be impractical if the database is generic containing a broad range of object types.

This figure, illustrates a different search concept for a car. An initial sketched curve fragment results in no useful return because it is insufficiently restrictive. A 2nd iteration involving the sketching of additional structure does result in some car hits among the 8 highest ranking returns.

Back to research topics.