Segmentation in Clutter

M.J. Bravo and H. Farid

Vision Sciences (VSS), Sarasota, FL, 2002

In a cluttered scene, it may be difficult to fully segment an object using only bottom-up cues. In such cases we may segment the object by first detecting one of its salient, distinctive parts and then using this part to predict the location and orientation of other object parts. For a rigid object, the predictive power of the salient part should depend on its symmetry. For example, a sphere which has infinite rotational symmetry (and so looks the same from all viewpoints) should have less predictive power than a cone.

To test this idea we constructed computer-generated, rigid objects composed of two pieces: a "handle" (a simple geometric shape) and a "tool" (two connected cylinders). In each scene, an object was presented at a random orientation amongst clutter composed of cylinders resembling the tool. A small black or white ring was placed around one of the tool's cylinders at a location that varied across trials. Similar rings were also placed on the clutter. The observer's task was to report the color of the ring located on the tool. Because the tool was camouflaged against the background of clutter, response times were expected to depend on the degree to which the salient handle could be used to predict the tool's location and orientation in the clutter. That is, response times were expected to depend on the symmetry of the handle.

The results supported this idea: response times increased monotonically as the symmetry of the handle increased from 0-fold to 2-fold to 4-fold. Response times for handles with infinite symmetry, however, were no longer than those with 4-fold symmetry.

We conclude that observers can use a salient part to predict the location and orientation of the rest of an object. The predictive power of these salient parts depends, up to a limit, on their symmetry.