Integrating non-semantic knowledge into image segmentation processes

The dissertation develops several techniques for automatically segmenting images into regions. The basic approach involves the integration of different types of non-semantic knowledge into the segmentation process such that the knowledge can be used when and where it is useful. These processes are intended to produce initial segmentations of complex images which are faithful with respect to fine image detail, balanced by a computational need to limit the segmentations to a fairly small number of regions. Natural scenes often contain intensity gradients, shadows, highlights, texture, and small objects with fine geometric structure, all of which make the calculation and evaluation of reasonable segmentations for natural scenes extremely difficult. The approach taken by this dissertation is to integrate specialized knowledge into the segmentation process for each kind of image event that can be shown to adversely affect the performance of the process. At the center of our segmentation system is an algorithm which labels pixels in localized subimages with the feature histogram cluster to which they correspond, followed by a relaxation labeling process. However, this algorithm has a tendency to undersegment by failing to find clusters corresponding to small objects; it may also oversegment by splitting intensity gradients into multiple clusters, by finding clusters for mixed pixel regions, and by finding clusters corresponding to microtexture elements. In addition, the relaxation process often destroys fine structure in the image.