A Comparative Evaluation of Automatic Rock Detection Methods

Enabling rock detection allows rovers to make the most of each command cycle by performing autonomous site characterization, and prioritization of the most important data for downlink. On Earth these algorithms assist data analysis by automating laborious image annotation tasks. We compare the performance of several detection algorithms on a representative set of Mars Exploration Rover data. Tested algorithms include strategies based on pixel intensity (Castano et al., 2004), filter cascades (Viola et al., 2002), shading models (Gulick et al., 2001) and stereo range data (Gor et al., 2001). The test dataset consists of 13 navigation images and 104 panoramic camera images under various terrain and lighting conditions. Together these images contain over 50,000 manually-labeled rocks. We assess detectors' performance on autonomous geology tasks: identifying targets for spectroscopy, estimating the fractional area of terrain covered by rocks, and identifying the contour outlines of rocks above 4cm in length. While detection performance varied considerably across different detection strategies, images, and tasks, some general trends are apparent. All rock detection algorithms underestimated fractional coverage area to varying degrees. Accurate identification of contour outlines was especially difficult; most detectors exhibit low recall and various biases in rock shape and size. However, all detectors performed significantly better than random on the target selection task, paralleling recent successes in autonomous spectrometer targeting. Comparative evaluation on field-typical datasets will remain important as rock detection technologies continue to mature. References: Castano et al, Intensity-based Rock Detection for Acquiring Onboard Rover Science, LPSC 2004. Gor et al, Autonomous rock detection for Mars terrain, AIAA Space 2001. Gulick et al, Autonomous image analysis during the 1999 Marsokhod rover field test, JGR 2001. Viola et al, Robust Real-Time Object Detection, IJCV 2002.