SRI developed a vision system for use in the robotic inspection and repair of the space shuttle's heat shield during its refurbishment at NASA's Kennedy Space Center. Over 20,000 heat-absorbing tiles must be inspected for chips, gouges, and other damage that may occur during reentry. The vision system compares an image of each tile to an image captured before the mission, and therefore demands precise control of the sensor viewpoint and the lighting. A range sensor precisely controls the position and orientation of the end effector with respect to the orbiter surface, and a high-resolution digital camera is used to precisely direct the end effector to a specific tile. Because of the high variation in the reflectance os each tile, the amount of illumination for the captured images was adaptive to each tile.

Inspection was performed in a combination of image processing hardware an software. The algorithms normalize for minor variations in image location, identify variations that are insignificant, and highlight the major anomalies for review by a human observer. The vision system is also used to guide the positioning of a specialized tool which injects waterproofing fluid into each tile. Extensive collaboration with the NASA and other subcontractors was required to engineer a solution that met the rigorous environmental and packaging requirements.