Rectification of 3-D Planar Scene Text

When the plane containing the text is at an angle relative to the image plane, several types of distortion can be present that makes it difficult to recognize the text. In the most general case, the distortion is described as a projective transformation between the plane containing the text and the image plane. Our approach is to estimate and correct this distortion by applying the appropriate "corrective" projective transformation to the image. That is, we warp the original image to create a synthetic image, which we call a "rectified image," in which the projective distortion has been removed. This image is then presented to the remainder of the video text recognition process, yielding improved recognition performance.

In our approach we take advantage of 3-D scene geometry to detect the relative orientation of the plane on which text is printed. This can be estimated in two ways, and we have developed techniques of both types. The first type of estimation uses the shape and orientation of the line of text in a single image. This technique is the most efficient, but it needs to identify text features and rely on assumptions about the geometry of text, which, while generally satisfied, are not always necessarily correct. The second type examines the motion of the text and plane through a sequence of video image frames. This technique does not require specific geometric assumptions, but it is more complex to implement, more sensitive to noise, and does not work with certain types of camera motion.