Computationally efficient implementation of video rectification in FPGA fo stereo vision applications

Abstract

In order to obtain depth perception in computer vision, one needs to process pairs of stereo images. This process is computationally challenging to be carried out in real-time, because it requires the search for matches between objects in both images. Such process is significantly simplified if the images are reflected. Stereo image reflection involves a matrix transformation which when done in software will not produce real-time results. But in stereo vision applications this features is very demanding. On the other hand, applying those transformations to the video frames is very restricted by real-time constraints. Therefore, the video streaming and matrix transformation are not usually implemented in the same system. Our product is a stereo camera pair which produces a rectified real time image output with a resolution of 320x240 at a frame rate of 15FPS and delivers then via 100-Ethernet interface. We use an Spartan 3E FPGA for real-time processing within we implement an image rectification algorithm.