FPGA implementation of high frame rate and ultra-low delay tracking with local-search based block matching

As more and more human-machine interactive applications call for higher frame rate and lower delay to get a better experience, there is an inevitable need for high frame rate and ultra-low delay image processing system. Current existing works based on vision chip target on video with simple patterns or simple shapes in order to get a higher speed, which is reasonable in the first trial of this new field. However, more complicated system is required for real-life applications. This paper proposes a KLT (Kanade-Lucas-Tomasi) based tracking system with high frame rate and ultra-low delay, implemented on FPGA board. And based on the framework of KLT, local maximum neighboring check, local-search based SAD (Sum of Absolute Difference) block matching and coordinate-addressed feature storage are proposed to localize the whole algorithm. In a broader framework, the whole system is made parallelized and pipelined in order to get a high throughput which can meet the high frame rate and ultra-low delay system's demand. Evaluation results show that the proposed hardware system can work at 784fps and 0.762ms delay with resolution of 640×480.