An FPGA platform for demonstrating embedded vision systems

Abstract

This thesis presents an FPGA platform that can be used to enable real-time embedded vision systems, specifically object detection. Interfaces are built between the FPGA and a high definition (1980 x 1080) HDMI camera, an off-chip DRAM, an FMC connector, an SD Card, and an HDMI display. The interface processing includes debayering for the camera input, arbitration for DRAM, and object annotation for the display. The platform must also handle the different clock frequencies of various interfaces. Real-time object detection at 30 frames per second is demonstrated by either connecting the platform to an object detection ASIC via the FMC connector, or directly implementing the object detection RTL on the FPGA. Using this platform, ASICs developed in the Energy-Efficient Multimedia Systems lab can be verified and benchmarked on both live video via the HDMI camera as well as pre-recorded media via an SD Card. Finally, a post-processing filter has been implemented on the FPGA to reduce false positives and interpolate missed object detections by leveraging temporal correlations.