Real-Time Refocusing using an FPGA-based Standard Plenoptic Camera
Abstract
Plenoptic cameras are receiving increasing attention in scientific and commercial applications because they capture the entire structure of light in a scene, enabling optical transforms (such as focusing) to be applied computationally after the fact, rather than once and for all at the time a picture is taken. In many settings, real-time interactive performance is also desired, which in turn requires significant computational power due to the large amount of data required to represent a plenoptic image. Although GPUs have been shown to provide acceptable performance for real-time plenoptic rendering, their cost and power requirements make them prohibitive for embedded uses (such as in-camera). On the other hand, the computation to accomplish plenoptic rendering is well-structured, suggesting the use of specialized hardware. Accordingly, this paper presents an array of switch-driven Finite Impulse Response (FIR) filters, implemented with FPGA to accomplish high-throughput spatial-domain rendering. The proposed architecture provides a power-efficient rendering hardware design suitable for full-video applications as required in broadcasting or cinematography. A benchmark assessment of the proposed hardware implementation shows that real-time performance can readily be achieved, with a one order of magnitude performance improvement over a GPU implementation and three orders of magnitude performance improvement over a general-purpose CPU implementation.
- Publication:
-
arXiv e-prints
- Pub Date:
- October 2020
- DOI:
- 10.48550/arXiv.2010.07746
- arXiv:
- arXiv:2010.07746
- Bibcode:
- 2020arXiv201007746H
- Keywords:
-
- Electrical Engineering and Systems Science - Image and Video Processing;
- Computer Science - Hardware Architecture
- E-Print:
- IEEE Trans. on Industrial Electronics