Low-Latency In Situ Image Analytics With FPGA-Based Quantized Convolutional Neural Network

Maolin Wang, Kelvin C.M. Lee, Bob M.F. Chung, Sharat Chandra Varma B, Ho Cheung Ng, Justin Wong, Ho Cheung Shum, Kevin K Tsia, Hayden Kwok-Hay So

Research output: Contribution to journalArticlepeer-review

13 Citations (Scopus)
53 Downloads (Pure)


Real-time in situ image analytics impose stringent latency requirements on intelligent neural network inference operations. While conventional software-based implementations on the graphic processing unit (GPU)-accelerated platforms are flexible and have achieved very high inference throughput, they are not suitable for latency-sensitive applications where real-time feedback is needed. Here, we demonstrate that high-performance reconfigurable computing platforms based on field-programmable gate array (FPGA) processing can successfully bridge the gap between low-level hardware processing and high-level intelligent image analytics algorithm deployment within a unified system. The proposed design performs inference operations on a stream of individual images as they are produced and has a deeply pipelined hardware design that allows all layers of a quantized convolutional neural network (QCNN) to compute concurrently with partial image inputs. Using the case of label-free classification of human peripheral blood mononuclear cell (PBMC) subtypes as a proof-of-concept illustration, our system achieves an ultralow classification latency of 34.2 μs with over 95% end-to-end accuracy by using a QCNN, while the cells are imaged at throughput exceeding 29,200 cells/s. Our QCNN design is modular and is readily adaptable to other QCNNs with different latency and resource requirements.

Original languageEnglish
Article number21924722
Pages (from-to)2853 - 2866
Number of pages14
JournalIEEE Transactions on Neural Networks and Learning Systems
Issue number7
Early online date12 Jan 2021
Publication statusPublished (in print/issue) - 1 Jul 2022

Bibliographical note

Publisher Copyright:


  • Low latency inference
  • Reconfigurable computing
  • FPGA
  • Cell image classification
  • QCNN
  • Multi-ATOM
  • Real-time Image Analytics
  • Neural network
  • Optical Cytometry


Dive into the research topics of 'Low-Latency In Situ Image Analytics With FPGA-Based Quantized Convolutional Neural Network'. Together they form a unique fingerprint.

Cite this