Low-Latency In Situ Image Analytics With FPGA-Based Quantized Convolutional Neural Network

Maolin Wang, Kelvin C.M. Lee, Bob M.F. Chung, Sharat Chandra Varma B, Ho Cheung Ng, Justin Wong, Ho Cheung Shum, Kevin K Tsia, Hayden Kwok-Hay So

Research output: Contribution to journalArticlepeer-review

3 Citations (Scopus)
22 Downloads (Pure)


Real-time in situ image analytics impose stringent latency requirements on intelligent neural network inference operations. While conventional software-based implementations on the graphic processing unit (GPU)-accelerated platforms are flexible and have achieved very high inference throughput, they are not suitable for latency-sensitive applications where real-time feedback is needed. Here, we demonstrate that high-performance reconfigurable computing platforms based on field-programmable gate array (FPGA) processing can successfully bridge the gap between low-level hardware processing and high-level intelligent image analytics algorithm deployment within a unified system. The proposed design performs inference operations on a stream of individual images as they are produced and has a deeply pipelined hardware design that allows all layers of a quantized convolutional neural network (QCNN) to compute concurrently with partial image inputs. Using the case of label-free classification of human peripheral blood mononuclear cell (PBMC) subtypes as a proof-of-concept illustration, our system achieves an ultralow classification latency of 34.2 μs with over 95% end-to-end accuracy by using a QCNN, while the cells are imaged at throughput exceeding 29,200 cells/s. Our QCNN design is modular and is readily adaptable to other QCNNs with different latency and resource requirements.

Original languageEnglish
Number of pages13
JournalIEEE Transactions on Neural Networks and Learning Systems
Early online date12 Jan 2021
Publication statusPublished - 12 Jan 2021

Bibliographical note

Publisher Copyright:


  • Low latency inference
  • Reconfigurable computing
  • FPGA
  • Cell image classification
  • QCNN
  • Multi-ATOM
  • Real-time Image Analytics
  • Neural network
  • Optical Cytometry


Dive into the research topics of 'Low-Latency In Situ Image Analytics With FPGA-Based Quantized Convolutional Neural Network'. Together they form a unique fingerprint.

Cite this