A Mixed-Signal Near-Sensor Convolutional Imager SoC with Charge-Based 4b-Weighted 5-to-84-TOPS/W MAC Operations for Feature Extraction and Region-of-Interest Detection

Lefebvre, Martin;Bol, David
(2024) 2024 IEEE Custom Integrated Circuits Conference (CICC) — Location: Denver (CO, USA) (21.April.2024)

Files

Lefebvre-2024-CICC-A_Mixed-Signal_Near-Sensor_Convolutional_Imager_SoC.pdf
  • Open Access
  • Adobe PDF
  • 978.31 KB
Download

Details

Authors
Abstract
The massive deployment of Internet-of-Things (loT) sensor nodes has fostered the development of machine learning algorithms and hardware at the edge. Regarding vision sensors, edge devices must be able to solve low- to medium-level computer vision tasks, e.g., feature extraction (FE) and region-of-interest (Rol) detection, before transmitting meaningful data to the cloud. This must be achieved within a sub-mW power budget, as loT sensor nodes are supplied by batteries or energy harvesting. Therefore, mixed-signal vision chips appear as a suitable solution as they outperform digital chips in terms of energy efficiency (EE) while maintaining reasonable processing accuracy. They can be divided into (i) in- [1] and (ii) near-sensor [2], [3], [4] vision chips, implemented with processing elements (PEs) inside and near the pixel array, and (iii) hybrid vision chips [5], [6], combining elements from both schemes. On the one hand, in-sensor chips [1] offer a massive parallelism but suffer from a large pixel pitch due to the pixel-level PEs. They are limited to low-level tasks as they employ 1-to-1.5b filter weights, and exploit raw [1] or amplified [5] photocurrents subject to significant mismatch. In addition, connections between PEs are limited to neighboring pixels, making it difficult to scale processing to image-level features. On the other hand, near-sensor and hybrid chips generally allow to implement medium-level tasks, either by employing large-size Haar-like filters [2], [5], [6], or by increasing the filter weights resolution [3], [4]. They employ conventional pixels such as DPS or 3T/4T APS, and operate at multiple spatial scales using image downsampling (DS) [1] or filter dilation [5]. An analog memory storing pixel values is often required, but entails power and/or area overheads [2]. Nonetheless, existing works fall short of preserving EE while simultaneously supporting low- and medium-level tasks, which require mismatch-compensated inputs, increased weights resolution, and multiscale operation.
Affiliations

Citations

  • APA
  • Chicago
  • FWB

Lefebvre, M., & Bol, D. (2024). A Mixed-Signal Near-Sensor Convolutional Imager SoC with Charge-Based 4b-Weighted 5-to-84-TOPS/W MAC Operations for Feature Extraction and Region-of-Interest Detection. Proceedings of the 2024 IEEE Custom Integrated Circuits Conference (CICC), 1-2. https://doi.org/10.1109/CICC60959.2024.10528961 (Original work published 2024)