Research and examination on implementation of super-resolution models using deep learning with INT8 precision

Shota Hirose, Naoki Wada, Jiro Katto, Heming Sun

Research output: Chapter in Book/Report/Conference proceedingConference contribution

2 Citations (Scopus)

Abstract

Fixed-point arithmetic is a technique for treating weights and intermediate values as integers in deep learning. Since deep learning models generally store each weight as a 32-bit floating-point value, storing by 8-bit integers can reduce the size of the model. In addition, memory usage can be reduced, and inference can be much faster by hardware acceleration when special hardware for int8 inference is provided. On the other hand, when inferences are carried out by fixed-point weights, accuracy of the model is reduced due to loss of dynamic range of the weights and intermediate layer values. For this reason, inference frameworks such as TensorRT and TensorFlow Lite, provide a function called "calibration"to suppress the deterioration of the accuracy caused by quantization by measuring the distribution of input data and numerical values in the intermediate layer when quantization is performed. In this paper, after quantizing a pre-trained model that performs super-resolution, speed and accuracy are measured using TensorRT. As a result, the trade-off between the runtime and the accuracy is confirmed. The effect of calibration is also confirmed.

Original languageEnglish
Title of host publication4th International Conference on Artificial Intelligence in Information and Communication, ICAIIC 2022 - Proceedings
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages133-137
Number of pages5
ISBN (Electronic)9781665458184
DOIs
Publication statusPublished - 2022
Event4th International Conference on Artificial Intelligence in Information and Communication, ICAIIC 2022 - Jeju lsland, Korea, Republic of
Duration: 2022 Feb 212022 Feb 24

Publication series

Name4th International Conference on Artificial Intelligence in Information and Communication, ICAIIC 2022 - Proceedings

Conference

Conference4th International Conference on Artificial Intelligence in Information and Communication, ICAIIC 2022
Country/TerritoryKorea, Republic of
CityJeju lsland
Period22/2/2122/2/24

Keywords

  • Quantization
  • Real-time inference
  • Super resolution
  • Tensor RT

ASJC Scopus subject areas

  • Artificial Intelligence
  • Computer Science Applications
  • Electrical and Electronic Engineering

Fingerprint

Dive into the research topics of 'Research and examination on implementation of super-resolution models using deep learning with INT8 precision'. Together they form a unique fingerprint.

Cite this