Sparse ternary connect: Convolutional neural networks using ternarized weights with enhanced sparsity

Canran Jin; Heming Sun; Shinji Kimura

doi:10.1109/ASPDAC.2018.8297304

Sparse ternary connect: Convolutional neural networks using ternarized weights with enhanced sparsity

Canran Jin, Heming Sun, Shinji Kimura

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

6 Citations (Scopus)

Abstract

Convolutional Neural Networks (CNNs) are indispensable in a wide range of tasks to achieve state-of-the-art results. In this work, we exploit ternary weights in both inference and training of CNNs and further propose Sparse Ternary Connect (STC) where kernel weights in float value are converted to 1, -1 and 0 based on a new conversion rule with the controlled ratio of 0. STC can save hardware resource a lot with small degradation of precision. The experimental evaluation on 2 popular datasets (CIFAR-10 and SVHN) shows that the proposed method can reduce resource utilization (by 28.9% of LUT, 25.3% of FF, 97.5% of DSP and 88.7% of BRAM on Xilinx Kintex-7 FPGA) with less than 0.5% accuracy loss.

Original language	English
Title of host publication	ASP-DAC 2018 - 23rd Asia and South Pacific Design Automation Conference, Proceedings
Publisher	Institute of Electrical and Electronics Engineers Inc.
Pages	190-195
Number of pages	6
ISBN (Electronic)	9781509006021
DOIs	https://doi.org/10.1109/ASPDAC.2018.8297304
Publication status	Published - 2018 Feb 20
Event	23rd Asia and South Pacific Design Automation Conference, ASP-DAC 2018 - Jeju, Korea, Republic of Duration: 2018 Jan 22 → 2018 Jan 25

Publication series

Name	Proceedings of the Asia and South Pacific Design Automation Conference, ASP-DAC
Volume	2018-January

Other

Other	23rd Asia and South Pacific Design Automation Conference, ASP-DAC 2018
Country/Territory	Korea, Republic of
City	Jeju
Period	18/1/22 → 18/1/25

ASJC Scopus subject areas

Electrical and Electronic Engineering
Computer Science Applications
Computer Graphics and Computer-Aided Design

Access to Document

10.1109/ASPDAC.2018.8297304

Cite this

Jin, C., Sun, H., & Kimura, S. (2018). Sparse ternary connect: Convolutional neural networks using ternarized weights with enhanced sparsity. In ASP-DAC 2018 - 23rd Asia and South Pacific Design Automation Conference, Proceedings (pp. 190-195). (Proceedings of the Asia and South Pacific Design Automation Conference, ASP-DAC; Vol. 2018-January). Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/ASPDAC.2018.8297304

Sparse ternary connect: Convolutional neural networks using ternarized weights with enhanced sparsity. / Jin, Canran; Sun, Heming; Kimura, Shinji.
ASP-DAC 2018 - 23rd Asia and South Pacific Design Automation Conference, Proceedings. Institute of Electrical and Electronics Engineers Inc., 2018. p. 190-195 (Proceedings of the Asia and South Pacific Design Automation Conference, ASP-DAC; Vol. 2018-January).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

Jin, C, Sun, H & Kimura, S 2018, Sparse ternary connect: Convolutional neural networks using ternarized weights with enhanced sparsity. in ASP-DAC 2018 - 23rd Asia and South Pacific Design Automation Conference, Proceedings. Proceedings of the Asia and South Pacific Design Automation Conference, ASP-DAC, vol. 2018-January, Institute of Electrical and Electronics Engineers Inc., pp. 190-195, 23rd Asia and South Pacific Design Automation Conference, ASP-DAC 2018, Jeju, Korea, Republic of, 18/1/22. https://doi.org/10.1109/ASPDAC.2018.8297304

Jin C, Sun H, Kimura S. Sparse ternary connect: Convolutional neural networks using ternarized weights with enhanced sparsity. In ASP-DAC 2018 - 23rd Asia and South Pacific Design Automation Conference, Proceedings. Institute of Electrical and Electronics Engineers Inc. 2018. p. 190-195. (Proceedings of the Asia and South Pacific Design Automation Conference, ASP-DAC). doi: 10.1109/ASPDAC.2018.8297304

Jin, Canran ; Sun, Heming ; Kimura, Shinji. / Sparse ternary connect : Convolutional neural networks using ternarized weights with enhanced sparsity. ASP-DAC 2018 - 23rd Asia and South Pacific Design Automation Conference, Proceedings. Institute of Electrical and Electronics Engineers Inc., 2018. pp. 190-195 (Proceedings of the Asia and South Pacific Design Automation Conference, ASP-DAC).

@inproceedings{18cb26a828374eec96a226e40225ea6a,

title = "Sparse ternary connect: Convolutional neural networks using ternarized weights with enhanced sparsity",

abstract = "Convolutional Neural Networks (CNNs) are indispensable in a wide range of tasks to achieve state-of-the-art results. In this work, we exploit ternary weights in both inference and training of CNNs and further propose Sparse Ternary Connect (STC) where kernel weights in float value are converted to 1, -1 and 0 based on a new conversion rule with the controlled ratio of 0. STC can save hardware resource a lot with small degradation of precision. The experimental evaluation on 2 popular datasets (CIFAR-10 and SVHN) shows that the proposed method can reduce resource utilization (by 28.9% of LUT, 25.3% of FF, 97.5% of DSP and 88.7% of BRAM on Xilinx Kintex-7 FPGA) with less than 0.5% accuracy loss.",

author = "Canran Jin and Heming Sun and Shinji Kimura",

note = "Funding Information: Thanks are due to members of Kimura-lab in Waseda University for their discussions. The work is supported in part by the fund from NEC. Publisher Copyright: {\textcopyright} 2018 IEEE.; 23rd Asia and South Pacific Design Automation Conference, ASP-DAC 2018 ; Conference date: 22-01-2018 Through 25-01-2018",

year = "2018",

month = feb,

day = "20",

doi = "10.1109/ASPDAC.2018.8297304",

language = "English",

series = "Proceedings of the Asia and South Pacific Design Automation Conference, ASP-DAC",

publisher = "Institute of Electrical and Electronics Engineers Inc.",

pages = "190--195",

booktitle = "ASP-DAC 2018 - 23rd Asia and South Pacific Design Automation Conference, Proceedings",

}

TY - GEN

T1 - Sparse ternary connect

T2 - 23rd Asia and South Pacific Design Automation Conference, ASP-DAC 2018

AU - Jin, Canran

AU - Sun, Heming

AU - Kimura, Shinji

PY - 2018/2/20

Y1 - 2018/2/20

N2 - Convolutional Neural Networks (CNNs) are indispensable in a wide range of tasks to achieve state-of-the-art results. In this work, we exploit ternary weights in both inference and training of CNNs and further propose Sparse Ternary Connect (STC) where kernel weights in float value are converted to 1, -1 and 0 based on a new conversion rule with the controlled ratio of 0. STC can save hardware resource a lot with small degradation of precision. The experimental evaluation on 2 popular datasets (CIFAR-10 and SVHN) shows that the proposed method can reduce resource utilization (by 28.9% of LUT, 25.3% of FF, 97.5% of DSP and 88.7% of BRAM on Xilinx Kintex-7 FPGA) with less than 0.5% accuracy loss.

AB - Convolutional Neural Networks (CNNs) are indispensable in a wide range of tasks to achieve state-of-the-art results. In this work, we exploit ternary weights in both inference and training of CNNs and further propose Sparse Ternary Connect (STC) where kernel weights in float value are converted to 1, -1 and 0 based on a new conversion rule with the controlled ratio of 0. STC can save hardware resource a lot with small degradation of precision. The experimental evaluation on 2 popular datasets (CIFAR-10 and SVHN) shows that the proposed method can reduce resource utilization (by 28.9% of LUT, 25.3% of FF, 97.5% of DSP and 88.7% of BRAM on Xilinx Kintex-7 FPGA) with less than 0.5% accuracy loss.

UR - http://www.scopus.com/inward/record.url?scp=85045315293&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85045315293&partnerID=8YFLogxK

U2 - 10.1109/ASPDAC.2018.8297304

DO - 10.1109/ASPDAC.2018.8297304

M3 - Conference contribution

AN - SCOPUS:85045315293

T3 - Proceedings of the Asia and South Pacific Design Automation Conference, ASP-DAC

SP - 190

EP - 195

BT - ASP-DAC 2018 - 23rd Asia and South Pacific Design Automation Conference, Proceedings

PB - Institute of Electrical and Electronics Engineers Inc.

Y2 - 22 January 2018 through 25 January 2018

ER -

Sparse ternary connect: Convolutional neural networks using ternarized weights with enhanced sparsity

Abstract

Publication series

Other

ASJC Scopus subject areas

Access to Document

Other files and links

Fingerprint

Cite this