Riemannian Stochastic recursive gradient algorithm with retraction and vector transport and its convergence analysis

Hiroyuki Kasai; Hiroyuki Sato; Bamdev Mishra

Riemannian Stochastic recursive gradient algorithm with retraction and vector transport and its convergence analysis

Hiroyuki Kasai^*, Hiroyuki Sato, Bamdev Mishra

^*Corresponding author for this work

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

5 Citations (Scopus)

Abstract

Stochastic variance reduction algorithms have recently become popular for minimizing the average of a large, but finite number of loss functions on a Riemannian manifold. The present paper proposes a Riemannian stochastic recursive gradient algorithm (R-SRG), which does not require the inverse of retraction between two distant iterates on the manifold. Convergence analyses of R-SRG are performed on both retractionconvex and non-convex functions under computationally efficient retraction and vector transport operations. The key challenge is analysis of the influence of vector transport along the retraction curve. Numerical evaluations reveal that R-SRG competes well with state-of-the-art Riemannian batch and stochastic gradient algorithms.

Original language	English
Title of host publication	35th International Conference on Machine Learning, ICML 2018
Editors	Jennifer Dy, Andreas Krause
Publisher	International Machine Learning Society (IMLS)
Pages	3912-3935
Number of pages	24
ISBN (Electronic)	9781510867963
Publication status	Published - 2018
Externally published	Yes
Event	35th International Conference on Machine Learning, ICML 2018 - Stockholm, Sweden Duration: 2018 Jul 10 → 2018 Jul 15

Publication series

Name	35th International Conference on Machine Learning, ICML 2018
Volume	6

Conference

Conference	35th International Conference on Machine Learning, ICML 2018
Country/Territory	Sweden
City	Stockholm
Period	18/7/10 → 18/7/15

ASJC Scopus subject areas

Computational Theory and Mathematics
Human-Computer Interaction
Software

Cite this

Kasai, H., Sato, H., & Mishra, B. (2018). Riemannian Stochastic recursive gradient algorithm with retraction and vector transport and its convergence analysis. In J. Dy, & A. Krause (Eds.), 35th International Conference on Machine Learning, ICML 2018 (pp. 3912-3935). (35th International Conference on Machine Learning, ICML 2018; Vol. 6). International Machine Learning Society (IMLS).

Riemannian Stochastic recursive gradient algorithm with retraction and vector transport and its convergence analysis. / Kasai, Hiroyuki; Sato, Hiroyuki; Mishra, Bamdev.
35th International Conference on Machine Learning, ICML 2018. ed. / Jennifer Dy; Andreas Krause. International Machine Learning Society (IMLS), 2018. p. 3912-3935 (35th International Conference on Machine Learning, ICML 2018; Vol. 6).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

Kasai, H, Sato, H & Mishra, B 2018, Riemannian Stochastic recursive gradient algorithm with retraction and vector transport and its convergence analysis. in J Dy & A Krause (eds), 35th International Conference on Machine Learning, ICML 2018. 35th International Conference on Machine Learning, ICML 2018, vol. 6, International Machine Learning Society (IMLS), pp. 3912-3935, 35th International Conference on Machine Learning, ICML 2018, Stockholm, Sweden, 18/7/10.

Kasai, Hiroyuki ; Sato, Hiroyuki ; Mishra, Bamdev. / Riemannian Stochastic recursive gradient algorithm with retraction and vector transport and its convergence analysis. 35th International Conference on Machine Learning, ICML 2018. editor / Jennifer Dy ; Andreas Krause. International Machine Learning Society (IMLS), 2018. pp. 3912-3935 (35th International Conference on Machine Learning, ICML 2018).

@inproceedings{0a88dbb4065047279d64008c2a2d0e91,

title = "Riemannian Stochastic recursive gradient algorithm with retraction and vector transport and its convergence analysis",

abstract = "Stochastic variance reduction algorithms have recently become popular for minimizing the average of a large, but finite number of loss functions on a Riemannian manifold. The present paper proposes a Riemannian stochastic recursive gradient algorithm (R-SRG), which does not require the inverse of retraction between two distant iterates on the manifold. Convergence analyses of R-SRG are performed on both retractionconvex and non-convex functions under computationally efficient retraction and vector transport operations. The key challenge is analysis of the influence of vector transport along the retraction curve. Numerical evaluations reveal that R-SRG competes well with state-of-the-art Riemannian batch and stochastic gradient algorithms.",

author = "Hiroyuki Kasai and Hiroyuki Sato and Bamdev Mishra",

year = "2018",

language = "English",

series = "35th International Conference on Machine Learning, ICML 2018",

publisher = "International Machine Learning Society (IMLS)",

pages = "3912--3935",

editor = "Jennifer Dy and Andreas Krause",

booktitle = "35th International Conference on Machine Learning, ICML 2018",

}

TY - GEN

T1 - Riemannian Stochastic recursive gradient algorithm with retraction and vector transport and its convergence analysis

AU - Kasai, Hiroyuki

AU - Sato, Hiroyuki

AU - Mishra, Bamdev

PY - 2018

Y1 - 2018

N2 - Stochastic variance reduction algorithms have recently become popular for minimizing the average of a large, but finite number of loss functions on a Riemannian manifold. The present paper proposes a Riemannian stochastic recursive gradient algorithm (R-SRG), which does not require the inverse of retraction between two distant iterates on the manifold. Convergence analyses of R-SRG are performed on both retractionconvex and non-convex functions under computationally efficient retraction and vector transport operations. The key challenge is analysis of the influence of vector transport along the retraction curve. Numerical evaluations reveal that R-SRG competes well with state-of-the-art Riemannian batch and stochastic gradient algorithms.

AB - Stochastic variance reduction algorithms have recently become popular for minimizing the average of a large, but finite number of loss functions on a Riemannian manifold. The present paper proposes a Riemannian stochastic recursive gradient algorithm (R-SRG), which does not require the inverse of retraction between two distant iterates on the manifold. Convergence analyses of R-SRG are performed on both retractionconvex and non-convex functions under computationally efficient retraction and vector transport operations. The key challenge is analysis of the influence of vector transport along the retraction curve. Numerical evaluations reveal that R-SRG competes well with state-of-the-art Riemannian batch and stochastic gradient algorithms.

UR - http://www.scopus.com/inward/record.url?scp=85057248675&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85057248675&partnerID=8YFLogxK

M3 - Conference contribution

AN - SCOPUS:85057248675

T3 - 35th International Conference on Machine Learning, ICML 2018

SP - 3912

EP - 3935

BT - 35th International Conference on Machine Learning, ICML 2018

A2 - Dy, Jennifer

A2 - Krause, Andreas

PB - International Machine Learning Society (IMLS)

T2 - 35th International Conference on Machine Learning, ICML 2018

Y2 - 10 July 2018 through 15 July 2018

ER -

Riemannian Stochastic recursive gradient algorithm with retraction and vector transport and its convergence analysis

Abstract

Publication series

Conference

ASJC Scopus subject areas

Other files and links

Fingerprint

Cite this