On-line learning in changing environments with applications in supervised and unsupervised learning

Noboru Murata; Motoaki Kawanabe; Andreas Ziehe; Klaus Robert Müller; Shun Ichi Amari

doi:10.1016/S0893-6080(02)00060-6

On-line learning in changing environments with applications in supervised and unsupervised learning

Noboru Murata, Motoaki Kawanabe, Andreas Ziehe^*, Klaus Robert Müller, Shun Ichi Amari

^*Corresponding author for this work

School of Advanced Science and Engineering

Research output: Contribution to journal › Article › peer-review

46 Citations (Scopus)

Abstract

An adaptive on-line algorithm extending the learning of learning idea is proposed and theoretically motivated. Relying only on gradient flow information it can be applied to learning continuous functions or distributions, even when no explicit loss function is given and the Hessian is not available. The framework is applied for unsupervised and supervised learning. Its efficiency is demonstrated for drifting and switching non-stationary blind separation tasks of acoustic signals. Furthermore applications to classification (US postal service data set) and time-series prediction in changing environments are presented.

Original language	English
Pages (from-to)	743-760
Number of pages	18
Journal	Neural Networks
Volume	15
Issue number	4-6
DOIs	https://doi.org/10.1016/S0893-6080(02)00060-6
Publication status	Published - 2002 Jun

Keywords

Adaptive learning rate
Blind source separation
ICA
On-line learning
Stochastic gradient descent
Supervised learning
Unsupervised learning

ASJC Scopus subject areas

Cognitive Neuroscience
Artificial Intelligence

Access to Document

10.1016/S0893-6080(02)00060-6

Cite this

@article{c0a3697183eb42c689c6f21d347e20d9,

title = "On-line learning in changing environments with applications in supervised and unsupervised learning",

abstract = "An adaptive on-line algorithm extending the learning of learning idea is proposed and theoretically motivated. Relying only on gradient flow information it can be applied to learning continuous functions or distributions, even when no explicit loss function is given and the Hessian is not available. The framework is applied for unsupervised and supervised learning. Its efficiency is demonstrated for drifting and switching non-stationary blind separation tasks of acoustic signals. Furthermore applications to classification (US postal service data set) and time-series prediction in changing environments are presented.",

keywords = "Adaptive learning rate, Blind source separation, ICA, On-line learning, Stochastic gradient descent, Supervised learning, Unsupervised learning",

author = "Noboru Murata and Motoaki Kawanabe and Andreas Ziehe and M{\"u}ller, {Klaus Robert} and Amari, {Shun Ichi}",

note = "Funding Information: We thank the participants of the 1997 on-line learning workshop at the Newton Institute in Cambridge for interesting discussions. K.R.M., M.K. and A.Z. gratefully acknowledge partial financial support from DFG under contracts JA 379/91, MU 987/11 and the BMBF BCI project and the EU in the Neurocolt 2 and the BLISS project (IST-1999-14190). ",

year = "2002",

month = jun,

doi = "10.1016/S0893-6080(02)00060-6",

language = "English",

volume = "15",

pages = "743--760",

journal = "Neural Networks",

issn = "0893-6080",

publisher = "Elsevier Limited",

number = "4-6",

}

TY - JOUR

T1 - On-line learning in changing environments with applications in supervised and unsupervised learning

AU - Murata, Noboru

AU - Kawanabe, Motoaki

AU - Ziehe, Andreas

AU - Müller, Klaus Robert

AU - Amari, Shun Ichi

N1 - Funding Information: We thank the participants of the 1997 on-line learning workshop at the Newton Institute in Cambridge for interesting discussions. K.R.M., M.K. and A.Z. gratefully acknowledge partial financial support from DFG under contracts JA 379/91, MU 987/11 and the BMBF BCI project and the EU in the Neurocolt 2 and the BLISS project (IST-1999-14190).

PY - 2002/6

Y1 - 2002/6

N2 - An adaptive on-line algorithm extending the learning of learning idea is proposed and theoretically motivated. Relying only on gradient flow information it can be applied to learning continuous functions or distributions, even when no explicit loss function is given and the Hessian is not available. The framework is applied for unsupervised and supervised learning. Its efficiency is demonstrated for drifting and switching non-stationary blind separation tasks of acoustic signals. Furthermore applications to classification (US postal service data set) and time-series prediction in changing environments are presented.

AB - An adaptive on-line algorithm extending the learning of learning idea is proposed and theoretically motivated. Relying only on gradient flow information it can be applied to learning continuous functions or distributions, even when no explicit loss function is given and the Hessian is not available. The framework is applied for unsupervised and supervised learning. Its efficiency is demonstrated for drifting and switching non-stationary blind separation tasks of acoustic signals. Furthermore applications to classification (US postal service data set) and time-series prediction in changing environments are presented.

KW - Adaptive learning rate

KW - Blind source separation

KW - ICA

KW - On-line learning

KW - Stochastic gradient descent

KW - Supervised learning

KW - Unsupervised learning

UR - http://www.scopus.com/inward/record.url?scp=0036592037&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=0036592037&partnerID=8YFLogxK

U2 - 10.1016/S0893-6080(02)00060-6

DO - 10.1016/S0893-6080(02)00060-6

M3 - Article

C2 - 12371524

AN - SCOPUS:0036592037

SN - 0893-6080

VL - 15

SP - 743

EP - 760

JO - Neural Networks

JF - Neural Networks

IS - 4-6

ER -

On-line learning in changing environments with applications in supervised and unsupervised learning

Abstract

Keywords

ASJC Scopus subject areas

Access to Document

Other files and links

Fingerprint

Cite this