Model parameter learning using Kullback–Leibler divergence

Chungwei Lin*, Tim K. Marks, Milutin Pajovic, Shinji Watanabe, Chih kuan Tung

*Corresponding author for this work

Research output: Contribution to journalArticlepeer-review


In this paper, we address the following problem: For a given set of spin configurations whose probability distribution is of the Boltzmann type, how do we determine the model coupling parameters? We demonstrate that directly minimizing the Kullback–Leibler divergence is an efficient method. We test this method against the Ising and XY models on the one-dimensional (1D) and two-dimensional (2D) lattices, and provide two estimators to quantify the model quality. We apply this method to two types of problems. First, we apply it to the real-space renormalization group (RG). We find that the obtained RG flow is sufficiently good for determining the phase boundary (within 1% of the exact result) and the critical point, but not accurate enough for critical exponents. The proposed method provides a simple way to numerically estimate amplitudes of the interactions typically truncated in the real-space RG procedure. Second, we apply this method to the dynamical system composed of self-propelled particles, where we extract the parameter of a statistical model (a generalized XY model) from a dynamical system described by the Viscek model. We are able to obtain reasonable coupling values corresponding to different noise strengths of the Viscek model. Our method is thus able to provide quantitative analysis of dynamical systems composed of self-propelled particles.

Original languageEnglish
Pages (from-to)549-559
Number of pages11
JournalPhysica A: Statistical Mechanics and its Applications
Publication statusPublished - 2018 Feb 1
Externally publishedYes


  • Kullback–Leibler divergence
  • Model learning
  • Real-space renormalization

ASJC Scopus subject areas

  • Statistics and Probability
  • Condensed Matter Physics


Dive into the research topics of 'Model parameter learning using Kullback–Leibler divergence'. Together they form a unique fingerprint.

Cite this