An Approach to Aligning Categorical and Continuous Time Series for Studying the Dynamics of Complex Human Behavior

Kentaro Kodama*, Daichi Shimizu, Rick Dale, Kazuki Sekine

*Corresponding author for this work

Research output: Contribution to journalArticlepeer-review

2 Citations (Scopus)


An emerging perspective on human cognition and performance sees it as a kind of self-organizing phenomenon involving dynamic coordination across the body, brain and environment. Measuring this coordination faces a major challenge. Time series obtained from such cognitive, behavioral, and physiological coordination are often complicated in terms of non-stationarity and non-linearity, and in terms of continuous vs. categorical scales. Researchers have proposed several analytical tools and frameworks. One method designed to overcome these complexities is recurrence quantification analysis, developed in the study of non-linear dynamics. It has been applied in various domains, including linguistic (categorical) data or motion (continuous) data. However, most previous studies have applied recurrence methods individually to categorical or continuous data. To understand how complex coordination works, an integration of these types of behavior is needed. We aimed to integrate these methods to investigate the relationship between language (categorical) and motion (continuous) directly. To do so, we added temporal information (a time stamp) to categorical data (i.e., language), and applied joint recurrence analysis methods to visualize and quantify speech-motion coordination coupling during a rap performance. We illustrate how new dynamic methods may capture this coordination in a small case-study design on this expert rap performance. We describe a case study suggesting this kind of dynamic analysis holds promise, and end by discussing the theoretical implications of studying complex performances of this kind as a dynamic, coordinated phenomenon.

Original languageEnglish
Article number614431
JournalFrontiers in Psychology
Publication statusPublished - 2021 Apr 16


  • quantification
  • rap
  • recurrence analysis
  • speech-motion coupling
  • visualization

ASJC Scopus subject areas

  • Psychology(all)


Dive into the research topics of 'An Approach to Aligning Categorical and Continuous Time Series for Studying the Dynamics of Complex Human Behavior'. Together they form a unique fingerprint.

Cite this