A case study on user evaluation of scientific publication summarization by Japanese students

Shintaro Yamamoto*, Ryota Suzuki, Tsukasa Fukusato, Hirokatsu Kataoka, Shigeo Morishima


研究成果: Article査読

1 被引用数 (Scopus)


Summaries of scientific publications enable readers to gain an overview of a large number of studies, but users’ preferences have not yet been explored. In this paper, we conduct two user studies (i.e., short-and long-term studies) where Japanese university students read summaries of English research articles that were either manually written or automatically generated using text summarization and/or machine translation. In the short-term experiment, subjects compared and evaluated the two types of summaries of the same article. We analyze the characteristics in the generated summaries that readers regard as important, such as content richness and simplicity. The experimental results show that subjects are mainly judged based on four criteria, including content richness, simplicity, fluency, and format. In the long-term experiment, subjects read 50 summaries and answered whether they would like to read the original papers after reading the summaries. We discuss the characteristics in the summaries that readers tend to use to determine whether to read the papers, such as topic, methods, and results. The comments from subjects indicate that specific components of scientific publications, including research topics and methods, are important to judge whether to read or not. Our study provides insights to enhance the effectiveness of automatic summarization of scientific publications.

ジャーナルApplied Sciences (Switzerland)
出版ステータスPublished - 2021 7月 2

ASJC Scopus subject areas

  • 材料科学(全般)
  • 器械工学
  • 工学(全般)
  • プロセス化学およびプロセス工学
  • コンピュータ サイエンスの応用
  • 流体および伝熱


「A case study on user evaluation of scientific publication summarization by Japanese students」の研究トピックを掘り下げます。これらがまとまってユニークなフィンガープリントを構成します。