Towards Automatic Evaluation of Reused Answers in Community Question Answering

Hsin Wen Liu*, Sumio Fujita, Tetsuya Sakai

*Corresponding author for this work

Research output: Chapter in Book/Report/Conference proceedingConference contribution


We consider the problem of reused answer retrieval for community question answering (CQA): given a question q, retrieve answers (Formula presented) posted in response to other questions qi(≠ q), where (Formula presented) serves as an answer to q. While previous work evaluated this task by manually annotating the relationship between q and (Formula presented), this approach does not scale for large-scale CQA sites. We therefore explore an automatic evaluation method for reused answer retrieval, which computes nDCG by defining the gain value of each retrieved answer as a ROUGE score that treats the original answers to q as gold summaries. Our answer retrieval experiment suggests that effective reused answer retrieval systems may not be the same as effective gold answer retrieval systems. We provide case studies to discuss the benefits and limitations of our approach.

Original languageEnglish
Title of host publicationInformation Retrieval Technology - 15th Asia Information Retrieval Societies Conference, AIRS 2019, Proceedings
EditorsFu Lee Wang, Haoran Xie, Wai Lam, Aixin Sun, Lun-Wei Ku, Tianyong Hao, Wei Chen, Tak-Lam Wong, Xiaohui Tao
Number of pages7
ISBN (Print)9783030428341
Publication statusPublished - 2020
Event15th Asia Information Retrieval Societies Conference, AIRS 2019 - Kowloon, Hong Kong
Duration: 2019 Nov 72019 Nov 9

Publication series

NameLecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume12004 LNCS
ISSN (Print)0302-9743
ISSN (Electronic)1611-3349


Conference15th Asia Information Retrieval Societies Conference, AIRS 2019
Country/TerritoryHong Kong


  • Answer retrieval
  • Community question answering

ASJC Scopus subject areas

  • Theoretical Computer Science
  • Computer Science(all)


Dive into the research topics of 'Towards Automatic Evaluation of Reused Answers in Community Question Answering'. Together they form a unique fingerprint.

Cite this