PIFE: Permutation Invariant Feature Extractor for Danmaku Games

Takuto Itoi, Edgar Simo-Serra

Research output: Chapter in Book/Report/Conference proceedingConference contribution


Dealing with unstructured complex patterns provides a challenge to existing reinforcement patterns. In this research, we propose a new model to overcome the difficulty in challenging danmaku games. Touhou Project is one of the bestknown games in the bullet hell genre also known as danmaku, where a player has to dodge complex patterns of bullets on the screen. Furthermore, the agent needs to react to the environment in real-time, which made existing methods having difficulties processing the high-volume data of objects; bullets, enemies, etc. We introduce an environment for the Touhou Project game'Phantasmagoria of Flower View.' which manipulates the memory of the running game and enables to control the character. However, the game state information consists of unstructured and unordered data not amenable for training existing reinforcement learning models, as they are not invariant to order changes in the input. To overcome this issue, we propose a new pooling-based reinforcement learning approach that is able to handle permutation invariant inputs by extracting abstract values and merging them in an order-independent way. Experimental results corroborate the effectiveness of our approach which shows significantly increased scores compared to existing baseline approaches.

Original languageEnglish
Title of host publication2022 IEEE Conference on Games, CoG 2022
PublisherIEEE Computer Society
Number of pages4
ISBN (Electronic)9781665459891
Publication statusPublished - 2022
Event2022 IEEE Conference on Games, CoG 2022 - Beijing, China
Duration: 2022 Aug 212022 Aug 24

Publication series

NameIEEE Conference on Computatonal Intelligence and Games, CIG
ISSN (Print)2325-4270
ISSN (Electronic)2325-4289


Conference2022 IEEE Conference on Games, CoG 2022


  • permutation invariance
  • pooling
  • reinforcement learning
  • touhou

ASJC Scopus subject areas

  • Artificial Intelligence
  • Computer Graphics and Computer-Aided Design
  • Computer Vision and Pattern Recognition
  • Human-Computer Interaction
  • Software


Dive into the research topics of 'PIFE: Permutation Invariant Feature Extractor for Danmaku Games'. Together they form a unique fingerprint.

Cite this