Adaptive energy selection for content-Aware image resizing

Kazuma Sasaki, Yuya Nagahama, Zheng Ze, Satoshi Iizuka, Edgar Simo-Serra, Yoshihiko Mochizuki, Hiroshi Ishikawa

Research output: Chapter in Book/Report/Conference proceedingConference contribution


Content-Aware image resizing aims to reduce the size of an image without touching important objects and regions. In seam carving, this is done by assessing the importance of each pixel by an energy function and repeatedly removing a string of pixels avoiding pixels with high energy. However, there is no single energy function that is best for all images: The optimal energy function is itself a function of the image. In this paper, we present a method for predicting the quality of the results of resizing an image with different energy functions, so as to select the energy best suited for that particular image. We formulate the selection as a classification problem; i.e., we 'classify' the input into the class of images for which one of the energies works best. The standard approach would be to use a CNN for the classification. However, the existence of a fully connected layer forces us to resize the input to a fixed size, which obliterates useful information, especially lower-level features that more closely relate to the energies used for seam carving. Instead, we extract a feature from internal convolutional layers, which results in a fixed-length vector regardless of the input size, making it amenable to classification with a Support Vector Machine. This formulation of the algorithm selection as a classification problem can be used whenever there are multiple approaches for a specific image processing task. We validate our approach with a user study, where our method outperforms recent seam carving approaches.

Original languageEnglish
Title of host publicationProceedings - 4th Asian Conference on Pattern Recognition, ACPR 2017
PublisherInstitute of Electrical and Electronics Engineers Inc.
Number of pages6
ISBN (Electronic)9781538633540
Publication statusPublished - 2018 Dec 13
Event4th Asian Conference on Pattern Recognition, ACPR 2017 - Nanjing, China
Duration: 2017 Nov 262017 Nov 29

Publication series

NameProceedings - 4th Asian Conference on Pattern Recognition, ACPR 2017


Other4th Asian Conference on Pattern Recognition, ACPR 2017


  • Content-Aware image resizing Seam carving Convolutional Neural Network

ASJC Scopus subject areas

  • Artificial Intelligence
  • Computer Vision and Pattern Recognition
  • Signal Processing


Dive into the research topics of 'Adaptive energy selection for content-Aware image resizing'. Together they form a unique fingerprint.

Cite this