Aesthetic Photo Collage with Deep Reinforcement Learning

10/19/2021
by   Mingrui Zhang, et al.
9

Photo collage aims to automatically arrange multiple photos on a given canvas with high aesthetic quality. Existing methods are based mainly on handcrafted feature optimization, which cannot adequately capture high-level human aesthetic senses. Deep learning provides a promising way, but owing to the complexity of collage and lack of training data, a solution has yet to be found. In this paper, we propose a novel pipeline for automatic generation of aspect ratio specified collage and the reinforcement learning technique is introduced in collage for the first time. Inspired by manual collages, we model the collage generation as sequential decision process to adjust spatial positions, orientation angles, placement order and the global layout. To instruct the agent to improve both the overall layout and local details, the reward function is specially designed for collage, considering subjective and objective factors. To overcome the lack of training data, we pretrain our deep aesthetic network on a large scale image aesthetic dataset (CPC) for general aesthetic feature extraction and propose an attention fusion module for structural collage feature representation. We test our model against competing methods on two movie datasets and our results outperform others in aesthetic quality evaluation. Further user study is also conducted to demonstrate the effectiveness.

READ FULL TEXT

page 1

page 5

page 6

page 7

research
05/18/2023

Automatic Design Method of Building Pipeline Layout Based on Deep Reinforcement Learning

The layout design of pipelines is a critical task in the construction in...
research
09/21/2021

AutoPhoto: Aesthetic Photo Capture using Reinforcement Learning

The process of capturing a well-composed photo is difficult and it takes...
research
05/19/2021

PPR10K: A Large-Scale Portrait Photo Retouching Dataset with Human-Region Mask and Group-Level Consistency

Different from general photo retouching tasks, portrait photo retouching...
research
06/17/2023

Deep Reinforcement Learning for Flipper Control of Tracked Robots

The autonomous control of flippers plays an important role in enhancing ...
research
06/30/2023

Federated Multi-Agent Deep Reinforcement Learning for Dynamic and Flexible 3D Operation of 5G Multi-MAP Networks

This paper addresses the efficient management of Mobile Access Points (M...
research
03/06/2020

Cost-Sensitive Portfolio Selection via Deep Reinforcement Learning

Portfolio Selection is an important real-world financial task and has at...
research
03/02/2023

Reinforced Labels: Multi-Agent Deep Reinforcement Learning for Point-feature Label Placement

Over the past few years, Reinforcement Learning combined with Deep Learn...

Please sign up or login with your details

Forgot password? Click here to reset