Trainable Self-Guided Filter for Multi-Focus Image Fusion

Abstract

Cameras are limited in their ability to capture all-in-focus images due to their limited depth of field. This results in blurriness for objects too far in front of or behind the focused point. To overcome this limitation, multi-focus image fusion (MFIF) approaches have been proposed. Although recent MFIF methods have shown promising results for this task, they still need to be improved in terms of artifacts and color degradation. Motivated by these observations, in this paper, we propose a new Generative Adversarial Network (GAN)-based MFIF model to improve fusion quality by predicting more accurate focus maps thanks to a trainable guided filter we incorporated. The proposed model comprises an encoder-decoder network, and a trainable self-guided filtering (TSGF) module that is specifically designed to enhance spatial consistency in the predicted focus map and to eliminate the requirements of post-processing in existing GAN-based methods. The encoder-decoder network first predicts raw focus maps, which are then passed to the TSGF to produce the final focus maps. To train the proposed model effectively, we define three objectives: L1 loss, GAN loss, and Focal Frequency Loss (FFL) in the frequency domain. L1 loss is defined on ground-truth and predicted focus maps, whereas GAN loss and FFL are defined on ground-truth all-in-focus images and fused images. Experimental results show that the proposed approach outperforms the existing GAN-based methods and achieves highly competitive performance with state-of-the-art methods in terms of standard quantitative image fusion metrics and visual quality on three MFIF benchmark datasets.