Style Adaptation Based On Image Processing Methods Using Cyclegan

Keywords:
Style Adaptation, Generative Adversarial Networks, Cycle-Consistency, Semi-Supervised Learning, Bootstrapping

Abstract:
Cycle-Consistent Generative Adversarial Networks (CycleGANs) are able to provide a highly under-constrained mapping between input and output data samples, i.e., source and target data domain, in cases when the aligned dataset is unavailable, in an unsupervised training fashion, using cycle-consistency loss mechanisms. On the other hand, most image-to-image and speech-to-speech translation tasks use the aligned, i.e., paired input-output training datasets. A large amount of data is necessary to train such architectures, while one of the domains could be scarce. Several possible improvements to the original CycleGAN architecture are analysed in this paper for the cases when only a small percentage of training samples are aligned among source and target data domains. A semi-supervised approach is proposed to achieve better translation accuracy and prevent overfitting of the scarce data domain discriminator during initial training iterations. The training database is augmented by adding samples generated by inverse CycleGAN mappings after several training epochs (when the network is sufficiently trained) into the training pool of the discriminator of scarce, i.e., reduced data domain. An additional optimization constraint is also proposed, aligning probability distributions of feature maps belonging to the same-depth neural network layers of direct GAN encoder and inverse GAN decoder, to reinforce resemblance among object representations in various data domains. Significantly better performances are obtained using proposed improvements in both image-to-image and speech-to-speech translation tasks, by observing standard qualitative and quantitative measures, in comparison to the baseline CycleGAN training approach.

CITATION:

IEEE format

Copy to clipboard

B. Popović, “Style Adaptation Based On Image Processing Methods Using Cyclegan,” in Sinteza 2023 - International Scientific Conference on Information Technology, Computer Science, and Data Science, Belgrade, Singidunum University, Serbia, 2023, pp. 9-16. doi:10.15308/Sinteza-2023-9-16

APA format

Copy to clipboard

Popović, B. (2023). Style Adaptation Based On Image Processing Methods Using Cyclegan. Paper presented at Sinteza 2023 - International Scientific Conference on Information Technology, Computer Science, and Data Science. doi:10.15308/Sinteza-2023-9-16

BibTeX format
Download

RefWorks Tagged format
Download