Latent Normalizing Flows for Many-to-Many Cross-Domain Mappings
Abstract
Learned joint representations of images and text form the backbone of several important cross-domain tasks such as image captioning. Prior work mostly maps both domains into a common latent representation in a purely supervised fashion. This is rather restrictive, however, as the two domains follow distinct generative processes. Therefore, we propose a novel semi-supervised framework, which models shared information between domains and domain-specific information separately. The information shared between the domains is aligned with an invertible neural network. Our model integrates normalizing flow-based priors for the domain-specific information, which allows us to learn diverse many-to-many mappings between the two domains. We demonstrate the effectiveness of our model on diverse tasks, including image captioning and text-to-image synthesis.
- Publication:
-
arXiv e-prints
- Pub Date:
- February 2020
- DOI:
- 10.48550/arXiv.2002.06661
- arXiv:
- arXiv:2002.06661
- Bibcode:
- 2020arXiv200206661M
- Keywords:
-
- Computer Science - Computer Vision and Pattern Recognition
- E-Print:
- Published as a conference paper at ICLR 2020