Abstract: VAEs and Flows are two of the most popular methods for density estimation. Well, actually GANs are more popular, but if we can show that VAEs and Flows are really two sides of the same coin, then together they stand strong and can beat GANs (maybe). A flow is based on deterministically transforming an input density through an invertible transformation to a target density. If the transformation changes a volume element, we pick up a log-Jacobian term. After decomposing the ELBO in the only way that was not yet considered in the literature, we find that the log-Jacobian corresponds to log[p(x|z)/q(z|x)] of a VAE, where the maps q and p are now stochastic. This suggests a third possibility that bridges the gap between the two: a surjective map which is deterministic and surjective in one direction, and probabilistic in the reverse direction. We find that these ideas unify many methods out there in the literature, such as dequantization, and augmented flows, and we also add a few new methods of our own based on our SurVAE Flows framework. If time permits, I will also say a few words on a new type of flow based on the exponential map which is trivially invertible and adds a new tool to the invertible flows toolbox.

Joint work with Didrik Nielsen, Priyank Jaini, Emiel Hoogeboom.

Bio: Max Welling is a research chair in Machine Learning at the University of Amsterdam and a VP Technologies at Qualcomm. He has a secondary appointment as a senior fellow at the Canadian Institute for Advanced Research (CIFAR) and fellow and founding board member of the European Lab for Learning and Intelligent Systems (ELLIS). He is co-founder of “Scyfer BV” a university spin-off in deep learning which was acquired by Qualcomm in summer 2017. In the past he held postdoctoral positions at Caltech (’98-’00), UCL (’00-’01) and the U. Toronto (’01-’03). He received his PhD in ’98 under supervision of Nobel laureate Prof. G. 't Hooft. Max Welling has served as associate editor in chief of IEEE TPAMI from[masked]. He serves on the board of the NeurIPS foundation since 2015 and has been program chair and general chair of NeurIPS in 2013 and 2014 respectively. He was also program chair of AISTATS in 2009 and ECCV in 2016 and general chair of MIDL 2018. He is recipient of the ECCV Koenderink Prize in 2010. Welling is co-founder and board member of the Innovation Center for AI (ICAI). He directs the Amsterdam Machine Learning Lab (AMLAB), and co-directs the Qualcomm-UvA deep learning lab (QUVA) and the Bosch-UvA Deep Learning lab (DELTA).

Reference:
https://arxiv.org/pdf/2006.01910.pdf

Sponsors
https://evolution.ai/ : Machines that Read - Intelligent data extraction from corporate and financial documents.

Man AHL: At Man AHL, we mix machine learning, computer science and engineering with terabytes of data to invest billions of dollars every day.