Accelerating generative models and nonconvex optimisation (IN-PERSON)

The Alan Turing Institute is the national institute for data science and artificial intelligence.

The Alan Turing Institute

This workshop will focus on theoretical and methodological challenges related to generative models and optimisation.

96 Euston Rd

A﻿bout the eventGenerative models have dominated the AI scene with their impressive performance while their precise theoretical and algorithmic understanding is still unclear. This workshop will act as the follow-up workshop of the Theory and Methods Challenges Fortnight on <a href="https://www.turing.ac.uk/research/theory-and-method-challenge-fortnights/accelerating-generative-models-and-nonconvex" target="_blank" data-msys-clicktrack="0" rel="nofollow noopener noreferrer">&quot;Accelerating Generative Models and Nonconvex Optimisation&quot;</a> and will focus on theoretical and methodological challenges related to generative models and optimisation, solutions to some of those challenges, and exciting future directions on the foundations of generative models. The workshop will also act as a vehicle for broad engagement with the Data Science and AI community worldwide on the theory and applications of generative models.T﻿he event will have a mixture of talks and software demos that were developed by a team of researchers from around the world during the Theory and Methods Challenge Fortnight event that took place in June and September 2022.International speakers that contributed to the TMCF challenge and others have been invited including Arnaud Doucet, Arnak Dalalyan and Emmanuel Gobet; please find the programme below.This event will be delivered in hybrid format, there is a limit of 45 in-person tickets for this event. If you prefer to attend the event online please reserve your place <a href="https://www.eventbrite.co.uk/e/accelerating-generative-models-and-nonconvex-optimisation-online-tickets-541349329417" target="_blank" data-msys-clicktrack="0" rel="nofollow noopener noreferrer">here</a>.P﻿lease note the coffee and lunch will be provided, thus we require people attending in person to reserve their place by 16 March 2023.O﻿rganisers:D﻿r Deniz Akyildiz, Lecturer at Imperial College London, UK.P﻿rofessor Sotirios Sabanis, Turing Fellow and University of Edinburgh, UK and NTU Athens, Greece.I﻿f you require further information please email tmcf@turing.ac.ukC﻿urrently, there is a call for new challenges, more information on the <a href="https://www.turing.ac.uk/tmcf2023" target="_blank" data-msys-clicktrack="0" rel="nofollow noopener noreferrer">call webpage</a>.T﻿his workshop is part of the <a href="https://ai-uk.turing.ac.uk/fringe-events/" target="_blank" data-msys-clicktrack="0" rel="nofollow noopener noreferrer">AI UK 2023</a> Fringe Events hosted by The Alan Turing Institute.

Agenda

Welcome

We investigate new parametrizations based on neural networks in order to approximate and sample multi-variate extreme values, especially in the case of heavy-tailed distributions. We discuss two approaches. First, transformations of Feedforward neural networks based on Rectified linear units (ReLU) are used. An analysis of the uniform error between the extreme quantile and its GAN approximation is provided, and shows that second-order parameters of the marginal data distributions play an important role. Second, eLU based NN are used, to efficiently get rid of the bias term in tails approximation, in the presence of arbitrary high-order parameters. These results are illustrated on synthetic and real data. Bio: Emmanuel Gobet graduated from Ecole Polytechnique - Paris in 1995, he got a PhD degree in probability at University Paris Diderot. He took different academic positions at University Pierre et Marie Curie, Grenoble Institute of Technology and he is currently Professor at Ecole Poly

Generative modelling of extremes with neural networks

In this talk we discuss a new interacting particle system used for implementing an expectation maximization (EM) procedure (or more generally, to optimize over the parameters of a latent variable model). This continuous-time interacting particle system has the property that it can be seen as a Langevin diffusion over an extended state space, where the number of particles acts almost like an inverse parameter in classical settings for optimization. This then allow for use of the well-developed theory Langevin diffusions to prove nonasymptotic concentration bounds for the optimization error of the maximum marginal likelihood estimator. In this talk we shall place this new algorithm in the context of existing approaches, and also discuss the structure of our proof and how it naturally lends itself to generalisation.

Interacting Particle Systems for EM

Coffee break

Parametric PDEs are ubiquitous in engineering practice. Being able to solve these problems efficiently is of great interest to engineers for model calibration, iterative design, and UQ. In this talk we discuss some of the recent advances in the learning of probabilistic emulators for parameters PDEs. We go through some of the work done in the TMCF on boundary condition enforcement for PINN based physics driven deep latent variable models. We also go over the TMCF work leading to the paper: ''Random Grid Neural Processes for parametric differential equations.

Improving the variational learning of physics driven neural generative models

DiffusionJAX is an open-sourced python package for verifying and applying new research in diffusion modelling, a high-dimensional sampling framework that is developing at a fast pace. In particular, some of the features of the package are • An accessible research tool that can be used for replicable and reproducible numerical experiments. • Training scores on (possibly, image) data and sampling from the generative model. Also, inverse problems, such as image inpainting. • Not many lines of code. Easy to use, extendable, easy to get started with the examples.

DiffusionJAX

Lunch break

Denoising diffusion models are a novel powerful class of techniques for generative modelling and inference. We will introduce these methods and present some of their limitations. We will then discuss how recent alternative based on transport ideas can resolve some of these limitations. In particular, we will focus on diffusion Schrodinger bridge, an entropy-regularized version of optimal transport, and discuss how it can be approximated numerically.

From denoising diffusion models to transport for generative modelling

Langevin Monte Carlo is an efficient and widely used method for generating random samples from a given target distribution in a high-dimensional Euclidean space. Various variants of the Langevin Monte Carlo method have been proposed and discussed in the literature; depending on the properties of the target distribution, some variants may be preferred to others. Among these variants, it has been shown that the Randomized Midpoint Langevin Monte Carlo (RMP-LCM) method has the best known non-asymptotic theoretical guarantees on the sampling error, when the log-density of the target distribution has a continuous Lipschitz gradient. The objective of this talk is to review these results, as well as to present some extensions and improvements.

Langevin Monte-Carlo: Randomised mid-point method revisited

Denoising diffusion models are a recent class of generative models exhibiting state-of-the-art performance in image and audio synthesis. Such models approximate the time-reversal of a forward noising process from a target distribution to a reference density, which is usually Gaussian. Despite their strong empirical results, the theoretical analysis of such models remains limited. In particular, all current approaches crucially assume that the target density admits a density w.r.t. the Lebesgue measure. This does not cover settings where the target distribution is supported on a lower-dimensional manifold or is given by some empirical distribution. In this paper, we bridge this gap by providing the first convergence results for diffusion models in this more general setting. In particular, we provide quantitative bounds on the Wasserstein distance of order one between the target data distribution and the generative distribution of the diffusion model.

Convergence of denoising diffusion models under the manifold hypothesis

Closing remarks

Our {communityGuidelinesLink} describe the sort of content we prohibit on Eventbrite. If you suspect an event may be in violation, you can report it to us so we can investigate.

To report other categories of prohibited or illegal content, submit {link}

Accelerating generative models and nonconvex optimisation (IN-PERSON)

More events from The Alan Turing Institute

Discover more events from The Alan Turing Institute, from Science & Tech to other experiences you might love.

Accelerating generative models and nonconvex optimisation (IN-PERSON)

More events from The Alan Turing Institute

Discover more events from The Alan Turing Institute, from Science & Tech to other experiences you might love.

More Science & Tech events

Browse more Science & Tech events with different dates, prices, and formats to find your next great experience.