with Seth Axen, Sean Pinkney, Nikolas Siccha, Bob Carpenter

We worked on transformations for constrained parameters for sampling with Hamiltonian Monte Carlo (HMC). Various mappings exist in the literature and implementations, for constraints like simplexes, positive semi-definite matrices, unit vectors, etc. However, choosing among these diffeomorphisms for efficiency and robustness is challenging due to trade-offs in posterior geometry, simplicity of implementation, statistical efficiency, and numerical stability. This project involves systematically reviewing statistical properties, computational efficiency, and geometry of existing canonical transformations. We propose improvements in terms of numerical stability and mathematical representations, as well as explanations for certain peculiarities based on the geometry. Over the summer I worked on transformations specifically on a simplex, after which we continued collaborating and are in the process of finishing up a paper. We worked through the Jacobians for these transformations, and provided an analysis using effective sample size, Leapfrog steps and Hessian information, informed by contextual needs of sampling uniformly on a simplex as well as sampling difficulties because of high curvature regions. The rest of the project, such as alternatives for transforming densities over the Stiefel Manifold (semi-orthogonal matrices) delves into a combination of sampling and differential geometry. In future scope, I am interested in connections between this project and the notion of incomplete manifesting as modifications to the metric geometry for Riemannian HMC. We wrote Stan and JAX implementations for the transformations | see linked repository.

with Aki Vehtari and Niko Siccha

Diagnosing Hamiltonian Monte Carlo in regions of high curvature

Reparametrizations as a way to augment the posterior geometry in well known pathological cases

linear and non-linear transformations on the posterior to compare relative correlation and sampling difficulty between transformed parameters

whitening transformations on Hessians instead of covariance matrices, these whitening transformations were constructed from sample covariance. The relative information of the transformed or whitened Hessian provides information about the local variation in curvature.

relationships between absolute Hamiltonian error and variation in curvature, storing initial momentum for exact recovery of errant trajectories for better identification of regions leading to divergences.

A well known problem with a lot of these diagnostic measures that involve linear transformations on parameters being sampled is the computational cost of matrix operations, especially in high dimensions.