Preventing dimensional collapse in self-supervised learning

Steve Oudot (INRIA)

Subtitle: "A tale of sphere packings, minimum spanning trees, and topological data analysis"

Summary

This talk is about recent work by my PhD student Julie Mordacq, who proposed a new regularizer based on minimum spanning tree length to prevent dimensional collapse in self-supervised learning. Focusing specifically on the joint-embedding problem, her work draws connections to dimension estimation, sphere packings, and topological data analysis. In the talk I will describe these connections and explain how they help address the problem via a remarkably simple method.