Deep generative model for Learning Streamline Embeddings from Tractography

This is a research project I’ve been working on since September 2022 in Dr. Paul Thompson’s lab at the Imaging Genetics Center at University of Southern California since my research rotation. Our goal is to utilize deep representation learning for analyzing tractography data for population analysis.


Tractography generated from diffusion MRI are often used to study white matter pathways. Each whole brain tractogram can generate from 500K to 1M streamlines, making processing and downstream analyses very computationally expensive. In this work, we present a framework using deep representation learning based on a convolutional variational autoencoder (ConvVAE) to learn low-dimensional embeddings of white matter tracts.


This figure plots 2D streamlines embeddings generated from a ConvVAE trained on a healthy control subject from ADNI3.

Figure (a) and (c) show similar cluster patterns for corresponding bundles from two subjects. Figure (b) and (d) shows distinction between left and right hemisphere, with the left hemisphere embeddings spanning more latent space.

2D Embeddings

This figure shows the 2D embeddings, original streamlines and reconstructed streamlines for 4 select tracts.

Bundle Reconstruction

ConvVAE generates 2D embeddings that preserve bundles’ spatial and shape information. It learns a smooth latent space from streamlines, which allows for meaningful decodings from sampled points and can be directly applied to new data. Distance-based algorithms can be used in downstream tasks, such as bundle labeling and intersubject comparisons.

Code Availability