Get more from less: Differential neural decoding for effective reconstruction from limited training data
19 December 2023
Thirza Dado & Umut Güçlü
Abstract
Decoding complex perceptual stimuli from limited noisy neural recordings remains a key challenge in systems neuroscience. Conventional neural decoders face difficulties due to high-dimensional nonlinear stimulus-response relationships and variability in neural responses. Recent approaches address these issues by mapping neural data to the latent spaces of generative models such as Generative Adversarial Networks (GANs) but learning robust mappings from instantaneous neural responses to stimuli or latents remains fundamentally challenging. Here, we present a novel differential neural decoding paradigm that reconstructs stimuli from relative changes between neural responses, offering inherent denoising and quadratic expansion of the training data. We leverage Euclidean geometry exhibited by GAN latent spaces to enable geometric stimulus manipulations aligned with how generators manipulate factors of variation. That is, we combine the predicted latent offsets, which represent perceived stimulus changes, with a reference latent code via vector addition. Overall, our geometrically principled framework facilitates more effective reconstruction of complex stimuli from limited noisy biological response data by emphasizing relative sensory processing.