Lidongsheng Xing

This doctoral thesis develops and validates a feature identification approach for extracting sound features from neural activity, with the goal of enabling the modulation of cortical and behavioral responses toward designed patterns. By combining high temporal resolution electroencephalography (EEG) and high spatial resolution functional magnetic resonance imaging (fMRI), this research establishes a methodological framework that integrates backward reconstruction and forward synthesis to link neural responses to acoustic stimulus features across temporal dynamics, semantic categories, and hemispheric asymmetries.

In this dissertation, we conducted three empirical studies to test the feasibility and generalizability of this feature identification approach across different dimensions of auditory processing. In Chapter 2, we applied a backward reconstruction method using a linear temporal response function (TRF) to reconstruct dynamic loudness patterns from EEG responses to basic tone sequences. We then developed a behavioral and neural validation paradigm to assess whether these reconstructions—originally evaluated only by correlation metrics—hold perceptual and neural significance for listeners. This established a more rigorous framework for evaluating reconstruction fidelity beyond conventional acoustic-domain metrics. In Chapter 3, we implemented a forward synthesis method using a deep neural network (DNN) to sonify layer-wise categorical sound representations embedded in the model. We then designed perceptual and neural validation experiments to test whether the synthesized sounds—though acoustically unnatural—could serve as effective probes for investigating causal relationships between DNN-derived features and auditory cortical processing. In Chapter 4, we extended this forward synthesis framework to target hemispheric differences, synthesizing sounds that maximized the predicted response difference between the left and right auditory cortices. Subsequently, we developed an acoustic feature analysis pipeline to characterize the distinguishing features of these left- and right-activating sounds using multiple acoustic models (e.g., spectrotemporal modulation and sound texture statistics). The methodological contributions and generalizability of this feature identification approach are discussed in the final chapter.