Targeted Neural Audio Embeddings
Task-targeted brain encoding from Qwen-Audio speech features to cortical activations.
Targeted Neural Audio Embeddings develops a brain encoding pipeline that maps complex speech features to cortical responses by isolating task-relevant audio subspaces.
Highlights
- Built a brain encoding model to map Qwen-Audio speech representations to cortical activations by extracting targeted task subspaces from large audio-language embeddings.
- Processed 150k+ audio examples across 15 benchmark datasets to engineer targeted embeddings that isolate specific auditory signals such as emotion and reasoning.
- Performed rigorous dimensionality sweeps across network architectures to maximize the performance gap between on-task signal capture and task-irrelevant noise.
Context
Research project with Columbia Zuckerman Institute.