Not Every Sound Deserves Your Attention