No voice chat coming through headset

So I'm having a problem that when I connect my headset I can't hear anyone on voice chat. I play on an xbox one and I'm pretty sure it's the game because we switched to warzone after cold war and the voice chat was working perfectly. My friends are saying they can hear me through my mic and when I don't have the mic plugged in I can hear them through my tv speakers, but as soon as I plug in only thing I can hear is the music and in game sounds. Anyone else having this problem?