Thanks, @SpiKe.
My problems seem to occur on both SDL and GLX, and it seems to be Xonotic only.
This has been going on for about 2 years. I switched to pipewire for audio work (btw, so happy that I no longer need to switch between PulseAudio and Jack 1, and Jack2!!), and all other programs I run work well (well, some bugs in the beginning, but very few nowadays). However, Xonotic seems to almost always have some problems. It's not very consistent either:
1. Occasionally I have to start Xonotic a few times to even hear sound in the beginning menu.
2. Even if I manage to have sound in the menu, quite often the sound will drop as soon as I join a server. (I still see the process in pavucontrol, via pipewire-pulse, but no sound).
3. Just as often I will have audio in the game for a while, and then suddenly it's gone several minutes into the game. I will then go back to step 1.
Any suggestion on how to narrow down the problem?