I see that you’re downloading your episodes. At first I wondered whether this could be the same root cause as another bug occurring when streaming, covered in thread Player audio randomly stops but seekbar continues progressing. I would suggest you tried the beta builds from there, but realize you’re unable to reliably trigger it?
In November of 2022 there was the thread Random bits of other podcasts cut into the intended one that should be playing, which seems to be similar to what you’re encountering. Right? That issue ought to be resolved long before 3.4.0, but maybe it could be worth reading through the thread in the hope of gaining some bright idea of a likely common factor?