I’ve chosen AntennaPod for the children specifically because it is so offline-friendly and bare-bones. Interface is configured to only show their playlist, so there is nothing for them to go digging in, just the list that’s for them. And the device is also in flight mode to prevent accidental internet exposure. So you might want to try that and see if it suits you.
(The only thing I think is missing is the discussed-to-death end of episode sleep timer. That would make it perfect from my parental perspective.)