Having dedicated apps for each type - music/podcasts/audiobooks is not worst solution, it just requires changing your approach a little. For audiobooks try AudioAnchor, it’s open source audiobook player, available on Github and F-droid. For music player try Music or Vinyl, both available on F-droid and Github (I would link to them but I’m not allowed to put more 2 links in post).