I toggled the setting to not remove from queue, but they do anyhow.
May I suggest that the feature “lock queue” prevents the episodes from deleting after being played. That is more intuitive to me, and actually what I thought the setting was for.
The setting you refer to doesn’t control whether played episodes are auto-removed from the queue (there is no setting for that at the moment).
If “delete removes from queue” is on, deleting a downloaded episode also removes it from the queue; if it’s off, deleting a downloaded episode does not remove it from the queue.
“Lock queue” only disables reordering episodes in the queue.
A discussion related to the behaviour you are describing is taking place in this thread, so I suggest you follow that.