You cannot delete a custom theme you currently have applied. It isn't immediately clear that Profile Themes can be deleted, especially when the only custom theme is equipped.

I didn't understand briefly that deleting Profile Themes was something that could be done since I did not want to make another theme knowing the possibility it was stuck there.