Moving your mouse around with either the UI open or closed will cause some relatively big FPS drops.

Small note for when testing for this, when you are already at the FPS cap for you monitor it's harder to notice since your PC could technically run at a higher frame rate.

Here are Videos of me showcasing the issue:

https://streamable.com/02vhhq