You could previously, I don't think that's new? The only thing we've changed recently with speech viewer is that it is now disabled in secure mode (2024.4) and it doesn't update when you move the mouse over it (2024.1). You could previously move focus TO that window and if the focus is on that window, if you press alt+f4.... it closes the window - as intended?

@NVAccess I could swear it never used to go away. I have a memory of being annoyed at having to go to the NVDA tools menu to dismiss it.
But who knows. If alt+f4 didn't work once perhaps I got so frustrated I just gave up once and never looked back!

It appears "always on top" so visually you will see it in front of other windows, even when it doesn't have focus - perhaps that was what you were thinking of? It usually appears as the LAST window in the alt+tab list so you might not have noticed it if you went through what you KNEW was open. You can alt+tab until you got to it or alt+shift+tab once to get to it.