I understand that impostors are more accurate to a user's avatar, but it comes with some caveats, one such is the lack of finger positions. While I don't have hearing or speech impairments, it does seem to affect those that do because it makes signing harder. Here's a comment that touches on it.

The main thing that concerns me in particular with bringing back fallbacks is face tracking. I use the PSVR2 with PSVR2Toolkit to enable eye tracking and it makes interacting with others feel much more natural. When I'm using a poor performance avatar, I'd rather people be able to see my eyes moving and have a more natural interaction with a different avatar than have a less natural interaction with a fake-looking assembly of billboards.

One cool idea I saw in the comment section of that reddit post is the ability to choose which avatars use their impostors and which ones use your fallback avatar.