Ever since I went ahead to update my world with the latest update in mind (SDK 3.9.1-beta.1], anyone trying to join it will receive an "Unusual Client Behavior" and be kicked out of the world immediately.
I tried to delete any changes done in the world in the hope to pinpoint the problem but the problem still occurred and even after downgrading to SDK 3.9.0 it still gives the error.
If it helps to solve the issue, I noticed that messing with my PlayerObject pool (despite the fact I have not touched it once for the update) and removing then adding back some object prevented the error from showing up.
Alas when I decided to arrange it after I noticed some script were broken with the newer PlayerObject setup in the world that fixed the error, the error showed up again, even when I used the backup scene I saved that worked before, that one ALSO DIDN'T WORK.
This is the world currently being affected by this issue : https://vrchat.com/home/world/wrld_37abea33-f34c-4706-8e6b-2f69e414fc5d/info
I have tried everything to see if the problem came from my world but the problem feels way too random for it to be the case.