The number of animation layers heavily affects performance due to unnecessary inverse kinematic calculations. Even blank animation layers with zero weight in the FX layer can trigger this behavior.
The offender is seen in the profiler as Animators.IKAndTwistBoneJob. It runs as many times as the currently loaded animation controller with the largest number of layers. Although the individual run time is small, they very quickly add up. Slower CPUs seem to be more adversely affected by this. The time taken by these extraneous calculations quickly grows larger than all normal animator activity.
This behavior only happens on animators that have a unity avatar/armature. When that is removed, this behavior goes away and the number of layers ceases to be a significant cause of lag. Most FX layers do not have any interaction with the armature, so it is likely that this behavior can be fixed for almost all FX layers. However from debugging the unity editor in visual studio, it appears the this happens in a native unity function named Animator::UpdateAvatars.
It is likely that in most if not all cases, these calculations are completely wasted on FX layers and provide no benefit at all while consuming significant amounts of main thread time. Somehow these calculations need to be only run for layers that require them.
Additionally providing a way to get parameters in sub animators would allow logic to be moved out of the main animator completely sidestepping this problem. They would not have a unity avatar associated with them, so this problem would not occur. It would also allow completely disabling animators when not in use.
Please attempt to find a way to prevent this behavior or consider convincing your Unity contacts to implement a fix. This seems to be one of the largest contributors to animator lag.
I have a full write-up with more profiler images and further explanations.