There is no definitive fix for this issue, it might sometimes get amplified if you use mods but I assume it's either because Rage engine itself sucks ass, or RageMP, or maybe GTAW's mods are just too much. But then again, not so sure. Best solution is to moderate your mod count and pray it doesn't happen.
One potentially high possibility could be the big amount of mod clothes on World, V's Rage engine has a limited number of .ymt slots per gender, so, every time the game gets updated, the slots get less, and this might cause issues when using the same number of mod clothes. Best solution is for the server to replace vanilla clothes instead of adding. But then again, this is just a possibility.
EDIT: The number of available .ymts went down from 8 to just 2 ever since the Summer update, so that's maybe why it kept getting worse by each update.