Speaking based on personal experience and speculations. Sometimes a certain IP address(es) allocated to you by your ISP maybe flagged by one of the connection route of the server in firewall, the IP address's previous owner was either hacked or was involved in malicious activity. This can often times break connectivity. In other instances, it could be because of ISP imposed firewall on certain ports sometimes because someone else in your ISP's network is getting DDoS'ed or for whatever reason ISP is throttling. There are many reasons behind the problem you're facing, and it's fine.
How to tackle?
I use Cloudflare WARP to tackle this issue. Cloudflare acts like a VPN, however, it's main goal is to encrypt your datapackets and prevent your ISP from snooping into your traffic by making changes in your connection, and DNS. Sometimes it may also hide your IP, but most of the time, it's open. The main thing it does is, it changes your connection route to Cloudflare's global network of datacenters and take you to the destination server as fast as possible, which in this case, is GTA World's OVH route. Best part? WARP is free. No, I do not work for cloudflare. I'm just trying to help.
You can check https://18.104.22.168 and download their app. I use it, and whenever my ISP is acting funny, I just turn it on and it cuts any potential lag.
Also note, this isn't ideal for daily browsing. Sometimes it can trigger captcha for sites who have high security enabled because as always, its a VPN IP and it can flag you suspicious. If you are forced to solve captchas, solve it once and turn it off. Use it only while gaming.