This is definitely not due to latency alone. A slow internet or even a misconfiguration anywhere in the uplink from your router to your local ISP could drop packets and increase latency.
Most likely cause is that your device physical locations are being broadcast or discovered somehow. Or your proxy is leaking information. With a different location proxy, there's a mismatch somewhere resulting in your getting flagged.
You mention a VPN in the proxy's country solves the problem. But in most cases, adding a VPN is just adding a different node between your system and the proxy.
One more tunnel. One more network hop. Possible encryption overhead. And another NAT layer.
The only way this would help is when the actual route between your ISP and your proxy is fucked and the VPN has a substantially better provider and peering route.
So if your system is in Germany and the proxy is in US you get 250ms
And then you add a VPN in the US with a better route which gives you 150ms between it and your system and there's 10ms between it in and your proxy.
This would offer a slight improvement, not enough to not get flagged.
Audit your system and potential leaks including locale (time, region, language etc.)