Better use short lived tokens with auto refresh. For example OAuth tokens. They can be used for the same use cases like PAT, but they expire after a max of 2 hours.
It would be nice if GitHub or GitLab could limit a token to a given IP, but with how proxies and dynamic ips work I would imagine it would be a hassle to get it working correctly for everyone, but it’s better than nothing.