Quick shout out to this blog post on where to set the Internet Proxy for ADFS. Basically, you gotta set it via netsh winhttp proxy.
[Aside] ADFS and Windows Proxy by rakhesh is licensed under a Creative Commons Attribution 4.0 International License.