I figured it out. Tools, Internet Options. Connections Tab. Select the PPPOE connection and click settings. Set the proxy settings for the PPPOE connection there.