Skip to content

user_data_dir & proxy #2368

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
Allmight3 opened this issue Dec 16, 2023 · 6 comments
Closed

user_data_dir & proxy #2368

Allmight3 opened this issue Dec 16, 2023 · 6 comments
Labels
question Someone is looking for answers UC Mode / CDP Mode Undetected Chromedriver Mode / CDP Mode

Comments

@Allmight3
Copy link

Allmight3 commented Dec 16, 2023

Hello.

This code works well for me. It creates a local directory profile, uses the proxy and adds my extensions.

driver = Driver(uc=True, headed=True, proxy=PROXY, user_data_dir=absolute_profile_directory, page_load_strategy="eager", extension_dir=extensions_dir)

My issue is when I want to come back to this profile later, it requires authentication from the proxy I used when I created it. The problem is, I rotate proxies and the authentication changes.

I tried regular Selenium:

options = webdriver.ChromeOptions()
options.add_argument(f'--user-data-dir={folder_path}')

As well as SB:
driver = Driver(uc=True, headed=True, user_data_dir=folder_path)

I just want to load that profile back up without a proxy requirement. I also snooped around in the profile folder looking to see where the proxy information was being stored, so I could delete it programmatically before launching. However I couldn't find that information. It wasn't in Preferences or anywhere else that I had noticed.

Am I trying to do something that's unsupported? If so, is there a work-around -- such as deleting where the proxy data is stored before relaunching the profile?

Thank you

@mdmintz mdmintz added the question Someone is looking for answers label Dec 16, 2023
@mdmintz
Copy link
Member

mdmintz commented Dec 16, 2023

To disable proxy settings for regular Selenium, use https://stackoverflow.com/a/49033706/7058266:

chrome_options.add_argument("--no-proxy-server")

With SeleniumBase, those options get passed via chromium_arg: (comma-separated for multiple)

chromium_arg="--no-proxy-server"

With the Driver(), that would be this:

driver = Driver(chromium_arg="--no-proxy-server")

@mdmintz mdmintz closed this as completed Dec 16, 2023
@mdmintz mdmintz added the UC Mode / CDP Mode Undetected Chromedriver Mode / CDP Mode label Dec 16, 2023
@epictroller64
Copy link

Hi, I am still having same issue with driver = Driver(chromium_arg="--no-proxy-server", browser="chrome", user_data_dir=session_dir, headless=False, undetected=True), keeps asking to reauthenticate. I even deleted the proxy file from downloaded_files directory.

@mdmintz
Copy link
Member

mdmintz commented Dec 29, 2023

@epictroller64 You probably mixed your UC Mode user_data_dir with a non UC Mode Chrome. (You can't "cross the streams".) Try from a brand new user_data_dir, or don't specify one at all when you run your script.

@epictroller64
Copy link

Seems the only solution is to delete user_data_dir but I really need to carry the session data over after changing proxy. Creating new profile worked only once, after I removed the proxy and used chromium_arg="--no-proxy-server", it still showed old details.

@mdmintz
Copy link
Member

mdmintz commented Dec 29, 2023

@epictroller64 You probably need to disable extensions:
chromium_arg="--no-proxy-server,--disable-extensions"

(Proxy details are configured via Chrome extension in SeleniumBase.)

Also, make sure you're not mixing a non UC Mode Chrome user_data_dir with a UC Mode Chrome user_data_dir.
If you "cross the streams", you'll need to start with a fresh user_data_dir for things to work.

@Allmight3
Copy link
Author

Seems the only solution is to delete user_data_dir but I really need to carry the session data over after changing proxy. Creating new profile worked only once, after I removed the proxy and used chromium_arg="--no-proxy-server", it still showed old details.

I found that creating the profile using UC mode and a proxy with username:pass authentication will cause the profile to always require that same proxy.

If you create a profile using UC mode and a proxy without username:pass, then it will work normally as expected. So that's what I did. If you only have proxies with username authentication then you could use proxy.py perhaps to forward the proxy, then use the downstream proxy to create the profiles.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
question Someone is looking for answers UC Mode / CDP Mode Undetected Chromedriver Mode / CDP Mode
Projects
None yet
Development

No branches or pull requests

3 participants