need help with proxy in selenium python

bomma clot

Newbie
Joined
Sep 8, 2023
Messages
2
Reaction score
0
have a script for botting in twitter will use proxy along with adspower to mask webrtc
running the proxy though code below works fine, am able to test it with whatsismyip.com and httpbin.org/ip and comes back proxy ip but when i put the code into my script it wont work? using it for twitter via selenium for scraping
inside the script there are multiple accounts want all the accounts to go though the proxy


code i use to test it alone on a new file works fine, like this
from seleniumwire import webdriver
from selenium.webdriver.chrome.service import Service as ChromeService
from webdriver_manager.chrome import ChromeDriverManager
from selenium.webdriver.common.by import By

proxy_username = 'fgrlkbxt'
proxy_password = 'cs01nzezlfen'
seleniumwire_options = {
'proxy': {
'http': f'http://{proxy_username}:{proxy_password}@185.199.229.156:7492',
'verify_ssl': False,
},
}

driver = webdriver.Chrome(
seleniumwire_options=seleniumwire_options
)
driver.get('http://httpbin.org/ip')
print(driver.find_element(By.TAG_NAME, 'body').text)

code i use on my script
from seleniumwire import webdriver
from selenium.webdriver.chrome.service import Service as ChromeService
from webdriver_manager.chrome import ChromeDriverManager
from selenium.webdriver.common.by import By

proxy_username = 'fgrlkbxt'
proxy_password = 'cs01nzezlfen'
seleniumwire_options = {
'proxy': {
'http': f'http://{proxy_username}:{proxy_password}@185.199.229.156:7492',
'verify_ssl': False,
},
}

is my code wrong or something?

i checked with logging into my twitter account and not seeing any "New login alert" from twitter with the location of the proxy. used another proxy that was working
 
if the proxy setup is working in a standalone environment but not within your actual Twitter scraping script, there might be other factors at play:

  1. Session & Cookies: Twitter and many other websites use cookies and session information to track user behavior. If your bot has previously accessed Twitter without a proxy or with a different proxy, then using the same session or cookies might result in using the old IP.
  2. User-Agent & Headers: If you're using a different User-Agent or headers in your main script compared to the standalone script, Twitter might detect this inconsistency. Ensure that every aspect of the browser session, including headers, mimics a legitimate user.
  3. WebRTC Leak: Even though you mentioned using adspower to mask WebRTC, double-check to ensure there's no WebRTC leak. WebRTC can expose your real IP address even if you're using a proxy. You can check for leaks by visiting websites like browserleaks.com/webrtc.
  4. Twitter's Sophisticated Detection: Twitter has advanced bot detection mechanisms. It's not just the IP they check. They look at the behavior, mouse movements, typing speeds, and more. If you're not mimicking human-like behavior (e.g., instantly moving to buttons or links), that could raise red flags.
  5. Full Initialization: In your main script, ensure that you're fully initializing the browser with the seleniumwire_options. Here's a more explicit initialization:
pythonCopy code
driver = webdriver.Chrome(
executable_path=ChromeDriverManager().install(),
seleniumwire_options=seleniumwire_options
)

  1. Multiple Accounts: If you're using multiple accounts with the same proxy, there's a risk that Twitter might recognize this behavior as suspicious. Each account logging in from the same IP in quick succession can appear as unusual behavior.
  2. Existing WebDriver Session: Ensure that you're not reusing a WebDriver session that was initialized without the proxy settings. Always create a new session with the desired proxy configuration.
  3. Debugging: To debug the issue further, you can try intercepting the request made by Selenium to see if the proxy headers are being sent correctly:
pythonCopy code
def interceptor(request):
print(request.headers)

driver.request_interceptor = interceptor

Lastly, consider implementing time.sleep() with randomized intervals between actions to simulate human-like behavior, if you haven't already.

If you've checked all of the above and are still facing issues, it might be helpful to see a more comprehensive view of your main script. There could b
 
Back
Top