torasu
Newbie
- May 31, 2016
- 3
- 2
Hi all, I am in the midst of learning Python and Selenium, and have created a simple Python script that scrapes HTTPS proxies from http://spys.one/, thought I'd share it with you all here:
Requirements: Python, Selenium Chrome Browser (can be downloaded from here: http://chromedriver.chromium.org/)
Do feel free to use it and enhance / tweak in any form you like! Enjoy scraping! =D
Requirements: Python, Selenium Chrome Browser (can be downloaded from here: http://chromedriver.chromium.org/)
Code:
#!/usr/bin/env python
# -*- coding: utf-8 -*-
from selenium import webdriver
from selenium.webdriver.chrome.options import Options
def browse_proxy_list():
# selenium chrome browser
chrome_options = Options()
chrome_options.add_argument("--headless")
browser = webdriver.Chrome(chrome_options = chrome_options)
# browse spys.one proxy site
browser.get("http://spys.one/en/https-ssl-proxy/")
browser.execute_script("document.getElementById('xpp').selectedIndex = 5; document.getElementById('xpp').onchange();")
# retrieve HTML elements and remove first element (table header)
proxy_list = browser.find_elements_by_xpath("//tr[@class='spy1xx' or @class='spy1x']")
proxy_list.pop(0)
for i in range(len(proxy_list)):
proxy_server = browser.find_element_by_xpath("//tr[@class='spy1xx' or @class='spy1x'][" + str(i+2) + "]//td[1]//font[@class='spy14']").text
print(proxy_server)
def main():
browse_proxy_list()
if __name__ == "__main__":
main()Do feel free to use it and enhance / tweak in any form you like! Enjoy scraping! =D
