I want to scrape an ajax website, but detects my bot (Ubot professional stealth edition).

I'm creating a mailing bot with ubot, but i have a problem when gathering my friends, because there is a button that has to be clicked, so there appear more friends. the probem is that ajax detects that i'm a robot, and the the button that loads more friends, begins to stay frozen for longer periods of time each time is clicked. An then, when the bot tries to click the button, it shows and error about jar and i can only scrape 415 users, and this when making it wat 20 seconds for the button, if i make the button wait 60 secons it gathers 715 users before crashing. when i open the page in a browser and i click by myself it lods after 5 seconds all the times.

I already use the javascript:

run javascript("window.scrollTo(0,document.body.scrollHeight);")

but it it just scrolls the page to the button.

I found atutorial in google that uses this phython code to scrape pages:

run python("import mechanize
import urllib2
import time
from selenium import webdriver
from selenium common exceptions import NoSuchAttributeException
from selenium.webdriver.common.keys import Keys
import os
import re
br.set_handle_robots(false)
br.set_handle_refresh(false)
br.addheaders=[(\'user-agent\'),(\'chrome\')]")


But when i try to run the scripts, it shows me errors all the time.

• Does somebody know an alternative way to prevent to load more button from doing this? (I does it provably because it detects a bot)
• How can i do to insert python code in ubot so it does not have errors?

mechanize from python is a headless webbrowser so it doesn't have js engine
You can use selenium or you can use javascript to click on that button with click() method

Hey there,

Have you asked on our forum? There may be a better response! Also, are you in our skype group ?

If you still need help with this, pm me I can help. I'm pretty experienced with selenium using python.

Your only option is to scrape with Selenium as you can choose your browser or get Exbrowser plugin. Ubot browser is too easily detectable. Though Selenium is quite easy to learn. I am still learning it and can't yet scrape pages but can do a lot of other stuff I don't need