Recent content by Crawlie

For Java check out HtmlUnit. It's lightweight and fast, parses JavaScript generated content. JSoup only parses static html. You can also use Selenium, but that eats more resources. A basic scraping script for a site is only a few lines of code.

Definitely Java should! Being the most popular language and all(save js).

If there is no other obstacle like captcha or worrying about proxy, then login bot is 5 minutes. Posting is another 5. Definitely worth it IMO.

I used to have a Firefox extension where it opens a new tab on double click, sadly Chrome does not have it, maybe you could do that.

Of course you need proxies, otherwise Google blacklists your ip pretty fast. Rotate them so each request is a new proxy. Also you can use timer to try and randomize your bot. Set max results in Google query, which I think is 100, then you can loop through pages. After some time Google gives...

I doubt you really want to use their API. That would not be scraping. Any language is good, python is the most popular for scraping. I personally like Java. Desktop or web app depends, which one are you more familiar with?

Oh and of course if you add wrong/invalid data to the input fields it won't log in...

Tested, works: import org.openqa.selenium.By; import org.openqa.selenium.chrome.ChromeDriver; import java.util.concurrent.TimeUnit; public class Main { public static void main(String[] args) { System.setProperty("webdriver.chrome.driver","your path to chromdriver")...

I used Firefox driver and PhanomJS, what is your concern?

You need to add Selenium or HtmlUnit and use these headless browsers (or Chrome/Firefox drivers). Maybe later in the day I can write something more detailed if you want.

Also, you might wanna look into HtmlUnit, it's more lightweight and faster than Selenium.

I found tutorialspoint to be very vague, I usually try to avoid it when looking for resources.

It's perfectly ok if you use proxies.

Scrapebox is Delphi, Xrumer is c++, but it doesn't matter. Just learn one language well.

Haven't done it before but planning to. If you have no coding experience then your only option is obviously to hire someone. If you have some time on your hands you might as well get into programming... It's generally good to have some knowledge in it.