I've read in the forums that Scrapebox operates with its own internal Java based browser. I'm not a programmer so forgive me if this is a silly question but, why doesn't it use some kind of http mimic instead of making its own browser? Of course I know it would be stupid to use IE though it has its uses in slow poster, but what's wrong with using some kind of http mimic when querying?
Scrapebox - Internal Java Browser
- Thread starter jb2008
- Start date
pyronaut
Senior Member
- Dec 9, 2008
- 1,189
- 1,345
I am really not sure if it is a true custom browser, or just an IE window in a new skin. But, from my own experiences with posting wordpress comments.
There is a plugin called "Cookies For Comments", that gives you a cookie based on running a piece of javascript. HTTP requests or Sockets cannot execute javascript. Thus they will never gain the cookie, and can never post a comment.
Using a browser based solution will execute the javascript and post the comment. Ofcourse it is slower, but it is a more thorough process.
There is a plugin called "Cookies For Comments", that gives you a cookie based on running a piece of javascript. HTTP requests or Sockets cannot execute javascript. Thus they will never gain the cookie, and can never post a comment.
Using a browser based solution will execute the javascript and post the comment. Ofcourse it is slower, but it is a more thorough process.
That's interesting pyro - it sounds plausible that a browser would be required to post comments if cookies are required.
I am more concerned with the harvesting part of SB, however; why is a javascript browser required for queries to Google? Of course using the API is not recommended for many reasons, but a http mimic of some sort sounds like it could work? Or does G have some sort of defenses against this. I am fascinated by scrapebox and having seen many URL harvesters on the internet and how much they struggle at scraping, the seamless ease with which Scrapebox harvests is quite amazing. Given the difficulty all these other pieces of software have, I am left wondering how on earth SB does it!
I am more concerned with the harvesting part of SB, however; why is a javascript browser required for queries to Google? Of course using the API is not recommended for many reasons, but a http mimic of some sort sounds like it could work? Or does G have some sort of defenses against this. I am fascinated by scrapebox and having seen many URL harvesters on the internet and how much they struggle at scraping, the seamless ease with which Scrapebox harvests is quite amazing. Given the difficulty all these other pieces of software have, I am left wondering how on earth SB does it!
kalrudra
BANNED
- Oct 29, 2010
- 271
- 221
I am C# programmer from 8 years...
You doesn't require webbrowser to send comments !!
You can do everything with http request response classes,
But It's little complicated with javascript execution.. (you have to calculate manually what it does...)
I don't think comment requires javascripts... If there is any javascript.. it's just for validation of input purpose, nothing more..
my 2 cents..
You doesn't require webbrowser to send comments !!
You can do everything with http request response classes,
But It's little complicated with javascript execution.. (you have to calculate manually what it does...)
I don't think comment requires javascripts... If there is any javascript.. it's just for validation of input purpose, nothing more..
my 2 cents..
Last edited:
cooooookies
Senior Member
- Oct 6, 2008
- 1,133
- 265
Yep. I use HtmlUnit - java library for a headless browser. Maybe this is also used by scrapebox. Blog-commenting does not require js-execution. Therefore I disable it
Even faster then relying on a headless browser is working directly with http requests - to be more specific to blindly to POST requests. But in some cases you want to parse the DOM-tree of html-tags to name the most important. I'd say there is a compromise between traffic and cpu usage by doing so.
Even faster then relying on a headless browser is working directly with http requests - to be more specific to blindly to POST requests. But in some cases you want to parse the DOM-tree of html-tags to name the most important. I'd say there is a compromise between traffic and cpu usage by doing so.