1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

Gscraper Taking Years To Scrape Targets!!!

Discussion in 'Black Hat SEO Tools' started by keith88, Mar 1, 2014.

Tags:
  1. keith88

    keith88 Regular Member

    Joined:
    Sep 14, 2010
    Messages:
    287
    Likes Received:
    23
    Occupation:
    Internet Marketer
    Location:
    Home
    Hey guys.....

    I'm doing some scraping for GSA and its taking AGES to scrape targets...

    I'm using footprints from Footprint factory so I have about 30k footprints. I'm running at max 1500 threads with 30 sec timeout...

    I'm removing duplicates at scraping so I think that may be a reason its taking some time.


    Here are my server details...

    PowerUp
    2 CPU
    60 GB
    2048 MB
    Unmetered Bandwith

    Any thoughts on why its taking so long to complete scrape?
     
    Last edited: Mar 1, 2014
  2. WinningIO

    WinningIO Regular Member Premium Member

    Joined:
    Dec 16, 2013
    Messages:
    230
    Likes Received:
    160
    Location:
    #1 on Google
    You don't have enough proxies.
     
  3. keith88

    keith88 Regular Member

    Joined:
    Sep 14, 2010
    Messages:
    287
    Likes Received:
    23
    Occupation:
    Internet Marketer
    Location:
    Home
    Im using gscraper's proxy service... the unlimited ones...
     
  4. divok

    divok Senior Member

    Joined:
    Jul 21, 2010
    Messages:
    1,015
    Likes Received:
    634
    Location:
    http://twitter.com/divok
    they were good , now they are worse than public proxies
     
  5. keith88

    keith88 Regular Member

    Joined:
    Sep 14, 2010
    Messages:
    287
    Likes Received:
    23
    Occupation:
    Internet Marketer
    Location:
    Home
    So go private? How many?
     
  6. JustUs

    JustUs Power Member

    Joined:
    May 6, 2012
    Messages:
    609
    Likes Received:
    452
    Two days ago I used Gscraper and 1000 key words with 500 + public proxies at 20 threads to scrape 900,000 sites in about three years hours.

    Here is the problem with GScraper, and it is a warning as well, GScraper sends every link and every proxy that you use to China. This means that when you are scraping, or using your own proxies, you are not just scraping for yourself, or keeping the proxies that you use to yourself. Ditto when you post.

    If Hawke wants to contest this, I will be more than happy to post the reverse compiled source on line.

    Because you are sending all your scraped links and proxies to China, you have a delayed response that slows down the scraping.
     
    Last edited: Mar 1, 2014
  7. bk071

    bk071 Jr. Executive VIP Jr. VIP Premium Member

    Joined:
    Nov 24, 2010
    Messages:
    3,105
    Likes Received:
    7,917
    Occupation:
    I don't have a job
    Location:
    .............
    That is a serious accusation. I wonder what the developers have to say about it.
     
  8. mudbutt

    mudbutt Jr. Executive VIP Jr. VIP Premium Member

    Joined:
    Jun 23, 2010
    Messages:
    1,817
    Likes Received:
    4,284
    Location:
    ghosted
    This is deserving of its own thread. I suggest you start one as this is a very serious accusation and if true, should be out in the open..
     
    • Thanks Thanks x 2
  9. divok

    divok Senior Member

    Joined:
    Jul 21, 2010
    Messages:
    1,015
    Likes Received:
    634
    Location:
    http://twitter.com/divok
    if you are serious and not trolling , you MUST shitlist them .
     
  10. JustUs

    JustUs Power Member

    Joined:
    May 6, 2012
    Messages:
    609
    Likes Received:
    452
    Draw you own conclusion as to whether I am bullshiting or trolling.

    [​IMG]

    And if you want to see it in the Visual Basic that it is written in, I can do that too. I converted to C# because I am a C family type of guy.
     
    Last edited: Mar 1, 2014
  11. keith88

    keith88 Regular Member

    Joined:
    Sep 14, 2010
    Messages:
    287
    Likes Received:
    23
    Occupation:
    Internet Marketer
    Location:
    Home
    What other options do I have?
     
  12. mudbutt

    mudbutt Jr. Executive VIP Jr. VIP Premium Member

    Joined:
    Jun 23, 2010
    Messages:
    1,817
    Likes Received:
    4,284
    Location:
    ghosted
    What does that all mean? Can you explain a little more on what it is theyre doing and what the implications are? Not everyone here is a programmer.
     
    • Thanks Thanks x 1
  13. divok

    divok Senior Member

    Joined:
    Jul 21, 2010
    Messages:
    1,015
    Likes Received:
    634
    Location:
    http://twitter.com/divok
    with my limited knowledge and if this is actually a method in gscraper . Then it is encrypting some data and returning it to some other variable or function .
    http://msdn.microsoft.com/en-us/library/system.security.cryptography.cryptostream(v=vs.110).aspx

    I may be completely wrong . Still it doesn't prove anything .
     
  14. Asif WILSON Khan

    Asif WILSON Khan Executive VIP Premium Member

    Joined:
    Nov 10, 2012
    Messages:
    10,119
    Likes Received:
    28,559
    Gender:
    Male
    Occupation:
    Fun Lovin' Criminal
    Location:
    London
    Home Page:
    I am not a coder but if you are talking about DESCryptoServiceProvider then I suggest you look it up.

    If not could you please explain how you drew your conclusion. I am interested.
     
  15. hatemachine

    hatemachine Regular Member

    Joined:
    Jan 14, 2011
    Messages:
    271
    Likes Received:
    1,138
    Well. The function encrypts string_0 and string_1. That doesn't say much without tracing the code back and checking what the result returned from smethod_3 is being used for. I don't see it being sent anywhere in that function, from where is smethod_3 being called? If someone has a valid license I could take a look and intercept the data. It would take like 10 minutes to verify the claims anyway.
     
  16. Asif WILSON Khan

    Asif WILSON Khan Executive VIP Premium Member

    Joined:
    Nov 10, 2012
    Messages:
    10,119
    Likes Received:
    28,559
    Gender:
    Male
    Occupation:
    Fun Lovin' Criminal
    Location:
    London
    Home Page:
    It would be interesting.

    Maybe somebody could run it through a network sniffer/packet analyzer and see if it is sending anything that way.
     
  17. Tensegrity

    Tensegrity Jr. VIP Jr. VIP Premium Member

    Joined:
    Apr 22, 2009
    Messages:
    1,824
    Likes Received:
    969
    I think it's insane that people use someone else's tool to scrape google when you can easily build a scraper yourself with a spare afternoon and some persistence.

    To help you guys out, here:

    - look up php curl
    - open firefox and turn off javascript and cookies
    - visit google and do a search
    - grab the url
    - write your php curl script to use private proxies and visit that url
    - use simple html dom to grab the links

    voila, you now have a scraper.
     
  18. Tosmekop

    Tosmekop Supreme Member

    Joined:
    Oct 24, 2011
    Messages:
    1,208
    Likes Received:
    813
    Interesting theory of yours, about GScraper. That code doesn't confirm it.
     
  19. JustUs

    JustUs Power Member

    Joined:
    May 6, 2012
    Messages:
    609
    Likes Received:
    452
    You would be mistaken in some of your assumptions. That particular function is used in a couple of different ways depending on where the function is called from. Called from one place the method encrypts the URL, called from another it encrypts the proxy, called from a third it seeks permission to run. There are two separate keys that are used, and I could show them as well. Also, the program is tied to the computer through a few different hardware aspects.

    BUT, and a big one, is that I did not put that piece of code up to debate what it means or how it is used. I put it up to demonstrate that I have, in fact, reverse compiled GScraper and just might have a good idea of what I am speaking about. You can cross test what I am speaking about with Wireshark and see the amount of traffic going to China. By putting that small piece of code up that does not reveal anything critical about the program, I have allowed others to some means to determine if I am speaking about what I know.

    The code is not for sale or view unless one can demonstrate a need. A want is not good enough, but it is enough to put Hawke on warning. I have privately taken this to his attention before.

    So please everyone, save the nit picking and fire up Wireshark to see where data is going when it is encrypted (208.98.62.34) and then ask yourself why the data would be encrypted?
     
    • Thanks Thanks x 1
    Last edited: Mar 1, 2014
  20. stugz

    stugz Junior Member

    Joined:
    Apr 14, 2013
    Messages:
    154
    Likes Received:
    33
    If I had built such a tool for sale I would be definitely do what is being accused in this thread. It is trivial to do and gets others to do work for you.