1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

gscraper question

Discussion in 'BlackHat Lounge' started by proxygo, Jun 21, 2015.

  1. proxygo

    proxygo Jr. VIP Jr. VIP

    Joined:
    Nov 2, 2008
    Messages:
    15,767
    Likes Received:
    9,596
    Occupation:
    PROVIDING PROXIES FOR GSA SCRAPING.
    Location:
    BHW
    Home Page:
    quick question to gscraper owners anyone else having 0 urls today
     
  2. whiteblackseo

    whiteblackseo Jr. VIP Jr. VIP

    Joined:
    Apr 11, 2015
    Messages:
    2,461
    Likes Received:
    918
    Home Page:
    Did a quick scrape, nope. Must be just you.
    Capture.PNG
     
  3. dekadent30

    dekadent30 Power Member

    Joined:
    Nov 10, 2008
    Messages:
    575
    Likes Received:
    152
    I have 0 urls scraped for 2 days, i think developer should update gscraper
     
  4. proxygo

    proxygo Jr. VIP Jr. VIP

    Joined:
    Nov 2, 2008
    Messages:
    15,767
    Likes Received:
    9,596
    Occupation:
    PROVIDING PROXIES FOR GSA SCRAPING.
    Location:
    BHW
    Home Page:
    odd i got friends with multiple proxy types public/private and nothing
     
  5. TrevorB

    TrevorB Senior Member

    Joined:
    Dec 21, 2011
    Messages:
    1,185
    Likes Received:
    361
    Location:
    Canada
    mine is not scraping either. Tried with public and private proxies.
     
    • Thanks Thanks x 1
  6. satyr85

    satyr85 Jr. VIP Jr. VIP

    Joined:
    Aug 7, 2011
    Messages:
    599
    Likes Received:
    460
    Location:
    Poland
    Its problem with google guys. Same thing happened last year around november as far as i remember.

    Problems depends on location - for some guys in middle europe Gscraper works, for some in USA not.
    Version 1.2.6.2 is working but LPM is low.
     
  7. proxygo

    proxygo Jr. VIP Jr. VIP

    Joined:
    Nov 2, 2008
    Messages:
    15,767
    Likes Received:
    9,596
    Occupation:
    PROVIDING PROXIES FOR GSA SCRAPING.
    Location:
    BHW
    Home Page:
  8. satyr85

    satyr85 Jr. VIP Jr. VIP

    Joined:
    Aug 7, 2011
    Messages:
    599
    Likes Received:
    460
    Location:
    Poland
    Yes gscraper was working fine till friday but same thing happened in november (and everything was fine after few days without changing Gscraper version).
    Its not problem with Gscraper, its problem with google. I have clients who dont have problems on version 1.2.8.5, others (including me) have problems with this version. Only difference is server location.
     
    • Thanks Thanks x 1
  9. proxygo

    proxygo Jr. VIP Jr. VIP

    Joined:
    Nov 2, 2008
    Messages:
    15,767
    Likes Received:
    9,596
    Occupation:
    PROVIDING PROXIES FOR GSA SCRAPING.
    Location:
    BHW
    Home Page:
    message from gscraper support
    Hello, Google have update the page information, and our technology updating the soft now.
     
    • Thanks Thanks x 2
  10. TrevorB

    TrevorB Senior Member

    Joined:
    Dec 21, 2011
    Messages:
    1,185
    Likes Received:
    361
    Location:
    Canada
    Still not working as of today. Would be nice if they posted an update about how long it will be for them to fix it.
     
    • Thanks Thanks x 1
  11. s4nt0s

    s4nt0s Jr. VIP Jr. VIP Premium Member

    Joined:
    Jul 10, 2009
    Messages:
    3,863
    Likes Received:
    1,970
    Location:
    Texas
    Ah, glad to hear it's not just me. I was trying to make a video and couldn't get it to work ... Grr.
     
  12. JustUs

    JustUs Power Member

    Joined:
    May 6, 2012
    Messages:
    626
    Likes Received:
    582
    It should not be too hard to fix this section of code:

    Code:
            public static string smethod_0(string string_0, int int_0, IWebProxy iwebProxy_0)
            {
                string result = "--";
                string text = Class14.smethod_5(string_0);
                string_0 = HttpUtility.UrlEncode(string_0);
                string text2 = "http://toolbarqueries.google.com.hk";
                string string_ = string.Concat(new string[]
           {
               text2, 
               "/tbr?client=navclient-auto&ie=UTF-8&ch=", 
               text, 
               "&features=Rank&q=info:", 
               string_0
           });
    
    
    The question is does the will exist by the author? Does patience run out first?

    Hawke, people paid you good money for the program, and pay you good money for your proxy service. Fix the program or remove the protection and release the source.
     
    Last edited: Jun 24, 2015
  13. *Hawke*

    *Hawke* Junior Member Premium Member

    Joined:
    Apr 9, 2013
    Messages:
    126
    Likes Received:
    81
    Home Page:
    Please update the newest version (v1.2.9.1), its fixed already.
    A few days ago,cause of Google page update, GScraper can't scrape results.
    We are sorry for this and Now we decide all users additional services expire time extend 7 days free.
     
  14. JustUs

    JustUs Power Member

    Joined:
    May 6, 2012
    Messages:
    626
    Likes Received:
    582
    Bullshit! I just updated to 1.2.9.1 entered %KW% as the footprint, entered house and mouse as keywords. Zero URL's were scraped with just tested proxies through GScraper.
     
  15. JustUs

    JustUs Power Member

    Joined:
    May 6, 2012
    Messages:
    626
    Likes Received:
    582
    Now it seems to be working. You are putting the keyword into the remark field.
     
  16. proxygo

    proxygo Jr. VIP Jr. VIP

    Joined:
    Nov 2, 2008
    Messages:
    15,767
    Likes Received:
    9,596
    Occupation:
    PROVIDING PROXIES FOR GSA SCRAPING.
    Location:
    BHW
    Home Page:
    running a scrape now, and hawk ty for email replys 2 days ago
    and justus, ill say this, stop being a dick.
    i emailed hawk as soon as the problem started 2 days ago, he replied
    in 24hrs, which is the same time frame i give my own subs for a reply 24hrs
    2 days its fixed. hes done his job hes fixed it. so be a good chap and stop
    being a dick.

    update took a list of gscraper proxies that went out to my subs today
    proxy tester working fine - scraping working fine

    [​IMG]
     
    Last edited: Jun 24, 2015
  17. JustUs

    JustUs Power Member

    Joined:
    May 6, 2012
    Messages:
    626
    Likes Received:
    582
    Proxygo, if I wanted to be a dick I would release his source code. I have not. I really do not expect a lot, I just expect the program to operate. For scraping the first ten pages I have my own scraper and it uses pretty much the same methods as GScraper, which is pretty much boiler plate. My own code is this:
    Code:
                        if (count != 0)
                        {
                            IP = this.listViewProxy.Items[index].SubItems[1].Text;
                            Port = Convert.ToInt32(this.listViewProxy.Items[index].SubItems[2].Text);
                            HttpWebRequest webRequest = (HttpWebRequest)WebRequest.Create(requestUriString);
                            WebProxy proxy = new WebProxy(IP, Port);
                            webRequest.Timeout = this.timeout;
                            webRequest.Proxy = proxy;
                            webRequest.UserAgent = usrAgent;
                            WebResponse response = webRequest.GetResponse();
                            Stream responseStream = response.GetResponseStream();
                            StreamReader streamReader = new StreamReader(responseStream);
                            text = streamReader.ReadToEnd();
                            str = "http://" + response.ResponseUri.Host.ToString();
                            str2 = "http://" + response.ResponseUri.Host.ToString() + "/";
                        }
    
    I had told Hawke that I would not release his source code and my honor means a lot to me.

    Now my own scraper did not fail. Based on this and the fact that I can change to any Google server in the world, I can make an assumption that the failure was not Google changing anything.
    [​IMG]
    [​IMG]

    Knowing the code, I pretty much know where the problem was. Think as you may. What probably happened when I made the second post about it not working is that I beat Hawke making the changes to his server that decrypts the information.
     
    Last edited: Jun 24, 2015
  18. proxygo

    proxygo Jr. VIP Jr. VIP

    Joined:
    Nov 2, 2008
    Messages:
    15,767
    Likes Received:
    9,596
    Occupation:
    PROVIDING PROXIES FOR GSA SCRAPING.
    Location:
    BHW
    Home Page:
    strange, your acting like a person who has none.
    now put your toys back in your pram and go play
     
    Last edited: Jun 24, 2015
  19. JustUs

    JustUs Power Member

    Joined:
    May 6, 2012
    Messages:
    626
    Likes Received:
    582
    Have to love you English folk. You use language that no American generally would. Shows a difference in culture when I have to look something as basic to you as the word Pram to understand that you meant a baby carriage. Now if a closely related language is different enough to send one to the Oxford and Cambridge Dictionary to find the meaning, that the cultures might also be as different?

    So nice to see that you understand that.
     
  20. proxygo

    proxygo Jr. VIP Jr. VIP

    Joined:
    Nov 2, 2008
    Messages:
    15,767
    Likes Received:
    9,596
    Occupation:
    PROVIDING PROXIES FOR GSA SCRAPING.
    Location:
    BHW
    Home Page:
    odd, as a brit im married to a yank from big apple she gets it.