1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

Hrefer stops working after a cpl minutes...proxies?

Discussion in 'Black Hat SEO Tools' started by Ranko Jones, Aug 23, 2011.

  1. Ranko Jones

    Ranko Jones BANNED BANNED

    Joined:
    Mar 3, 2011
    Messages:
    1,677
    Likes Received:
    146
    I go to scrape and it is taking in a nice amount of urls then it drops to 0 characteristic of your classic G ban.

    I am only using the default proxies...I was told these are good enough (tho not sure whether to believe guy as he's a known bullshitter).

    Thing is I shut down hrefer and load it up again and begin scraping again and again it works for a cpl mins then stops. If it was a G ban why would it suddenly work again?

    If it's proxies how do I get decent (free) ones? Should I use SB? I tried pressing the refresh proxies in hrefer and nothing seems to happen it just says it's getting them and stays greyed out.
     
  2. jb2008

    jb2008 Senior Member

    Joined:
    Jul 15, 2010
    Messages:
    1,158
    Likes Received:
    972
    Occupation:
    Scraping, Harvesting in the Corn Fields
    Location:
    On my VPS servers
    No, of course the default proxies are not good enough. G is strict as hell on IP bans these days and only becomes stricter. Sometimes I perform genuine browser searches and I am soft banned (have to fill a captcha, ridiculous).

    My tip is to harvest your own proxies with your own ingenuity and hard work, yes SB, turn off autorefresh in hrefer and manually enter fresh proxies tested by SB from time to time. Even then G is going to be a bitch.

    Thankfully, Bing is getting better and better these days. Even in general manual searches, they just provide nearly all the relevant results G does in multiple languages, unlike yahoo which is restricted to english unless you train hrefer for other yahoo language sites. e.g. yahoo.cn . Also, yahoo is becoming powered by bing, I don't know if that's happened already entirely but it will be shortly.
     
    • Thanks Thanks x 1
  3. Ranko Jones

    Ranko Jones BANNED BANNED

    Joined:
    Mar 3, 2011
    Messages:
    1,677
    Likes Received:
    146
    I can only see the ability to put one url in for proxies wtf is that about?

    Can't I just load up a txt file with all the proxy lists like in SB?
     
  4. jb2008

    jb2008 Senior Member

    Joined:
    Jul 15, 2010
    Messages:
    1,158
    Likes Received:
    972
    Occupation:
    Scraping, Harvesting in the Corn Fields
    Location:
    On my VPS servers
    Yes, you turn off auto proxy refresh and load up manually in hrefer folder xproxy.txt .