Y T Nuke  
Results 1 to 3 of 3
I have a tool that scrapes specific information from Google. My server was already banned/blocked ...
  1. #1
    crackaway is offline Newbies
    Join Date
    Jun 2011
    Posts
    37
    Reputation
    10
    Thanks
    8
    Thanked 2 Times in 2 Posts

    Default Scraping from Google...Private or Public?

    I have a tool that scrapes specific information from Google. My server was already banned/blocked by them so we've rewritten the tool to be able to use proxies. What is the best way to do this out getting blocked again? What we do is scrap the information and then cache it. We rescrap/update the information every week or so. Should I spend money on something like squidproxy or use the $5 public proxy offerred from thebigproxylist.com? Your thoughts?

  2. #2
    Execute's Avatar
    Execute is offline Regular Member
    Join Date
    Aug 2010
    Location
    United Kingdom
    Posts
    373
    Reputation
    258
    Thanks
    261
    Thanked 4,202 Times in 291 Posts

    Default Re: Scraping from Google...Private or Public?

    What I usually do is scrape data with public proxies that I personally scrape with software that I own. Then when it comes to posting I use private proxies to speed up the process.

    Of course you can use private proxies to scrape with but I do not think it is needed.

    Hope that helped

  3. The Following User Says Thank You to Execute For This Useful Post:

    crackaway (12-24-2011)

  4. #3
    crackaway is offline Newbies
    Join Date
    Jun 2011
    Posts
    37
    Reputation
    10
    Thanks
    8
    Thanked 2 Times in 2 Posts

    Default Re: Scraping from Google...Private or Public?

    I guess I need to shed some more light on my intentions. I scrape Google by using a wordpress plugin. the plugin will pull information from a source (one of google's services) and then display it on the site. the information I pull is geographically specific. meaning the info I pull in US will be slightly different in another country (currently, listing may be in a different language, etc). Though I have users from all over the world, it would be hard to cater to each ones location. I think the best thing to do is go with the majority. The majority of my users are US based so I should use US proxies to pull the information. i am not worried what proxy I use to display the information, I think all it matters is what proxy pulls the information. So I guess my question is, where can I get really really fast US proxies from?

AdStract


Advertise on Black Hat World

Similar Threads

  1. Are you having problems scraping Google?
    By playboy in forum Proxies
    Replies: 12
    Last Post: 01-04-2012, 03:35 PM
  2. ScrapeBox: Private VS Public proxies
    By dowser in forum Black Hat SEO Tools
    Replies: 1
    Last Post: 08-13-2011, 11:52 PM
  3. Replies: 17
    Last Post: 11-03-2010, 02:56 AM
  4. Replies: 11
    Last Post: 11-02-2010, 05:52 AM

Bookmarks

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •  
  SEnukeX SEO Software
Proudly Powered by Hostwinds.com Web Hosting Click Here For Exclusive BHW Discounts!

Cheap Web Hosting


1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76