1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

What kind of Bot / Web Scraper do you need ??

Discussion in 'Black Hat SEO Tools' started by DrGonzo, Dec 6, 2007.

  1. DrGonzo

    DrGonzo Junior Member

    Joined:
    Nov 14, 2007
    Messages:
    106
    Likes Received:
    46
    Hi

    I want to building a Bot/Web scraper and i would like to get some ideas from the forum of what type of information they would like to get scraped from websites, forums,blogs etc etc.

    Throw your suggestions at me please.

    Thanks

    _drgonzo_
     
  2. stan

    stan Newbie

    Joined:
    Dec 1, 2007
    Messages:
    45
    Likes Received:
    3
    Hey dude

    You know what would be awsome. If you build a software that will allow your html and htm sites to be scrapped by thousands of other sites. Rather than building rss feeds to accomplish this. Also the software should have a built in function or attachment that will crawl your websites and extract other links within it.

    Now that would be cool
     
    • Thanks Thanks x 1
  3. xXKingdom_SEOXx

    xXKingdom_SEOXx BANNED BANNED

    Joined:
    Nov 1, 2007
    Messages:
    646
    Likes Received:
    51
    that would be awesome. stan suggestion is great.
     
  4. stan

    stan Newbie

    Joined:
    Dec 1, 2007
    Messages:
    45
    Likes Received:
    3
    Once again thank you bro. If I can get a list of sites that you can ping html to for scrapping I may try my hand at building one of these myself and share it among my new friends here at BHW. :)
     
  5. Wombat

    Wombat Registered Member

    Joined:
    Apr 26, 2007
    Messages:
    84
    Likes Received:
    66
    Prstorm hits a lot of sites in the hope that some of them make their stats page public, which most don't.

    If you Google
    http://www.google.com/search?hl=en&q=webalizer+intitle%3A%E2%80%9Dusage+statistics+for+www%E2%80%9D&btnG=Google+Search
    it shows 148,000 sites that do.

    Is there some way to get those 148,000 URLs ?
    That would be very handy. :D
     
  6. asgdude

    asgdude Newbie

    Joined:
    Dec 4, 2007
    Messages:
    15
    Likes Received:
    0
    Yes, this would be a very nice idea.
     
  7. stan

    stan Newbie

    Joined:
    Dec 1, 2007
    Messages:
    45
    Likes Received:
    3
    I wouldn't mess with PRStorm if I were you. If your ISP won't ban you for using up all their bandwidth for running this program or for the complaints the
    webmasters will send to them for you referral log spamming their sites. You probably will still have to be worried about being banned from google, yahoo and msn once they realize where your backlinks are coming from. Or the hacker sites automatically sending surprises back to you once you start spamming them.

    To each his own. If you still wanna try it have fun.
     
  8. Denise

    Denise Junior Member

    Joined:
    May 1, 2007
    Messages:
    127
    Likes Received:
    32
    I'd like a scraper bot that scrapes the feeds that you tell it to scrape, automatically rewrites the articles and posts them to your blog. lol
     
  9. Wombat

    Wombat Registered Member

    Joined:
    Apr 26, 2007
    Messages:
    84
    Likes Received:
    66
    Blog Solution does all that and more.