1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

In need of a bot which scrapes real estate agent listing site

Discussion in 'Black Hat SEO Tools' started by MaverickonRoids, Jul 28, 2012.

  1. MaverickonRoids

    MaverickonRoids Power Member

    Joined:
    Jun 3, 2012
    Messages:
    783
    Likes Received:
    439
    Location:
    Rep Score: Infinity
    Hey, I don't know if something like this exists, but I am looking for a bot that can scrape a site, for example realtor.ca, of each real estate agent within the settings that I set (for example, city specific). Then after that, log into my Wordpress backend to a site that has the "Connections" Wordpress plugin activated.

    The bot would then enter the scraped info from that realtor site into the specific fields in Connections. Is there a bot that can do this, and if not, what is the cheapest I can pick one up for?

    Thanks.
     
  2. Duffers5000

    Duffers5000 Elite Member

    Joined:
    Apr 1, 2012
    Messages:
    2,467
    Likes Received:
    7,615
    A good bot maker can put together something that will scrape whatever you want. Sounds like you just need a yellow pages scraper unless you want to take their listings as well
     
    • Thanks Thanks x 1
  3. jkwilson78

    jkwilson78 Regular Member Premium Member

    Joined:
    Jun 24, 2010
    Messages:
    224
    Likes Received:
    311
    If you're up for scraping the data yourself you could try Web Content Extractor. Just Google the name and the site will come up. You can configure it to scrape any website and it has proxy support and multithreading. A bit of a learning curve but I have used a lot of different scrapers like this and for the price it is one of the best and easiest to use. Plus if you find other sites you want to scrape you can use the same tool.
     
    • Thanks Thanks x 1
  4. MaverickonRoids

    MaverickonRoids Power Member

    Joined:
    Jun 3, 2012
    Messages:
    783
    Likes Received:
    439
    Location:
    Rep Score: Infinity
    Thanks for the replies.

    I would also need a bot that would be able to take that scraped information (name of agent, tel #, address, etc) and make posts with it on my site. The thing is, it would have to "talk" with the plugin I currently use, which is WP Connections.

    At the moment, I am just manually entering data from real estate sites into that WP Connections plugin, which displays it nicely on my site.

    Thanks again for your assistance.
     
  5. jkwilson78

    jkwilson78 Regular Member Premium Member

    Joined:
    Jun 24, 2010
    Messages:
    224
    Likes Received:
    311
    I think WP Connections can import CSV files? Not sure though. But you could scrape the data, save as csv, then import in bulk?
     
  6. MaverickonRoids

    MaverickonRoids Power Member

    Joined:
    Jun 3, 2012
    Messages:
    783
    Likes Received:
    439
    Location:
    Rep Score: Infinity
    That would be great, but the plugin doesn't have a CSV import option.
     
  7. jkwilson78

    jkwilson78 Regular Member Premium Member

    Joined:
    Jun 24, 2010
    Messages:
    224
    Likes Received:
    311
    I did some digging because I swore it could import. I was right and wrong.

    The free version doesn't but they have an import plugin you can buy for $15. At least I think it is the same plugin you are using :)

    Here's the link:
    http://connections-pro.com/pro-module/csv-import/
     
  8. loki_

    loki_ Registered Member

    Joined:
    May 20, 2012
    Messages:
    63
    Likes Received:
    8
    Location:
    #!/bin/bash/
    scrape yellow pages, split the data, insert into some db(mysql?)

    takes 2 hours to make such a bot from scratch really....