1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

Help with locating a program that pulls data from websites (scraper, extractor etc.)

Discussion in 'White Hat SEO' started by truemarketing123, May 29, 2015.

  1. truemarketing123

    truemarketing123 Registered Member

    Joined:
    Sep 17, 2008
    Messages:
    50
    Likes Received:
    15
    Is their a program available that can pull data from websites?

    I am in commercial real estate in a particular niche. Their is no actual 'Multiple Listing Service' like residential property so I am not able to feed property data to my site. I manually pull them from broker emails which I receive daily or directly from the individual broker website and put it on my site. It would be nice to get this data and collect it either via csv or directly to the site

    Looking at pulling listing data, price, cap rate etc.

    Can someone point me to a program that is able to do this? Scraper, aggregator etc. I appreciate your help

    Thanks!
     
  2. billbraskey

    billbraskey Newbie

    Joined:
    Jan 18, 2015
    Messages:
    18
    Likes Received:
    1
    I'd be interested in something like this too
     
  3. ThopHayt

    ThopHayt Jr. VIP Jr. VIP Premium Member

    Joined:
    Jul 25, 2011
    Messages:
    6,704
    Likes Received:
    2,548
    You can simply subscribe to a MLS feed, that sounds like the same data you're asking about.

    As for scraping, I have no idea. I know that zillo builds their massive list by scraping, but it is obviously custom. Also... are you sure that the info you want to scrape isn't copyrighted? Food for thought.
     
  4. Repulsor

    Repulsor Power Member

    Joined:
    Jun 11, 2013
    Messages:
    770
    Likes Received:
    279
    Location:
    PHP Scripting ;)
    A custom made scrapper can pull all the datas from your individual brokers and put it into your site. There wont be anything already that does this, you need to make a new one for yourself. I am pretty good in doing that, by the way.

    Also, make sure there is no copyright issues or so, just to be safe. And if the emails are always in a constructed way, we can have a script that connects to your email, fetch the datas, process and feed it into your site. Works fine as long as all the emails are in the same format.
     
  5. mistachewy

    mistachewy Jr. VIP Jr. VIP

    Joined:
    Mar 31, 2008
    Messages:
    136
    Likes Received:
    35
    Location:
    Southern Oracle
    Check out https://import.io/
    I have been messing around with it for a bit now and I am able to scrape all sorts of data. Plus it is free and they have good tutorials so it is easy to figure out.
     
    • Thanks Thanks x 1
  6. hidedora

    hidedora Regular Member

    Joined:
    Sep 21, 2011
    Messages:
    213
    Likes Received:
    35
    OP you'll need to hire someone to create an html parser for the sites that you're interested in. This way you'll be able to manually harvest all the information that you want from the site and import it directly into your website. The process is a bit complex, but you can do it as you want. Regarding the email that you mentioned, I think that the only method is to manually import them or create an appropriate csv file and import it if you have a site like wordpress (it has csv importer).
     
  7. Machkiavelli

    Machkiavelli Junior Member

    Joined:
    Jan 31, 2015
    Messages:
    163
    Likes Received:
    46
    Occupation:
    Dev
    Location:
    InterWebz
    I can code a cron job to automatically updates your website when an article is added on your broker's website