1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

[Tutorial] Making Money with Scraped Content - Real Estate Edition

Discussion in 'Making Money' started by outscrape, Jan 19, 2017.

  1. outscrape

    outscrape Jr. VIP Jr. VIP

    Joined:
    Nov 23, 2016
    Messages:
    122
    Likes Received:
    80
    All,

    My guide to making money with web scraping seems pretty popular so I wanted to add to it. This is content derived from an upcoming ebook about web scraping I am putting together. You can also join my team's Web Scraping mastermind here if you like this sort of content.


    Web Scraping and the Real Estate Market
    [​IMG]

    Real estate is one of my favorite categories for web scraping. In my experience scraping the web and talking to scrapers, this is one of the big ones. Realtors, home buyers or sellers and even renters have to work fast, to get new sales data and to grab options or clients before someone else does. This makes scraped data in the real estate market extremely useful and valuable.

    The market is big: The Association of Real Estate License Law Officials (ARELLO) estimates that there are about 2 million active real estate licensees in the United States alone.

    The market is valuable: According to the 2007 Economic Census, there are 109,472 real estate brokerage firms operating in the United States. And in 2010, primary residences accounted for 29.5% of total family assets, according to the Federal Reserves Survey of Consumer Finances,


    And one last thing - the market and sales data is, mostly, public. So the market is great for scraping.



    So let's get into it. Here are the 3 basic ways I've seen to make money by scraping real estate data:

    1. Collecting property data to create databases/sites.
    2. Collecting property agent or owner data (Leadgen)
    3. Learning about trends



    • Collecting property data to create databases/sites.

    Heres something that may surprise you if you've never bought a property: the MLS (Multiple Listing Service) is a disaster. Yet it's been one of the most popular tools online for finding and selling homes!

    The MLS lets you list or search homes for sale (although sometimes you must have a special license or passcode to enter).

    Every local region and even small cities often have their own MLS, with totally different designs and often with different amounts and arrangements of data.

    Thats how Zillow, Homes.com, etc capitalize on the market - they aggregate the data from a variety of ugly, non-standardized, but usually free sources like the MLS or tax records. Then they offer a simple, standardized, national or global, search - with ads or premiums.

    That's also partly how a buyer's real estate agent makes money too - presenting you with clean, narrowed, daily results.

    You've probably been to Zillow before. Its grown over time, but it just began as a site that aggregated homes for rent or sale and placed them on a map, using trends and other information to estimate their worth.

    There was a time when Zillow's data was probably all scraped from other sites or pulled via APIs - now that they're big enough, you want to list your home with Zillow directly if it's for sale, add photos, etc. They make money from ads + premium accounts, and everybody wins, except possibly whoever they scraped the data from to begin with. It would take a lot to compete with Zillow at this stage.

    But thinking about what Zillow did, you can certainly come up other ways to use their data, and data from other real estate or property sites, to create value. And keep in mind that because home sales are so heavily regulated (well, theoretically), most sales data is public. You just have to collect it.

    In conversations with web scraping professionals, one scraper told me he pulled listings from craigslist, anything from *.appfolio.com, zillow, radpad, and more, and normalized all the data and listings based on address lookup. Basically, he did what Zillow did to get started - he didn't tell me why, but you can imagine it was probably to create some kind of valuable database.

    Another told me he specifically scraped all this data to create the following website: simplybyowner.com. Here's an image of that site:

    [​IMG]

    Pretty cool, huh? You'd never know that the backend of this was just collected with a bunch of scraped data. But thats often how the web works these days. There's some awesome free directory themes for wordpress and some other open source site options that you could probably use to throw together a site in literally a day or two for a realtor, with every house that's for sale, updated daily.

    Examples:

    You can pretty easily find this data all over the web and pull it, whether you're trying to repost it to draw traffic, or to create a specific database of some sort. The data is free, it comes and goes, and it's highly likely nobody is going to sue you for listing their home.



    • Collecting property or agent data for Lead generation


    Theres really nothing more valuable, sold regularly, than houses.

    Realtors can walk away from a few hours worth of work with a $20,000 commission check, especially in an upmarket (which it currently is).

    This means a couple of things, all of which you can use to your advantage.

    First, realtors NEED clients.

    If someone lists their home with a realtor, that realtor gets paid when the home sells. How do you find a realtor? A lot of the time, the same way you find a dentist: you go on Google and search.

    That means realtors with great, up-to-date content on their website, and those with decent SEO, or those with expensive ads, have a good chance to pull in more clients.

    And that means they're willing to spend a lot of money on SEO, on marketing, on content, and on finding clients, like up-to-date FSBO (For Sale By Owner) opportunities that they can convert into for-sale-by-them opportunities.

    You can offer them these things, if you can contact them - which you can, thanks to scraped data.

    One scraper told me his job was scraping from Realtor.com for a particular states' search result with the label 'new' on the listing. All information about the broker agency and information including number of previous sales and price difference between each sales was recorded. The content scraped was directly dumped to a database.

    So, the database had the following info:

    • New listings
    • The realtors contact info
    • The potential money that could be made
    I imagine you can think of some great ways to sort this data and use it.

    Another scraper said "the client needed a list of real estate info just to see agency/agent/house availability so they can contact the leads)," pulled from from Realestate.com.au & Rightmove.co.uk. Again, you can probably think of lots of great ways to make money with this.

    Second, lenders need clients.

    If you think realtors make a lot of money from a home sale, holy sh*t do lenders make a lot more. A 5% mortgage over 30 years on a $200,000 home brings the bank ALMOST $200,000 profit (over 30 years). So lenders will pay a LOT of money for leads - I know, my dad was a mortgage broker. From the loan officer to the mortgage broker, if you can find a way to get a client whose interested in buying a home, you've got a big payday coming.

    Yet another scraper specifically said he scraped Zillow for leads for loan officers. Remember - when someone sells their home, they usually buy another soon. That means a home for sale is a mortgage client waiting to happen.

    And to give you some insider info from the industry, a lot of mortgage brokers have salespeople combing through stacks of mortgage papers or online databases of mortgage or home sale records to find potential clients for refinancing, as well. Find someone who hasn't refinanced their home and you might just have a golden opportunity to grab a new client.


    • People want to learn about trends

    Huge troves of public data, lots of money at stake, swings in the economy - all of this means theres plenty of trend-related data available. If you thought the MLS was ugly - consider that basically every town/county has their own auditor website, custom-built. It spits out pretty much any data you might want, but it's a mess.

    One scraper told me he used real estate data to understand pricing trends and merge information from similar listings found on various real estate listing websites, to create trend data from all the mess of online scraped data in different places.

    You could do this too - and create all sorts of other information. There are obviously entire college departments dedicated to doing trend research on the economy, but usually on a global or national scale. Imagine what you could do with a bit of very specific local data from your area's sales data or MLS data.






    Alright, that's all the information I've got to share today. I hope this was helpful. If you have any suggestions, please add them to this thread. If you've done real estate scraping and don't mind sharing what you learned, I'm sure others will find it useful.

    I'm working on an ebook on how anyone can make money with web scraping - the internet, after all, is literally a huge collection of data. Follow this thread and I'll announce it later.

    Thanks for reading



    Outscrape
     
    • Thanks Thanks x 17
  2. mynameisfrankenstein

    mynameisfrankenstein Regular Member

    Joined:
    Apr 2, 2015
    Messages:
    431
    Likes Received:
    346
    Gender:
    Male
    Location:
    BC, Canada
    Another great contribution!

    I'm really interested in selling to Realtors, a guy with tech skills can really do damage for them.

    This insider knowledge is quite useful. Will dive into it for sure.
     
    • Thanks Thanks x 1
  3. The Curator

    The Curator Senior Member

    Joined:
    Dec 27, 2013
    Messages:
    1,054
    Likes Received:
    445
    I have a financial niche this would be great for, how do you translate the data into the template? I would love to make a template that scrapes data and puts it together for site visitors to peruse. I love to experiment with scraping - have Webharvy which is kinda neat. I would like to get Ubot eventually but that's farther down the list right now...
     
  4. WhereIsMyMoney

    WhereIsMyMoney Newbie

    Joined:
    Apr 7, 2016
    Messages:
    28
    Likes Received:
    11
    Thanks for sharing. This gave me some new ideas how I could use web scrapping to earn money. Do notify me when your book is ready!
     
  5. outscrape

    outscrape Jr. VIP Jr. VIP

    Joined:
    Nov 23, 2016
    Messages:
    122
    Likes Received:
    80
    What do you mean by how do you translate the data into the template ?
     
  6. san99

    san99 Jr. VIP Jr. VIP

    Joined:
    Nov 4, 2015
    Messages:
    122
    Likes Received:
    11
    Gender:
    Male
    Occupation:
    SEO Maniac
    Location:
    India
    Well thanks for sharing ......... buts it little hard to implement though !!!
     
  7. techn1k

    techn1k Junior Member

    Joined:
    Mar 4, 2013
    Messages:
    116
    Likes Received:
    33
    Nice idea. I have 2 questions:

    1. What software do You recommend for scraping? When someone is just starting out and don't want to invest a ton in high-end soft?
    2. What is Your favourite site / blog that addresses scraping business?

    Thanks,
    t.
     
    • Thanks Thanks x 1
  8. Boriss

    Boriss Elite Member

    Joined:
    Nov 7, 2009
    Messages:
    1,504
    Likes Received:
    597
    Location:
    Inside a Monitor
    Good thread OP. Thanks!
     
    • Thanks Thanks x 1
  9. jb1rd78

    jb1rd78 Regular Member

    Joined:
    Oct 22, 2014
    Messages:
    263
    Likes Received:
    56
    Gender:
    Male
    Location:
    USA
    I'm definitely going to do more research on this. I would be interested in your ebook as well, OP. More guides are welcome too.
     
    • Thanks Thanks x 1
  10. kredbroker

    kredbroker Newbie

    Joined:
    Sep 30, 2015
    Messages:
    25
    Likes Received:
    0
    I make scrap for clients job last 7 month (realtors to) and must say that you need quality. Free software that i know cant give you title, first and last name. You can use contact form submitter - paigham bot. They have demo but I did not succeed with this program.
     
  11. vebxperts

    vebxperts Jr. VIP Jr. VIP

    Joined:
    Nov 15, 2008
    Messages:
    1,999
    Likes Received:
    386
    Home Page:
    Where and how to scrape the data for own new website?
    I'm thinking of something like, I update the listings daily for latest available properties for sale/rent
    But where to scrape all this info from?