1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

Expert wanted Collect 1.000.000 Tinder profiles

Discussion in 'Hire a Freelancer' started by maneu55, Oct 11, 2016.

  1. maneu55

    maneu55 Newbie

    Joined:
    Apr 3, 2013
    Messages:
    7
    Likes Received:
    0
    Hi,

    I search an expert freelancer who can do this:


    Tinder profile scraper
    Please scrape the data of 1.000.000 Australian Tinder profiles and store them into a MySQL database and pictures into a S3 bucket.
    We need 500k male and 500k female Tinder profiles.
    It should be easily possible to scrape more profiles in other countries like Switzerland or US.
    All code must be done in PHP.
    If you need Tinder credits, please buy them and bill us. Scrape the data only using straight profiles, so you need 2 profiles for scraping: a straight man and a straight woman.
    Before running the big scrape, please share some data with us to double check and get feedback.
    1) Store all profile pictures
    • store all images to a folder structure like AU/AU_profile_id/main.jpg
    • AU is for Australia, similar to Switzerland CH/CH_profile_id/pic1.jpg
    • profile_id is the tinder profile id
    • store inside main.jpg which is their main picture and all following pictures as pic1.jpg, pic2.jpg, ...
    • all profile pictures must be saved later to AWS S3 bucket
    2) Collect all data in mysql tables
    tinder_profile
    • profile id
      should be index
    • country like DE (= Germany), AU (= Australia)
    • name
    • birthday / age
    • all data you receive from tinder
    • number of profile pictures
    • location lon / lat
    • distance_km (distance in kilometers)
    • ... anything else you get
    tinder_friends
    • profile_id
      should be index
    • facebook_id_friend
      should be index
    • if possible store all friends (facebook) a user has and all the data that comes with a friend
    • ...
    tinder_profile_pictures
    store all meta picture informations here (not the picture itsself but the picture informations)
    profile_id and picture is unique key together
    • profile_id
    • picture (main, pic1, pic2, pic3, ...)
    • country
    • height
    • width
    • file
      like DE/profile_id/main.jpg
    make sure to check if the picture file actually exists, is a valid image and not filesize zero before adding an entry to the tinder_profile_pictures table. There should be no orphans. The number of profile picture files must be equal to the number of count(tinder_profile_pictures)
    3) Handles proxies
    Proxy management through a mysql table proxies.
    Should automatically pick a working proxy from a list. If one of the provided proxies becomes invalid, it should be marked (including timestamp) as invalid.
    An additional single php script should check which of the proxies is online.
    4) Handles Tinder tokens
    Wether defining a set of tokens is more stable or generating own ones, we have to discuss.
    5) Show data
    Make a show_data.php script that has a query in the beginning to select tinder profiles and show all data you stored like pictures, age, ...
    Finally
    PHP-Code
    • You are free on what to use, maybe coding just some scripts is the best solution
    • Use Composer to integrate further libraries
    • Do not check in into Github Credentials (like for the database).
      keep them in a config/config.php, but commit a config/config.php.sample
    You will get
    • a Github Repository to store the code
    • a MySQL database
    • an AWS S3 bucket to store the profile pictures.
    After starting please share
    • your GitHub username to share the repo with you
    • your E-Mail address

    Please make sure you read everything.
    Can you do that?
    Please message me.

    Thank you!
     
  2. sam.hunt0710

    sam.hunt0710 Jr. VIP Jr. VIP

    Joined:
    Feb 22, 2011
    Messages:
    4,999
    Likes Received:
    1,823
    Gender:
    Male
    Occupation:
    Owner Of a IT Comapny
    Location:
    India
  3. maneu55

    maneu55 Newbie

    Joined:
    Apr 3, 2013
    Messages:
    7
    Likes Received:
    0
    Thank you!
    Are there more experts who can solve that?
     
  4. maneu55

    maneu55 Newbie

    Joined:
    Apr 3, 2013
    Messages:
    7
    Likes Received:
    0
    Are there more devs around?
     
  5. Improving

    Improving Newbie

    Joined:
    Jun 20, 2017
    Messages:
    1
    Likes Received:
    0
    Hello everyone, is this project still going?