1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

Automating Tinder with Machine Learning

Discussion in 'AI - Artificial Intelligence in Digital Marketing' started by Chris22, Feb 11, 2015.

Thread Status:
Not open for further replies.
  1. Chris22

    Chris22 Regular Member

    Joined:
    Sep 29, 2010
    Messages:
    400
    Likes Received:
    1,061
    Automating Tinder with Eigenfaces

    While my friends were getting sucked into "swiping" all day on their phones with Tinder, I eventually got fed up and designed a piece of software that automates everything on Tinder.

    [​IMG]

    Since Tinder's rising popularity, including its use by Olympic athletes such as snowboarder Rebecca Torr, Tinder has achieved critical adoption as a launchpoint for singles meeting singles. Its rising popularity has encouraged a wave of "Tinderbot" inventions by nerds doing things such as "swiping right" for everyone near their location (and of course those pesky spammers). It wasn't my intention to "one-up" the competition, but using the facial recognition algorithm Eigenfaces I built a bot that learns when to swipe right (like a person) AND swipe left (dislike a person) AND start your conversations.

    Dubbed "Tinderbox", the first version only took 3 weeks to build. It uses an existing Tinder account and taps into Tinder APIs, which is nice so you don't have to create an entirely new account. Tinderbox recreates the Tinder app in your browser, including the inbox and discovery preferences. The workflow is simple:

    • The built-in bot builds facial models using your likes/dislikes
    • Bot examines profile images, cropping faces
    • Faces are loaded into an "average" face representing choices
    • Eigenfaces are computed from average faces
    • Bot then makes future selections based on Eigenface comparison
    • Comparisons are essentially k-nearest neighbor selection
    [​IMG]

    Eigenface is a quick and easy algorithm to implement facial recognition without the use of complex software like OpenCV. Tinderbox first extracts faces using the Viola-Jones framework - specifically the jViolaJones implementation - and converts them to grayscale. Only pictures with single, identifiable faces are used (to filter out false positives). Then after each image is normalized, its pixels are converted into a matrix where they are then appended to a list of models. The models are then averaged into a single face used for future comparison (as noted in the workflow above). The bot requires you to make 60 yes/no choices before it has enough data to choose on your behalf. Then its on full auto-pilot.

    If you're interested in the code behind this, here's a snippet of the two main defs for computing Eigenfaces:
    Code:
      /**
       * Computes the EigenFaces matrix using a pixel matrix of multiple images.
       * @param pixelMatrix
       * @param meanColumn
       */
      def computeEigenFaces(pixelMatrix: Array[Array[Double]], meanColumn: Array[Double]): DoubleMatrix2D = {
        val diffMatrix = MatrixHelpers.computeDifferenceMatrixPixels(pixelMatrix, meanColumn)
        val covarianceMatrix = MatrixHelpers.computeCovarianceMatrix(pixelMatrix, diffMatrix)
        val eigenVectors = MatrixHelpers.computeEigenVectors(covarianceMatrix)
        computeEigenFaces(eigenVectors, diffMatrix)
      }
    
      /**
       * Computes the EigenFaces matrix for a dataset of Eigenvectors and a diff matrix.
       * @param eigenVectors
       * @param diffMatrix
       */
      def computeEigenFaces(eigenVectors: DoubleMatrix2D, diffMatrix: Array[Array[Double]]): DoubleMatrix2D = {
        val pixelCount = diffMatrix.length
        val imageCount = eigenVectors.columns()
        val rank = eigenVectors.rows()
        val eigenFaces = Array.ofDim[Double](pixelCount, rank)
    
        (0 to (rank-1)).foreach { i =>
          var sumSquare = 0.0
          (0 to (pixelCount-1)).foreach { j =>
            (0 to (imageCount-1)).foreach { k =>
              eigenFaces(j)(i) += diffMatrix(j)(k) * eigenVectors.get(i,k)
            }
            sumSquare += eigenFaces(j)(i) * eigenFaces(j)(i)
          }
          var norm = Math.sqrt(sumSquare)
          (0 to (pixelCount-1)).foreach { j =>
            eigenFaces(j)(i) /= norm
          }
        }
        val eigenFacesMatrix = new DenseDoubleMatrix2D(pixelCount, rank)
        eigenFacesMatrix.assign(eigenFaces)
      }
    
    And computing the distance is just as easy:

    Code:
      /**
       * Computes the distance between two images.
       * @param pixels1
       * @param pixels2
       */
      private def computeImageDistance(pixels1: Array[Double], pixels2: Array[Double]): Double = {
        var distance = 0.0
        val pixelCount = pixels1.length
        (0 to (pixelCount-1)).foreach { i =>
          var diff = pixels1(i) - pixels2(i)
          distance += diff * diff
        }
        Math.sqrt(distance / pixelCount)
      }
    
    If you're interested learning more about Eigenfaces, this paper has a relatively good overview of the math/matrix operations as well as a selection process using k-nearest neighbor. The paper is a description of another Eigenface facial recognition system.

    I also added a couple of features that are only in the paid version of Tinder. "Undo" buttons are there in the event a wrong swipe was made or the bot made a poor choice. Also, you can set your location to anywhere in the world for travel or curiosity purposes.

    The bot that runs in the background also has a messaging system that starts conversations. Using StanfordNLP, the bot analyzes the sentiment of each chat response and classifies it as positive or negative. Using a "message tree" (see diagram below), the bot selects from pre-programmed chat messages as a response based on the sender's sentiment. This continues up to 3 replies until the user is notified that a chat is ready to enter. The advantage of this? It removes the time involved in filtering new Tinder matches since a lot of people tend to drop off and "go dark" early in the process. Extended conversation is a strong indicator of interest.

    [​IMG]

    What are the results so far? The bot is amazingly effective. I would estimate an accuracy of up to 70% in its selections - though there may be a hindsight bias. Using a brand new account, I did a quick test to see how quickly the bot could get results. In 48 hours, the bot registered 21 matches (starting all of those conversations), made 4 extended conversations, and the bot itself made over 300 moves. A "move" is any step the bot makes in either sending a message or making a swipe. And in that time I barely needed to touch the app. I also created a dashboard to give me an overview of my metrics (see screenshot below).

    [​IMG]

    What do girls think of the bot? I've gone on at least 10 dates with the help of the bot and I've shown my partners the bot in its entirety. One date literally didn't believe me and thought I was pulling her leg. Another person thought it was really cool and wanted the full tour. All were in agreement that it is not creepy, though some felt it was borderline. Kind of nice considering it's not something you'd come across everyday.
    Am I still using the bot? I've actually turned it off for now. Admittedly, it worked too well and started to conflict with work. Although in a couple cases I had follow-ups and I'm still seeing one person.

    Check out the code online: https://github.com/crockpotveggies/tinderbox. Please feel free to contribute. I don't have current plans to pursue this commercially, though the license allows personal use and modification.







    Source: http://crockpotveggies.com/2015/02/09/automating-tinder-with-eigenfaces.html

    Thought this will be useful to a lot of people here, if you are going to run this yourself make sure you edit the message tree in the 'tinderbox/blob/master/app/models/bot/tasks/message/FunMessages.scala' file.
     
    • Thanks Thanks x 2
  2. GroundUp

    GroundUp Regular Member

    Joined:
    Jan 1, 2015
    Messages:
    333
    Likes Received:
    305
    thats amazingly geeky. good stuff. too lazy to read all the papers now. maybe some day when im bored.
    one question. can you play with the location / distance settings. if you wanted to extend your reach? is this getting creepier lol
     
  3. Chris22

    Chris22 Regular Member

    Joined:
    Sep 29, 2010
    Messages:
    400
    Likes Received:
    1,061
    Yes, I think the app lets you set your location
     
  4. TheSnowman

    TheSnowman Newbie

    Joined:
    Jun 10, 2014
    Messages:
    14
    Likes Received:
    1
    That's actually really clever! I'm impressed!

    One thing- as far as my understanding of eigen-models go this would limit you to one gender (you can't be bi) and one skin color (it would mess up the model if you had mixed-colored faces, wouldn't it?).

    You should take all the models from all the users and stack them together into perfect eigenface. This would make beauty competitions soo much more objective.

    Kudos.
     
  5. eshelt

    eshelt Junior Member

    Joined:
    Jan 11, 2010
    Messages:
    146
    Likes Received:
    98
    didnt tinder just change it so you have limited swipes per day now?
     
  6. Trepanated

    Trepanated Supreme Member

    Joined:
    Sep 18, 2010
    Messages:
    1,395
    Likes Received:
    5,381
    Chris,

    Is this something you built yourself?

    I only ask because I read about it on one of the big tech sites yesterday, so you are already getting some great media coverage.
     
  7. Chris22

    Chris22 Regular Member

    Joined:
    Sep 29, 2010
    Messages:
    400
    Likes Received:
    1,061
    I did not make this myself, I saw it on reddit and thought people here would find it useful. I have built something very similar to the chat part of it before though.
     
Thread Status:
Not open for further replies.