naughtynewell
Regular Member
- Mar 24, 2013
- 249
- 92
Right.
As some of you may be aware I've done a few of these over the years, and always petered out pretty quickly.
I'm a programmer. Elsewhere on this forum I've got a thread detailing some code I wrote that makes it easier for anyone to built a decent cam site that leverages Chaturbate's performers. If any programmers are part of the affiliate program there and have had a look at the kind of data that is available, you'll immediately understand why I felt it was a requirement to create something better and more streamlined.
For those that aren't programmers, let me explain a little bit about what I mean. Chaturbate offer a way of accessing the 'raw' data behind their website. That is, they have a web address that you can go to and it will give you a data representation of all of the performers that are currently online. The data that sits under this url is updated every minute with new data as performers come online and go offline, and viewer counts change etc.
That's great. But for someone that wants to build a website off of this data, it poses a really significant issue: the url downloads around 6Mb every time you load the page, which means hugely blown out page loads and a lot of data arriving on your website that you probably don't need. Say for example I want a single page with 20 female performers that have the most people watching them. Chaturbate usually has around ~6000 performers online at any one time, so each time I make that call, I immediately discard 5980 performers as I don't care about them.
So, what's to be done? How is it possible to change this monstrosity into something that is workable and functional for people to use in a better manner? Well - that's what I've already built, let me break down what exactly I've created and then I'll talk a little bit about where the rest of the journey leads.
The system that I've built is comprised of three parts:
- Cron
- API
- Live
Cron.
If you're not aware Cron just means something that happens on a regular schedule. The Cron component sits on a server and every minute makes a request to the Chaturbate API and downloads the 6Mb file. Then, connecting to my own database it aggregates the data that it has gotten from CB and extrapolates information that the current official API doesn't contain. Things like "when was this user last seen?" and "is this user online?" Something the official API is seriously lacking is that it only shows the users that are currently online, not the ones that are offline. It also removes a lot of the data that is contained in the official API that is defunct and generally not required.
The more complex part of the cron element is that it also monitors a performers 'sessions.' That is, it detects when a user comes online, tracks how many users are watching that performer, how long the session is, and when the session finishes. All of this happens every minute and is all stored in a database for later retrieval.
API
The api component of the system is a way for users to contact the database and retrieve data from it. It is currently public accessible by anyone (this will shortly change, continue reading for why and how).
The API has four different calls:
https://cb-meta.com/api/v1/performer/
https://cb-meta.com/api/v1/performer/siswet19/
https://cb-meta.com/api/v1/performer/siswet19/sessions/
https://cb-meta.com/api/v1/status/
Each of these does pretty much what you'd expect. Although there are a few more options that the top one supports.
The performer url also allows you to add some filters to the request to further narrow down the performers you'd like to find.
https://cb-meta.com/api/v1/performer/?online&gender=f&username=girl
The above call will only return performers that are online, female and have the word 'girl' in their username. There is also support for sorting:
https://cb-meta.com/api/v1/performer/?online&gender=f&sort=num_users&desc
This will return performers that are online, female and sort them by the most people watching first.
My API is paginated, meaning that the maximum number of performers returned from a call is 20. However, by appending another query element you are able to 'page' through results.
https://cb-meta.com/api/v1/performer/?online&gender=f&skip=20
You're probably working this out by now, but this call will return performers that are online, female and skip the first 20 results and show the second 20.
The second call is "https://cb-meta.com/api/v1/performer/siswet19/." You can think of this as viewing the details of a particular performer.
Taking the performer 'siswet19' (as I know she's a popular performer.) You are returned information specific to her. The interesting parts of this call are a short history of the most recent sessions she has performed. The maximum number of users she's had watching her over all sessions, her average number of users over all sessions, the longest session she's performed (in minutes), her average session length (in minutes) and the total number of watched minutes that she has had whilst my API has been tracking her. It also then contains links to her thumbnail, as well as iframe links.
The final call is to inspect the sessions of a particular user.
https://cb-meta.com/api/v1/performer/siswet19/sessions/
This is a paginated call that shows the session information for a particular performer, again, at a maximum of 20 per page, with the ability to use skip.
Live
The live part of the system is the newest and probably the one that is going to receive the most change and attention. It's a web socket server that allows anyone to connect to it (again, this will change in the future.) And every minute it publishes information in regards to users that came online and went offline. From the other thread that I've made on my work, this has been a feature that is MOST sought after. The socket can be connected to at https://ws.cb-meta.com/ and ever minute will post a 'tick' event which lists the users that went offline and came online as well as some other information that the cron component gathers while it performs the aggregation and data storage.
__________________________________________________________________________________
IMPORTANT
So - for the more keen eyed, or affiliates that have played with the API before, you've probably noticed something about the response data. It contains MY affiliate code. Meaning that if you were to use this API in your own website, it would actually drive traffic to my affiliate tracker and hence I would generate revenue from it.
I have implemented a way for you to use your own affiliate code while using the API though. If you would like to send a request to the server and have it use your affiliate code, then you can supply a cookie named "X-CB-API-KEY" where the value is your api key, if my server picks up that cookie is supplied it will put your code into the relevant urls in the response data.
__________________________________________________________________________________
Right. Now that I've gotten all of that explained and out of my head, lets talk about what my journey is going to entail and what I'm hoping to achieve by the end of it.
There are a few things that I'd LIKE to happen.
Number 1.
Spend the time building a frontend website that uses my API and is arguably better than what Chaturbate currently has to offer. Obviously I would still be using their performers, and the iframe which links back to their website, but the rest of the site could be mine to create and build on as I see fit.
Number 2.
The concept of 'achievements' based off of data that I am able to aggregate and discern. For example, a performer could get an achievement for their first session that's over 10 minutes long, or 30 minutes long, or 60 minutes long etc. They could get ones for reaching certain viewership milestones, or average viewers. They could have them for performing on consecutive days, or a certain amount of time in a given month, week or year. There is a huge wealth of data that is available that I am able to 'reward' performers for. All of which could be displayed on the frontend website that I build.
Number 3.
Privatise and monetise the REST api that I have built. Charging other web developers a monthly access fee to use the system would enable me to upgrade the servers that everything is running on to provide better, faster and more reliable access to the data that I currently already have stored. Creating a 'sign up' form is really easy and then implementing a security layer on top of that is really not that much of a challenge so that only users that are 'approved' are able to access the data. I am going to leave the API open and accessible to anyone for now, as I want as many people to play with it as is possible so that any kinks in the code can be worked out.
This journey isn't specifically about making money, but it's about making something that is solid, reliable and maintainable that I can use in the future to build things off of.
If you have any questions, or would like to know more about the API or how to use it. Follow the thread and ask questions!
As some of you may be aware I've done a few of these over the years, and always petered out pretty quickly.
I'm a programmer. Elsewhere on this forum I've got a thread detailing some code I wrote that makes it easier for anyone to built a decent cam site that leverages Chaturbate's performers. If any programmers are part of the affiliate program there and have had a look at the kind of data that is available, you'll immediately understand why I felt it was a requirement to create something better and more streamlined.
For those that aren't programmers, let me explain a little bit about what I mean. Chaturbate offer a way of accessing the 'raw' data behind their website. That is, they have a web address that you can go to and it will give you a data representation of all of the performers that are currently online. The data that sits under this url is updated every minute with new data as performers come online and go offline, and viewer counts change etc.
That's great. But for someone that wants to build a website off of this data, it poses a really significant issue: the url downloads around 6Mb every time you load the page, which means hugely blown out page loads and a lot of data arriving on your website that you probably don't need. Say for example I want a single page with 20 female performers that have the most people watching them. Chaturbate usually has around ~6000 performers online at any one time, so each time I make that call, I immediately discard 5980 performers as I don't care about them.
So, what's to be done? How is it possible to change this monstrosity into something that is workable and functional for people to use in a better manner? Well - that's what I've already built, let me break down what exactly I've created and then I'll talk a little bit about where the rest of the journey leads.
The system that I've built is comprised of three parts:
- Cron
- API
- Live
Cron.
If you're not aware Cron just means something that happens on a regular schedule. The Cron component sits on a server and every minute makes a request to the Chaturbate API and downloads the 6Mb file. Then, connecting to my own database it aggregates the data that it has gotten from CB and extrapolates information that the current official API doesn't contain. Things like "when was this user last seen?" and "is this user online?" Something the official API is seriously lacking is that it only shows the users that are currently online, not the ones that are offline. It also removes a lot of the data that is contained in the official API that is defunct and generally not required.
The more complex part of the cron element is that it also monitors a performers 'sessions.' That is, it detects when a user comes online, tracks how many users are watching that performer, how long the session is, and when the session finishes. All of this happens every minute and is all stored in a database for later retrieval.
API
The api component of the system is a way for users to contact the database and retrieve data from it. It is currently public accessible by anyone (this will shortly change, continue reading for why and how).
The API has four different calls:
https://cb-meta.com/api/v1/performer/
https://cb-meta.com/api/v1/performer/siswet19/
https://cb-meta.com/api/v1/performer/siswet19/sessions/
https://cb-meta.com/api/v1/status/
Each of these does pretty much what you'd expect. Although there are a few more options that the top one supports.
The performer url also allows you to add some filters to the request to further narrow down the performers you'd like to find.
https://cb-meta.com/api/v1/performer/?online&gender=f&username=girl
The above call will only return performers that are online, female and have the word 'girl' in their username. There is also support for sorting:
https://cb-meta.com/api/v1/performer/?online&gender=f&sort=num_users&desc
This will return performers that are online, female and sort them by the most people watching first.
My API is paginated, meaning that the maximum number of performers returned from a call is 20. However, by appending another query element you are able to 'page' through results.
https://cb-meta.com/api/v1/performer/?online&gender=f&skip=20
You're probably working this out by now, but this call will return performers that are online, female and skip the first 20 results and show the second 20.
The second call is "https://cb-meta.com/api/v1/performer/siswet19/." You can think of this as viewing the details of a particular performer.
Taking the performer 'siswet19' (as I know she's a popular performer.) You are returned information specific to her. The interesting parts of this call are a short history of the most recent sessions she has performed. The maximum number of users she's had watching her over all sessions, her average number of users over all sessions, the longest session she's performed (in minutes), her average session length (in minutes) and the total number of watched minutes that she has had whilst my API has been tracking her. It also then contains links to her thumbnail, as well as iframe links.
The final call is to inspect the sessions of a particular user.
https://cb-meta.com/api/v1/performer/siswet19/sessions/
This is a paginated call that shows the session information for a particular performer, again, at a maximum of 20 per page, with the ability to use skip.
Live
The live part of the system is the newest and probably the one that is going to receive the most change and attention. It's a web socket server that allows anyone to connect to it (again, this will change in the future.) And every minute it publishes information in regards to users that came online and went offline. From the other thread that I've made on my work, this has been a feature that is MOST sought after. The socket can be connected to at https://ws.cb-meta.com/ and ever minute will post a 'tick' event which lists the users that went offline and came online as well as some other information that the cron component gathers while it performs the aggregation and data storage.
__________________________________________________________________________________
IMPORTANT
So - for the more keen eyed, or affiliates that have played with the API before, you've probably noticed something about the response data. It contains MY affiliate code. Meaning that if you were to use this API in your own website, it would actually drive traffic to my affiliate tracker and hence I would generate revenue from it.
I have implemented a way for you to use your own affiliate code while using the API though. If you would like to send a request to the server and have it use your affiliate code, then you can supply a cookie named "X-CB-API-KEY" where the value is your api key, if my server picks up that cookie is supplied it will put your code into the relevant urls in the response data.
__________________________________________________________________________________
Right. Now that I've gotten all of that explained and out of my head, lets talk about what my journey is going to entail and what I'm hoping to achieve by the end of it.
There are a few things that I'd LIKE to happen.
Number 1.
Spend the time building a frontend website that uses my API and is arguably better than what Chaturbate currently has to offer. Obviously I would still be using their performers, and the iframe which links back to their website, but the rest of the site could be mine to create and build on as I see fit.
Number 2.
The concept of 'achievements' based off of data that I am able to aggregate and discern. For example, a performer could get an achievement for their first session that's over 10 minutes long, or 30 minutes long, or 60 minutes long etc. They could get ones for reaching certain viewership milestones, or average viewers. They could have them for performing on consecutive days, or a certain amount of time in a given month, week or year. There is a huge wealth of data that is available that I am able to 'reward' performers for. All of which could be displayed on the frontend website that I build.
Number 3.
Privatise and monetise the REST api that I have built. Charging other web developers a monthly access fee to use the system would enable me to upgrade the servers that everything is running on to provide better, faster and more reliable access to the data that I currently already have stored. Creating a 'sign up' form is really easy and then implementing a security layer on top of that is really not that much of a challenge so that only users that are 'approved' are able to access the data. I am going to leave the API open and accessible to anyone for now, as I want as many people to play with it as is possible so that any kinks in the code can be worked out.
This journey isn't specifically about making money, but it's about making something that is solid, reliable and maintainable that I can use in the future to build things off of.
If you have any questions, or would like to know more about the API or how to use it. Follow the thread and ask questions!
