1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

[Journey] Automated movie streaming website

Discussion in 'My Journey Discussions' started by robertforveve, Jun 24, 2017.

  1. robertforveve

    robertforveve Newbie

    Joined:
    Jul 11, 2015
    Messages:
    27
    Likes Received:
    15
    First of all, i want to say that i was inspired for this by @RazeByte 's journey about his movie streaming website.

    So i decided to make a similar platform and see how it goes. I just started to work on this, so it will take some time till it's finished (as i have some other projects for two clients ongoing, so they've got priority), but i do my best to allocate some time for this platform while I'm also working on my clients projects.

    I'll make two apps, let's talk about the first app. I already started working on it, it's an simple app, that makes http requests for pages 1-200 of every movie category with different types of ordering(popularity, rate etc.) (1-200 because on page 201+ IMDb gives 404 error). So it will scrape at least 10k movies per category (with it's name, description, IMDb link). This app is already built, what i still need to do on it is that I have to add support for mysql database (so all of this movies will go to a database), i haven't added support yet because i still don't have a host, my PC is broken right now so I have only my acer aspire one laptop that's not good enough to host a database while also scraping. On a single thread on my acer aspire one (intel atom with 1.6GHz) it's scraping with a speed of 700-1000 movies/minute, it will be slower after I connect it to a database, but I think I'll also make it to run on multiple threads and run it on a better PC. But even if I'll still run it on this shit laptop, I think I'll get arround 300-500 movies/minute (title, imdb link, description, imdb rate) so it's still good because it's gonna scrape arround 24k movies/hour. Also after it scraped all sections the app will start to look every day for coming soon movies.

    The second app will get every movie from the database and scrape the web for a link of it on openload or rapidvideo, then it will add the links to the remote upload features of the both websites, get new links and then auto add them to the database, from where the php script of the website will start working. For movies that it isn't able to get a video link it will skip them and let them sit here until next search takes place and so on (in future I'll make it to also search sites like thepiratebay and download if found then upload to openload, but only if sites start to generate some money as I don't want to pay for a good dedicated server with fast internet and also torrent allowed if it doesn't pay itself).

    For the php script, whenever it sees a movie with a watch link it will make the movie available on the site and also feature it on the home page of the website.

    For now these are my plans, I'll keep this thread updated when I get new ideas. As for the website hosting I think I'll start with blueangelhost as they ignore DMCA (will upgrade if sites starts to generate visits and profit).

    I estimate the launch of the site in ~1-2 months(I could do it in a week or two, but as I said I'm also working on other projects right now so I don't have much free time for this project).
    I don't have a goal for the minimum money that i want to get from this as I still don't know if the site will make profit because there are a lot of websites where people can watch movies so I have some big competition and also I'm noob at SEO. When the site is ready I'll buy hosting for one year (expand it to a VPS if it pays itself), so my only goal for now is to gain experience by doing this and at least make it pay itself.
     
    • Thanks Thanks x 3
  2. SkyNades

    SkyNades Newbie

    Joined:
    Nov 1, 2015
    Messages:
    47
    Likes Received:
    3
    Good luck man, and stay safe.
     
  3. Wineberg

    Wineberg Junior Member

    Joined:
    Jan 2, 2016
    Messages:
    102
    Likes Received:
    22
    Gender:
    Male
    Good luck with your journey mate.
     
  4. Writing Package

    Writing Package Jr. VIP Jr. VIP

    Joined:
    May 26, 2017
    Messages:
    130
    Likes Received:
    72
    Gender:
    Male
    Good luck with your journey! This journey is exciting! I hope you find huge success following RazeByte's Method.
     
    • Thanks Thanks x 1
  5. RazeByte

    RazeByte Junior Member

    Joined:
    Mar 13, 2014
    Messages:
    129
    Likes Received:
    106
    Gender:
    Male
    Occupation:
    Student
    Good luck bud!
     
    • Thanks Thanks x 1
  6. Writing Package

    Writing Package Jr. VIP Jr. VIP

    Joined:
    May 26, 2017
    Messages:
    130
    Likes Received:
    72
    Gender:
    Male
    The man himself! Awesome to see you coming by and updating the thread. It's awesome to see people being inspired by your method.
     
    • Thanks Thanks x 1
  7. stanleynguyen

    stanleynguyen Registered Member

    Joined:
    Dec 15, 2016
    Messages:
    50
    Likes Received:
    9
    Gender:
    Male
    What kind of code do you use to get movie links? Do you do code by yourself? Anw. Best of luck bro.
     
  8. robertforveve

    robertforveve Newbie

    Joined:
    Jul 11, 2015
    Messages:
    27
    Likes Received:
    15
    Thank you, guys

    Yes, i do the code by myself, currently the first program is written in python, but I think I'll write it also in php.
     
  9. robertforveve

    robertforveve Newbie

    Joined:
    Jul 11, 2015
    Messages:
    27
    Likes Received:
    15
    Quick update: I've decided to do the imdb scraping on my acer aspire one laptop, so i also hosted a mysql server on this laptop. It is a lot slower at scraping now because the laptop is pretty bad and the mysql database adds a lot of overheat to it (i paused it right now, i'll take some time to go out and chill with friends and till i'll be back home the laptop will be shut down). I'll start the imdb scrap when i get home. It scraped arround 50k titles (movie title, description, imdb id, genres) in like 3-4 hours.
    imdb.png

    Edit: I'm back home but there is a storm so I don't have electricity power right now so no internet, and I don't wanna scrape using my mobile internet.
     
    Last edited: Jun 26, 2017
  10. masta2133

    masta2133 Registered Member

    Joined:
    Nov 27, 2015
    Messages:
    65
    Likes Received:
    16
    Occupation:
    Busy
    Location:
    Nowhere
    Good luck man
     
  11. Writing Package

    Writing Package Jr. VIP Jr. VIP

    Joined:
    May 26, 2017
    Messages:
    130
    Likes Received:
    72
    Gender:
    Male
    That's amazing, Robertforveve! How did you learn to code?
     
  12. robertforveve

    robertforveve Newbie

    Joined:
    Jul 11, 2015
    Messages:
    27
    Likes Received:
    15
    When i was 7th grade I learned the basics of c++ with the help of my IT teacher. That were just the basic, like how to include a library, simple math, print to the console. Then i started to learn by myself watching tutorials on different sites (youtube, udemy courses etc.), reading ebooks til i got familiar with around 10 programming languages (once you know one it's easy and pretty fast to learn another), databases, json, csv and pretty much everything i need. I'm not an expert or advanced programmer, but i can say i'm in the moderate "class/category".
     
  13. Writing Package

    Writing Package Jr. VIP Jr. VIP

    Joined:
    May 26, 2017
    Messages:
    130
    Likes Received:
    72
    Gender:
    Male
    No way! I wish I had a little bit of help myself. What programming language do you advise to start first in order to quickly grasp the other languages quickly?
     
    • Thanks Thanks x 1
  14. robertforveve

    robertforveve Newbie

    Joined:
    Jul 11, 2015
    Messages:
    27
    Likes Received:
    15
    I think learning basic c/c++ is best(but don't dive in the desktop apps, scrapping, automation with these because they are also the hardest) to give you an idea. Or you can also start with javascript/python. You can go to the site codeacademy (i can't post links yet) and just make a free account, the site will dive you step to step on programming basics. After you've learned the basics, just watch free courses on the internet/youtube and watch programming livestreams so you'll help yourself make an idea and understand how others code.
     
    • Thanks Thanks x 1
  15. hualdo171

    hualdo171 Registered Member

    Joined:
    Dec 11, 2010
    Messages:
    59
    Likes Received:
    6
    Location:
    Flushei maior Malandro
    GL, the thread of @RazeByte inspired me do the same thing and i learn to code and buid my movie and series site with search and embed the link in real time.
     
    • Thanks Thanks x 1
  16. robertforveve

    robertforveve Newbie

    Joined:
    Jul 11, 2015
    Messages:
    27
    Likes Received:
    15
    Good luck with your journey, mate.

    Update: Finally IMDb scraping is done for now (the script will still search for coming soon movies once i'll put it on a VPS).
    imdb2.png

    125k movies is good for now, even if I know not every one will be available to find a watch link, but I'm hoping that at least 50-80k of them will be found by the second scraper after i write the code for it.

    I'll take some time for other things now, not related to this site (even if i pretty much didn't anything time-intensive for this project yet, as i wrote the imdb scraping script in 10 mins, another 5 to rewrite it but multithreaded and then just let it run). I just wanted to post this update for now.

    Next plan: I'll do some research about my competition (other websites) and decide the source from where the second scraper will get the movie links.
     
  17. Writing Package

    Writing Package Jr. VIP Jr. VIP

    Joined:
    May 26, 2017
    Messages:
    130
    Likes Received:
    72
    Gender:
    Male
    Awesome! I'll definitely check those out. Thanks for the insights, Robertforveve. I can't wait to see how your scrapper turns out!
     
    • Thanks Thanks x 1
  18. RazeByte

    RazeByte Junior Member

    Joined:
    Mar 13, 2014
    Messages:
    129
    Likes Received:
    106
    Gender:
    Male
    Occupation:
    Student
    The hard part is finding a reliable link source that you can easily scrape without getting flagged for scraping and doing it smartly so... good luck!
     
    • Thanks Thanks x 1
  19. robertforveve

    robertforveve Newbie

    Joined:
    Jul 11, 2015
    Messages:
    27
    Likes Received:
    15
    Yeah, I found some, but as I have 125k movies in my database I can't scrape all of them with the same ip cause they'll ban me, i think i'll use some residential ips. Also i'm thinking of adding a delay between the requests or a limit/hour because idk if the source will handle that many requests.
     
  20. RazeByte

    RazeByte Junior Member

    Joined:
    Mar 13, 2014
    Messages:
    129
    Likes Received:
    106
    Gender:
    Male
    Occupation:
    Student
    Yeah... that's more or less what I did.
     
    • Thanks Thanks x 1