1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

Developing social media webbots

Discussion in 'Other Languages' started by rodzzlessa93, Jan 3, 2014.

  1. rodzzlessa93

    rodzzlessa93 Newbie

    Joined:
    Dec 9, 2013
    Messages:
    1
    Likes Received:
    0
    Hello I'm interested in developing advanced web-bots for social media websites like twitter, Instagram, YouTube, and Pintrest. To give you guys a bit of a background of myself I started off my programming adventure very simple building HTML and CSS static websites. About a month into that I discovered PHP so I could start building more advanced websites and I immediately became addicted to actual programming. Now about a year and a half froward I've become pretty good at PHP and Python and I can also work my way around JavaScript( obviously jquery too). Someway somehow I came across web scrapping and it was like a new found passion for programming I built a couple of decent scrappers and crawlers nothing to intense small projects to just have fun. Now though I want to take my bot making skills to next level the thing is I don't know if to continue using the programming languages I know or if to learn something like c# which by the research I've done is probably the best or one of the best languages for building bots(I also know python is good but is it good because how easy it is or how powerful you can make your bots?) Well I am interested to hear your advice. Take it easy and Happy New Years!
     
  2. mindlesswizard

    mindlesswizard Supreme Member

    Joined:
    Sep 3, 2010
    Messages:
    1,359
    Likes Received:
    282
    Occupation:
    Designer/Developer, Internet Marketer
    Location:
    in the shade of Everest
    Cool man ! when you are ready come to me ! I have lots of bots to build :D
     
  3. jamb0ss

    jamb0ss Junior Member

    Joined:
    Feb 9, 2012
    Messages:
    125
    Likes Received:
    45
    Occupation:
    Bots programming
    if you are already familiar with Python, then there is no need to waste your time for learning c#. I've created a lot of web-bots written in Python, so I would be happy to help you. feel free to ask any questions.
     
  4. Eightomic

    Eightomic Registered Member Premium Member

    Joined:
    Oct 20, 2012
    Messages:
    60
    Likes Received:
    13
    Occupation:
    Web Developer / Designer
    Location:
    United States
    Home Page:
    Awesome! I personally use PHP for scraping, but I've heard Python is great for multi-threaded scraping. Here are a few fundamentals if you're new to scraping:

    - Make sure to integrate decent proxies into your scripts to avoid 403 blocks.
    - Read the TOS for web pages you are scraping to make sure they allow it.
    - Review all proxy laws in your country. In some countries it's illegal to intentionally circumvent a block or scrape data.
    - If you're using PHP and you run into issues with Javascript scraping protection, check out PhantomJS and CasperJS.

    Hope this helps! Happy scraping :)
     
  5. Avocado Plugins

    Avocado Plugins Junior Member Premium Member

    Joined:
    Jun 23, 2014
    Messages:
    108
    Likes Received:
    50
    Occupation:
    Software Developer & Entrepreneur
    Location:
    Boston
    Home Page:
    Scrappers are fun but not very reliable and it is just not good practice in my opinion. BUT if you have to do it anyways make sure you do it through creating layers of abstraction by separating the logic of scraping deep down in one place most preferably so that in case the web UI changes which it will you don't have to go all over the place to make your scraping logic changes - it is just encapsulated in one place.

    This way you can actually make a scraping engine and inject different behaviors into your scrapper. just my two cents.
     
  6. TheVegan

    TheVegan Junior Member

    Joined:
    Mar 6, 2013
    Messages:
    179
    Likes Received:
    33
    Occupation:
    blackhat
    Location:
    Prague
    Dude, Python. Its powerful, fast, and there's tones of libraries... I've been building bots n crawlers with python for a while and I love it.. Let me know if your interested in collaborating on some projects. Right now I'm working on building bots that go through the TOR network. I wana set up a vps and let them work nonstop.
     
  7. mister_digital

    mister_digital Junior Member

    Joined:
    Jun 22, 2016
    Messages:
    104
    Likes Received:
    5
    Haha building an automated army of social media bots