Developing social media webbots

Hello I'm interested in developing advanced web-bots for social media websites like twitter, Instagram, YouTube, and Pintrest. To give you guys a bit of a background of myself I started off my programming adventure very simple building HTML and CSS static websites. About a month into that I discovered PHP so I could start building more advanced websites and I immediately became addicted to actual programming. Now about a year and a half froward I've become pretty good at PHP and Python and I can also work my way around JavaScript( obviously jquery too). Someway somehow I came across web scrapping and it was like a new found passion for programming I built a couple of decent scrappers and crawlers nothing to intense small projects to just have fun. Now though I want to take my bot making skills to next level the thing is I don't know if to continue using the programming languages I know or if to learn something like c# which by the research I've done is probably the best or one of the best languages for building bots(I also know python is good but is it good because how easy it is or how powerful you can make your bots?) Well I am interested to hear your advice. Take it easy and Happy New Years!

Cool man ! when you are ready come to me ! I have lots of bots to build

if you are already familiar with Python, then there is no need to waste your time for learning c#. I've created a lot of web-bots written in Python, so I would be happy to help you. feel free to ask any questions.

Awesome! I personally use PHP for scraping, but I've heard Python is great for multi-threaded scraping. Here are a few fundamentals if you're new to scraping:

- Make sure to integrate decent proxies into your scripts to avoid 403 blocks.
- Read the TOS for web pages you are scraping to make sure they allow it.
- Review all proxy laws in your country. In some countries it's illegal to intentionally circumvent a block or scrape data.
- If you're using PHP and you run into issues with Javascript scraping protection, check out PhantomJS and CasperJS.

Hope this helps! Happy scraping

Scrappers are fun but not very reliable and it is just not good practice in my opinion. BUT if you have to do it anyways make sure you do it through creating layers of abstraction by separating the logic of scraping deep down in one place most preferably so that in case the web UI changes which it will you don't have to go all over the place to make your scraping logic changes - it is just encapsulated in one place.

This way you can actually make a scraping engine and inject different behaviors into your scrapper. just my two cents.

Dude, Python. Its powerful, fast, and there's tones of libraries... I've been building bots n crawlers with python for a while and I love it.. Let me know if your interested in collaborating on some projects. Right now I'm working on building bots that go through the TOR network. I wana set up a vps and let them work nonstop.

TheVegan said:

Dude, Python. Its powerful, fast, and there's tones of libraries... I've been building bots n crawlers with python for a while and I love it.. Let me know if your interested in collaborating on some projects. Right now I'm working on building bots that go through the TOR network. I wana set up a vps and let them work nonstop.

Click to expand...

Haha building an automated army of social media bots