Automation / scraping with raspberry pi

[Nash Commerce]

Wondering if there could be any good ways to scrape web info / automate social media with a raspberry pi?

Or any other ideas that could benefit a marketing campaign.

Thanks guys!

 

[bartosimpsonio]

Why would you do that?

Pretty sure you can run bots on pi's but they're too limited in storage and memory.

 

[Grudge]

Sure you could do. Just make sure to either store the data on cloud (in case it's a huge amount) or get a big SD card.

Why would you do that?

Pretty sure you can run bots on pi's but they're too limited in storage and memory.

If you have small scripts that you need to run I think it's worth it on the long run to get a pi + you can always sell it after you're done with it / use it for other projects.
As for the space, you can always get a cheap SD card (20$ for 128GB from amazon)

 

[dreadpixel]

Of course, you can run anything on a Raspberry Pi.
Make sure to enable RDP or SSH so you are login and see scripts running.
I particularly like the TMUX library from Linux. It is really great to run multiple stuff, otherwise, you can use CRON jobs.

 

[Bleach17]

Indeed Raspberry is cheap, small and effective tool for most of us. There are tons of tutorials about automation for Pi you van check them out.

 

[bartosimpsonio]

Sure you could do. Just make sure to either store the data on cloud (in case it's a huge amount) or get a big SD card.


If you have small scripts that you need to run I think it's worth it on the long run to get a pi + you can always sell it after you're done with it / use it for other projects.
As for the space, you can always get a cheap SD card (20$ for 128GB from amazon)

Why not just run it on any PC though? Sounds like a good excuse just to buy a cool gadget

 

[Grudge]

Why not just run it on any PC though? Sounds like a good excuse just to buy a cool gadget

Mostly because of the small space it occupies + depending on the country it might be cheaper to buy a pi rather than a PC
But I do get your point, pretty easy to fall into the "oh shoot, that'd be cool to have"

 

[joshgarbinson]

Why not just run it on any PC though? Sounds like a good excuse just to buy a cool gadget

Maybe you don't want to deteriorate your main PC keeping it on 24/7 and save money not having to rent a crappy VPS for $5/month.

 

[bartosimpsonio]

Maybe you don't want to deteriorate your main PC keeping it on 24/7 and save money not having to rent a crappy VPS for $5/month.

Meh. Raspberry's are cool but not a strong enough motive to run one for this purpose IMO.

 

[reversengineer]

Meh. Raspberry's are cool but not a strong enough motive to run one for this purpose IMO.

The latest generation is getting stronger than you think. https://www.raspberrypi.org/products/raspberry-pi-4-model-b/specifications/ The base model is $35 and comes with 2GB ram and a quad-core ARM v8 processor. According to https://medium.com/@ghalfacree/benchmarking-the-raspberry-pi-4-73e5afbcd54b the Pi model 4 is approximately four times faster than a model 2b. Compare to a VPS, which may have a faster server processor, but it is split between 50+ users which are all given fractional time on it. Most importantly, with the Pi, you own the hardware, and pay very little ongoing costs (low power draw).

 

[Nash Commerce]

The latest generation is getting stronger than you think. Here's the official site with tech specs. The base model is $35 and comes with 2GB ram and a quad-core ARM v8 processor. According to this benchmark the Pi model 4 is approximately four times faster than a model 2b. Compare to a VPS, which may have a faster server processor, but it is split between 50+ users which are all given fractional time on it. Most importantly, with the Pi, you own the hardware, and pay very little ongoing costs (low power draw).

Yeah I have the Pi4 with 8 gigs of ram.

 

[ristoriel]

Automation/scraping can be run on any PC, why have you chosen raspberry pi specifically?

 

[bobbyeiken]

Maybe lower electric bill, save more space.
In my opinion, event raspberry pi seems cheaper, vps might be a better option if you want to scrape a lot of pages.

For example raspberry pi priced around 100-120$, there are many 10$/month VPS with 2-4 core, that definitely much more powerful than raspberry pi.