If you could scrape data from any website, what would you scrape?

[Icey Dan]

As a career web programmer, my latest project has been learning how to use PHP to efficiently scrape data from various websites. I have finally mastered the art of scraping and structuring the data how I need it for the web applications I build as a hobby.

Data is very valuable in this day and age as we all know, so my question is this: If you had the power to scrape data from any site, what site would you scrape? And why?

 

[Akermi]

We're in the same boat.
I hope members here could help with useful ideas.

 

[Aiden L]

As a career web programmer, my latest project has been learning how to use PHP to efficiently scrape data from various websites. I have finally mastered the art of scraping and structuring the data how I need it for the web applications I build as a hobby.

Data is very valuable in this day and age as we all know, so my question is this: If you had the power to scrape data from any site, what site would you scrape? And why?

Any adult website as adult and dating are very profitable niches

 

[Icey Dan]

Any adult website as adult and dating are very profitable niches

Interesting idea. Mind elaborating on what specifically you would scrape from adult sites?

 

[Aiden L]

Interesting idea. Mind elaborating on what specifically you would scrape from adult sites?

What are you capable of?

 

[BlackMickey]

Create userful information for daytraders with data from the market.

I am not gonna say a lot about it, because it is alredy a plan I am working on.

Send me a DM if interested

 

[robertfresh]

I would scar for all the car parts, I mean every single piece of a car and every maker + year and so on.
A database like that would be nice.

 

[Ipnp]

scraping data is old shit now, we are using API instead

 

[LocalSeoguy40]

the sites email lists

 

[ygg]

I'd scrape ads and sell the data

 

[HNB123]

this is really helpfull. i was wondering why all that scrapers data for what?
i have an idea for newbie like me lol
do you mind share 1 sample site with what data to scrape inside? it will be a training for me to try to scrape and understand the process

 

[Strover]

scraping data is old shit now, we are using API instead

APIs are made using scraped data...

 

[vario150]

scraping data is old shit now, we are using API instead

Google and Bing must be shit since they are biggest scraper sites of all

 

[Aiden L]

scraping data is old shit now, we are using API instead

April fools?

 

[Deleted member 1333509]

bobs and vegene.

 

[Ipnp]

April fools?

I scraped +20mb of data from the time when this thread was written

 

[Ipnp]

Google and Bing must be shit since they are biggest scraper sites of all

Do you prefer make request and get what u need , or spend hours to scrap 3000000 row ?

 

[vario150]

Do you prefer make request and get what u need , or spend hours to scrap 3000000 row ?

I prefer spend days to scrape 10,000,000 instead of got rate limited

 

[Ipnp]

Are U scared from losing your source of income then , did you now that you can make any website an api shit in few lines ?

 

[vario150]

Are U scared from losing your source of income then , did you now that you can make any website an api shit in few lines ?

The choice whether to use a few lines or thousands lines of code doesn't matter. Instead, what matters is writing code that suitable for its intended use.

I'm not against API whatsoever. I use web scraping whenever the API is suck.

I'm free anyway. If not spent most of my time watching anime, i write code to kill time