How to scrap usernames or profile links of members of a subreddit?

MehtaM

Regular Member
Joined
Mar 9, 2020
Messages
475
Reaction score
232
If possible I want to scrap all members.
Otherwise I want to scrape usernames who created a post/thread in a specific subreddit in last 6 months.
 

ZonedOut23

Newbie
Joined
Jan 29, 2019
Messages
26
Reaction score
14
Not sure of anything public for this. It isn't possible for all members of a subreddit as the Reddit API does not support this. However, it is possible (and easy) to scrape all of the posts in the past 6 months.
 

MehtaM

Regular Member
Joined
Mar 9, 2020
Messages
475
Reaction score
232
Not sure of anything public for this. It isn't possible for all members of a subreddit as the Reddit API does not support this. However, it is possible (and easy) to scrape all of the posts in the past 6 months.
How to do that?
No need to scrap all members. Only need to scrap list of members who posted in last 6 months.
 

Vegeta

Registered Member
Joined
Mar 30, 2009
Messages
54
Reaction score
9
Why do you want to throw away usernames?

1647873819582.png
 

Invictus

Jr. VIP
Jr. VIP
Joined
Feb 9, 2018
Messages
1,170
Reaction score
380
Website
invictusproxies.com
selenium/puppeteer
Any already available tool for this task?
most likely no. And you can't rely on public things, build one yourself :)
 

Evander

Jr. VIP
Jr. VIP
Joined
Aug 19, 2021
Messages
792
Reaction score
460
Website
LegitFame.com
Well, there's no popular bot for that. You better a hire a coder (preferably freelancer) to develop a bot.
It won't cost a fortune to build such basic scrapers. Also note that, you may need to spend on proxies and VPS.
Good luck!
 

ibobo

Newbie
Joined
Oct 9, 2016
Messages
38
Reaction score
3
hi
it seem to be an easy one task scrap, did you find someone ?
 

Algo

Jr. VIP
Jr. VIP
Joined
Mar 27, 2020
Messages
513
Reaction score
282
If you have some basic coding skills. You can do it easily with selenium or puppeteer.

Or use a no code bot builder like BAS or RTILA, I think RTILA is available on appsumo as a lifetime deal.

Or hire a freelancer to do it.
 

ibobo

Newbie
Joined
Oct 9, 2016
Messages
38
Reaction score
3
check octoparse, they have video to show you how use their tools, you can scrap for free 10 000 data
 

proxygo

Jr. Executive VIP
Jr. VIP
Joined
Nov 2, 2008
Messages
46,908
Reaction score
21,792
Website
www.localproxies.com
If possible I want to scrape all members.
Otherwise I want to scrape usernames who created a post/thread
in a specific subreddit in last 6 months.

not sure if scrapebox can help you, wont hurt to ask @loopline
 

loopline

Jr. Executive VIP
Jr. VIP
Joined
Jan 25, 2009
Messages
6,130
Reaction score
3,510
Website
contactformmarketing.com
If possible I want to scrap all members.
Otherwise I want to scrape usernames who created a post/thread in a specific subreddit in last 6 months.
If you turn off javascript in your browser and you are not logged into reddit, can you still see what you want to see in a browser?

if so then you can probably do it with scrapebox.
 

The Undefined

Regular Member
Joined
Sep 28, 2017
Messages
314
Reaction score
185
Hey there @MehtaM,
If you're only doing this on a smaller scale and you have basic knowledge of JavaScript, you can do the following:
  • Let's say you're going to get the list of users from /r/pics; the subreddit URL would be: reddit.com/r/pics
  • If you add .json to the ending (almost) all Reddit URLs, it will return the page's content in JSON format.
With that said, by adding .json to reddit.com/r/pics, the URL would become reddit.com/r/pics.json.
Now by using a bit of JavaScript magic, you can get a list of usernames.

JavaScript:
const response = await fetch("https://www.reddit.com/r/pics.json")
const json = await response.clone().json()

const usernames = json.data.children.map(element => element.data.author)

The usernames variable will contain a list of strings (without the /u/ prefix) that you can use to your liking.
You might need to implement pagination based on the data.after field returned by the API or handle pagination manually by adding the after to the URL query.

To limit the date, you must manually check created_utc field of each post and stop pagination when it doesn't match your requirements.
 

proxygo

Jr. Executive VIP
Jr. VIP
Joined
Nov 2, 2008
Messages
46,908
Reaction score
21,792
Website
www.localproxies.com
If you turn off javascript in your browser and you are not logged into reddit,
can you still see what you want to see in a browser?
if so then you can probably do it with scrapebox.

Cheers mat, hope you didnt mind me name dropping you.
 
Top