How to scrap usernames or profile links of members of a subreddit?

MehtaM · Mar 18, 2022

If possible I want to scrap all members.
Otherwise I want to scrape usernames who created a post/thread in a specific subreddit in last 6 months.

ZonedOut23 · Mar 19, 2022

Not sure of anything public for this. It isn't possible for all members of a subreddit as the Reddit API does not support this. However, it is possible (and easy) to scrape all of the posts in the past 6 months.

MehtaM · Mar 21, 2022

ZonedOut23 said:
Not sure of anything public for this. It isn't possible for all members of a subreddit as the Reddit API does not support this. However, it is possible (and easy) to scrape all of the posts in the past 6 months.

How to do that?
No need to scrap all members. Only need to scrap list of members who posted in last 6 months.

GVald · Mar 21, 2022

Python, use Selenium

Vegeta · Mar 21, 2022

Why do you want to throw away usernames?

Awoken · Mar 21, 2022

MehtaM said:
How to do that?
No need to scrap all members. Only need to scrap list of members who posted in last 6 months.

check out praw reddit python api

MehtaM · Mar 23, 2022

GVald said:
Python, use Selenium

Any already available tool for this task?

Invictus · Mar 24, 2022

selenium/puppeteer

MehtaM said:
Any already available tool for this task?

most likely no. And you can't rely on public things, build one yourself

Evander · Apr 3, 2022

Well, there's no popular bot for that. You better a hire a coder (preferably freelancer) to develop a bot.
It won't cost a fortune to build such basic scrapers. Also note that, you may need to spend on proxies and VPS.
Good luck!

ibobo · Jun 24, 2022

hi
it seem to be an easy one task scrap, did you find someone ?

Algo · Jun 24, 2022

If you have some basic coding skills. You can do it easily with selenium or puppeteer.

Or use a no code bot builder like BAS or RTILA, I think RTILA is available on appsumo as a lifetime deal.

Or hire a freelancer to do it.

ibobo · Jun 24, 2022

check octoparse, they have video to show you how use their tools, you can scrap for free 10 000 data

Greatguynevermethim · Jun 25, 2022

it’s an easy scrap, you don‘t need proxies for read only, and selenium is not even needed

proxygo · Jun 27, 2022

MehtaM said:
If possible I want to scrape all members.
Otherwise I want to scrape usernames who created a post/thread
in a specific subreddit in last 6 months.

not sure if scrapebox can help you, wont hurt to ask @loopline

loopline · Jun 28, 2022

MehtaM said:
If possible I want to scrap all members.
Otherwise I want to scrape usernames who created a post/thread in a specific subreddit in last 6 months.

If you turn off javascript in your browser and you are not logged into reddit, can you still see what you want to see in a browser?

if so then you can probably do it with scrapebox.

The Undefined · Jun 28, 2022

Hey there @MehtaM,
If you're only doing this on a smaller scale and you have basic knowledge of JavaScript, you can do the following:

Let's say you're going to get the list of users from /r/pics; the subreddit URL would be: reddit.com/r/pics
If you add .json to the ending (almost) all Reddit URLs, it will return the page's content in JSON format.

With that said, by adding .json to reddit.com/r/pics, the URL would become reddit.com/r/pics.json.
Now by using a bit of JavaScript magic, you can get a list of usernames.

JavaScript:

const response = await fetch("https://www.reddit.com/r/pics.json")
const json = await response.clone().json()

const usernames = json.data.children.map(element => element.data.author)

The usernames variable will contain a list of strings (without the /u/ prefix) that you can use to your liking.
You might need to implement pagination based on the data.after field returned by the API or handle pagination manually by adding the after to the URL query.

To limit the date, you must manually check created_utc field of each post and stop pagination when it doesn't match your requirements.

proxygo · Jun 28, 2022

loopline said:
If you turn off javascript in your browser and you are not logged into reddit,
can you still see what you want to see in a browser?
if so then you can probably do it with scrapebox.

Cheers mat, hope you didnt mind me name dropping you.

loopline · Jun 28, 2022

proxygo said:
Cheers mat, hope you didnt mind me name dropping you.

any time, I appreciate it.

Thanks!

jasnlcas · Jun 29, 2022

PM me and I will scrape this information for you.

proxygo · Jun 29, 2022

jasnlcas said:
PM me and I will scrape this information for you.

@jasnlcas thats nice of you, just remember you cant charge for it.

How to scrap usernames or profile links of members of a subreddit?

Regular Member

Newbie

Regular Member

Power Member

Registered Member

Junior Member

Regular Member

Jr. VIP

Jr. VIP

Newbie

Jr. VIP

Newbie

Newbie

Jr. Executive VIP

Jr. Executive VIP

Regular Member

Jr. Executive VIP

Jr. Executive VIP

Registered Member

Jr. Executive VIP

Main Menu

Marketplace

Making Money

BlackHatWorld

Other