reddit python scraper

In Summary:​

The bot searches for Reddit posts mentioning a specific keyword, filters the results based on the creation date and maximum number of posts requested, and prints relevant details in a clear and organized format.

Bot Functionality:​

1. Reddit API Configuration:
   • Uses Reddit API credentials (client_id, client_secret, user_agent) to access Reddit through the praw (Python Reddit API Wrapper) library.
2. Elapsed Time Calculation:
   • Function time_ago: Calculates the time elapsed since the post was created in an abbreviated format (e.g., "2h" for 2 hours, "1d" for 1 day, "5m" for 5 months, etc.), excluding minutes.
3. Title Truncation:
   • Function truncate_title: Shortens post titles to a maximum of 50 characters, adding "..." if the title is too long.
4. Searching Reddit Posts:
   • Function cerca_keyword: Performs a search on Reddit using a specified keyword (keyword). The search parameters include:
     • subreddit: The subreddit to search in, with the default set to "all" to search across all subreddits.
     • max_posts: The maximum number of posts to return (limited to a specified value, e.g., 5).
     • max_days_old: Limits the search to posts created within the last X days (optional).
5. Filtering Results:
   • The search results are filtered to include only posts created within the specified maximum number of days (max_days_old).
6. Output of Results:
   • Prints the results in a readable format with the following information for each post:
     • Post URL: Full link to the post on Reddit.
     • Score: Number of upvotes on the post.
     • Comments: Number of comments on the post.
     • Creation Date: Time elapsed since the post was created.
     • Title: Title of the post, truncated if it exceeds 50 characters.

install praw
pip install praw



CODE PYTHON

import praw
from datetime import datetime, timezone, timedelta

# Impostare le credenziali dell'applicazione Reddit
reddit = praw.Reddit(
client_id='il_tuo_client_id',
client_secret='il_tuo_client_secret',
user_agent='il_tuo_user_agent'
)

# Funzione per calcolare il tempo trascorso in forma abbreviata (senza minuti)
def time_ago(created_utc):
now = datetime.now(timezone.utc)
post_time = datetime.fromtimestamp(created_utc, tz=timezone.utc)
diff = now - post_time

seconds = diff.total_seconds()
hours = seconds / 3600
days = hours / 24
months = days / 30
years = days / 365

if years >= 1:
return f"{int(years)}y"
elif months >= 1:
return f"{int(months)}m"
elif days >= 1:
return f"{int(days)}d"
elif hours >= 1:
return f"{int(hours)}h"
else:
return "Just now"

# Funzione per troncare titoli troppo lunghi
def truncate_title(title, length=50):
if len(title) > length:
return title[:length] + "..."
return title

# Funzione per cercare keyword e filtrare i risultati
def cerca_keyword(keyword, subreddit="all", max_posts=10, max_days_old=None):
risultati = []
now = datetime.now(timezone.utc)

# Se viene specificato max_days_old, calcoliamo la data limite
if max_days_old is not None:
max_date = now - timedelta(days=max_days_old)

for submission in reddit.subreddit(subreddit).search(keyword, limit=100): # Cerca fino a 100 post
post_time = datetime.fromtimestamp(submission.created_utc, tz=timezone.utc)

# Se max_days_old è specificato, salta i post più vecchi della data limite
if max_days_old is not None and post_time < max_date:
continue

time_passed = time_ago(submission.created_utc) # Calcola il tempo trascorso
truncated_title = truncate_title(submission.title) # Tronca il titolo
post_url = f"https://www.reddit.com{submission.permalink}" # Link del post

risultati.append({
'title': truncated_title,
'url': post_url,
'created_at': time_passed,
'score': submission.score, # Numero di upvote
'comments': submission.num_comments, # Numero di commenti
})

# Interrompe la raccolta dei post se raggiunge il limite max_posts
if len(risultati) >= max_posts:
break

return risultati

# Esempio di utilizzo per cercare "Python" su Reddit
keyword = "Python"
max_posts = 5 # Limita i risultati a 5 post
max_days_old = 30 # Filtra i post creati negli ultimi 30 giorni

risultati = cerca_keyword(keyword, max_posts=max_posts, max_days_old=max_days_old)

for post in risultati:
print(f"{post['url']} (Score: {post['score']}, Comments: {post['comments']}, Created: {post['created_at']}): {post['title']}")


see you next script

Could be useful, thanks

Grande lavoro.
I’m from Italy too. I’m focusing on searching abandoned subs with py
Any suggestion?
Grazie