Recent content by notmyrealnameoryear88

Also should note i tested with an email, no inbox found, created it and 3 minutes later inbox found. so it HAS to be pinging in some way to know it was created and didnt exist prior. This only works on gmail for them specifically if you use a google domain they run into the catch all where it...

I understand how Zerobounce works for the most part when it comes to catch-all (some cases im not sure, but i assume part might be heuristics around response time) but for gmail specifically it says mailbox_not_found which they claim is 100% accurate. if its 100% accurate it must not be a...

I'm scraping spotify, my personal IP only gets 1 429 per like day and i sleep for 10 seconds and all set. i do 4 "scrapes" per 2 seconds. the payload per page is ~ 100KB I should note. Data center proxies seem to get 429'd almost instantly.

I'm trying to scrape a website a few million pages, and I can do it locally with 0 issue no rate limiting after a week and all, but i've been looking around and see a lot about getting residential IPs for this task. However, I see that theres no real static residential IP's so I'm not really...

I want S5proxies Trial Plan

I saw something about using chrome dev tools protocol to bypass cloudflare I know the website im scraping uses cloudflare do you have any knowledge of that?

Im trying to figure out the best way to do this. As of right now with a regular IP i get rate limited from visiting pages after 1000 ish. I have all 10 million URLs and my first thought is to use proxies, so i scraped free proxies and rotated them, but that was a waste of time as it immediately...

First. no google couldnt take legal action. its been held up in the supreme court that scraping is legal. could i be banned? absolutely, could i be arrested? not in a million years. Second, I'm in a blackhat forum, is this not a blackhat community lmao im confused here, the whole thing with...

Update: the website acts the same as google. the search functions only gives 1000 reults before saying no more results.

I did try the site scraper method, but im running into some issue and im just trying to see if there is an alternative before i go this route. The website dynamically loads the results i want, its similar to pagination. you scroll down, more results "products" lets use, the search function will...

couldnt find that info in the sitemap sadly.

Currently I want to publicize a database from a company. all their data is public, but to guess every valid url would be 1x10^83 possible options. So i had the bright idea of doing "site:the url" for that website. This info is denied from the companies API so i thought well if i get every google...

So if you dont mind me asking, I am trying to scrape about 66 million google search results lmao. And at current rates of waiting 17 seconds per pagination itll take about 5 months. sadly i want this data to be updated monthly (rip me). What I'm wondering is if I host these lets say 12 proxies...