Chimmy1990
Junior Member
- Mar 24, 2018
- 199
- 158
I’ve found a website that lists several sites I’m interested in. However, each listing uses an affiliate link/button that redirects to a blog page, where the actual URL is revealed.
Sure. I'll give it a try. ThanksI think you should try Playwright. It spins up a headless browser, runs all the client-side stuff, and gives you the final URL after everything’s loaded. If you’re doing this at scale def go with the async version and rotate proxies so you don’t get flagged. But if the redirect is just a simple meta tag or something static, you could run with a lighter setup like requests plus BeautifulSoup
That's actually a pretty good ideaSo? (Let somebody) Code something in python to process the data you want.
Oh, I see. It makes it quite hard to scrape all 5,000 domains because I have to click on each one individually to get the actual URL.Probably "paid" promotion, so they would be able to track the clicks,
Or just cloaking the redirects or something similar