How I can get all pages from URL like this?

[ChrisLi]

Hi, Guys,
Can any body share a strategy that getting all pages from URL like : http://www.xxx.com/show.php?id=5q7uny8u http://www.xxx.com/show.php?id=3t5n8kmg http://www.xxx.com/show.php?id=8i9qfr0n

Can I get them from Semrush or I have to make a tool?
Thank you

 

[MorganX]

You can use SEMrush or Screaming Frog to find all the pages on your site, including those with dynamic URLs. If these tools don’t cover everything, you might need a custom script or tool to get all the URLs.

 

[StrixSeo]

You can use the Screaming Frog SEO Spider tool to extract all the page URLs

 

[protoadamantium]

Can I get them from Semrush or I have to make a tool?

If the URLs that you want to collect are hyperlinked on the given pages, then SEMrush, Screaming Frog or any website audit tool can collect the URLs provided that you have enough level of subscription in which all pages can be crawled.
There are also some plugins that can fetch internal and external links on pages so if there are few pages only then you can use that as well.

You can build your own customized tool, then you can do that in python with libraries like BeautifulSoup

If URLs are in text format then it could be more tricky as you will need to process text and extract URLs with match pattern through regex.

 

[bananasubmarine]

For some websites you won't be able to do that. The reason is that the parameter in the URL is a token or something similar.
It could also be a secret link to share a resource or similar. Think sharing links. Or links to confirm your email.

 

[Rank 360]

If the URLs are linked somewhere on your site, you can use tools like Screaming Frog to crawl and collect them. However, if the URLs are dynamically generated or aren't linked anywhere, you might need to create a custom script to scrape them. For that, using Python with libraries like BeautifulSoup or Scrapy can help you extract these URLs by crawling through the site or matching specific URL patterns.

 

[DozeX]

is it for import video?

 

[Ding Dong]

If you need detailed information about pages with specific query parameters, you might need a custom tool. For instance, you could build a web scraper or use existing scraping tools to crawl the site and extract URLs based on the pattern you're interested in.If you don't have a tool, starting with Semrush or a similar SEO tool is a good way to get a broad view, and if you need more specific data, you might consider custom scraping solutions.

 

[ChrisLi]

Thank you all above, buddy.