I'm running a web crawler to pull social media links from URLs. Right now the crawler goes through every link within each URL i provide, but I need it to crawl each URLs landing page, then move to the next without searching every link inside each URL. Can anyone assist with this?
You are using an out of date browser. It may not display this or other websites correctly.
You should upgrade or use an alternative browser.
You should upgrade or use an alternative browser.
Web Crawler Question
- Thread starter MGold2018
- Start date