1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

Web Crawler Question

Discussion in 'Black Hat SEO' started by MGold2018, Jul 6, 2017.

  1. MGold2018

    MGold2018 Newbie

    Joined:
    Jul 6, 2017
    Messages:
    1
    Likes Received:
    0
    Gender:
    Male
    I'm running a web crawler to pull social media links from URLs. Right now the crawler goes through every link within each URL i provide, but I need it to crawl each URLs landing page, then move to the next without searching every link inside each URL. Can anyone assist with this?