1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

Scrapebox on Password Protected Websites

Discussion in 'Black Hat SEO' started by OnticStein, May 30, 2017.

  1. OnticStein

    OnticStein Newbie

    Joined:
    Mar 12, 2013
    Messages:
    13
    Likes Received:
    7
    Hello,
    I'm currently trying to scrape data from 5 webpages that require a log in for access to the information. When I load the URL's into Scrapebox - it doesn't scrape properly due to the login requirement - does anyone have experience on how to navigate around this? I have the log in information for the website - I just don't know how to navigate it. Thanks.
     
  2. Ste Fishkin

    Ste Fishkin Jr. VIP Jr. VIP Premium Member

    Joined:
    May 14, 2011
    Messages:
    2,047
    Likes Received:
    10,422
    You can do this with screaming frog.

    I'm not sure scrapebox supports password protected pages.
     
    • Thanks Thanks x 1
  3. OnticStein

    OnticStein Newbie

    Joined:
    Mar 12, 2013
    Messages:
    13
    Likes Received:
    7
    Do you know if you can on the free version of screaming frog?
     
  4. Ste Fishkin

    Ste Fishkin Jr. VIP Jr. VIP Premium Member

    Joined:
    May 14, 2011
    Messages:
    2,047
    Likes Received:
    10,422
    Why don't you check their website. I'm not a frog salesman.
     
    • Thanks Thanks x 2
  5. loopline

    loopline Jr. VIP Jr. VIP

    Joined:
    Jan 25, 2009
    Messages:
    3,726
    Likes Received:
    1,993
    Gender:
    Male
    Home Page:
    Scrapebox does not support authentication, so you can't do this.
     
    • Thanks Thanks x 1
  6. OnticStein

    OnticStein Newbie

    Joined:
    Mar 12, 2013
    Messages:
    13
    Likes Received:
    7
    I've been using a Chrome extension called "Advanced Web Scraper" that seems to be working well with login authentication.