1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

Free Web Site Crawling [LARGE!]

Discussion in 'Black Hat SEO Tools' started by serge.w1lllliii, Aug 2, 2017.

  1. serge.w1lllliii

    serge.w1lllliii Newbie

    Joined:
    Aug 2, 2017
    Messages:
    2
    Likes Received:
    0
    Gender:
    Male
    Hi all!

    I'm developing web crawler for big websites (1-10 millions pages) more than half a year. Now I testing it on everything.
    What I store:
    - page info: titles, desc, headers, status code, load time, distance to index, out links, in links, more than 20 attributes
    - links: anchor, follow, index, alt text, image, robots blocked and more

    I offer a FREE crawl of any website (size less than 10 mln pages). As result I give you CSV, Excel file or access to MySQL database with all contents. Your data will NOT shared to anyone.

    Just write me PM.


    links.png pages.png
     
  2. valdu02100

    valdu02100 Newbie

    Joined:
    Aug 6, 2017
    Messages:
    2
    Likes Received:
    0
    Gender:
    Male
    Can you MP me ?
     
  3. bhlearner

    bhlearner Newbie

    Joined:
    Dec 17, 2009
    Messages:
    14
    Likes Received:
    0
    How are you crawling the pages?
     
  4. Gogol

    Gogol Jr. VIP Jr. VIP

    Joined:
    Sep 10, 2010
    Messages:
    3,478
    Likes Received:
    3,108
    Gender:
    Male
    Sure he is not using JCBs or shovels.
     
  5. tugeye

    tugeye Newbie

    Joined:
    Aug 14, 2017
    Messages:
    3
    Likes Received:
    0
    Gender:
    Male
    Are you still offering the free scrape?
     
  6. greatx

    greatx Newbie

    Joined:
    Feb 20, 2017
    Messages:
    2
    Likes Received:
    0
    Gender:
    Male
    Hi can u PM me pls

    i really need this also keeping learning for web penetrating

    i am lookig forward!
     
  7. tugeye

    tugeye Newbie

    Joined:
    Aug 14, 2017
    Messages:
    3
    Likes Received:
    0
    Gender:
    Male
    lol..how the hell do you pm on here?
     
  8. greatx

    greatx Newbie

    Joined:
    Feb 20, 2017
    Messages:
    2
    Likes Received:
    0
    Gender:
    Male
    ireally dunno but just wanna try it if he gonna make his promise