1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

crawlers

  1. KM1
    Thread

    Block majestic, ahrefs, moz, etc

    Anybody has a robots.txt file to block out these crawlers? Majestic Moz Ahrefs Other major crawlers.. I can make one, but it takes time. If...
    Thread by: KM1, Jul 27, 2017, 2 replies, in forum: Black Hat SEO
  2. chrismartin4u
    Thread

    Google crawl rate and indexing related issue

    Hello Friends, I need help regarding google crawl rate and indexing related issue. Is it possible that google crawler start indexing old content...
    Thread by: chrismartin4u, May 10, 2017, 0 replies, in forum: White Hat SEO
  3. tinyboxer
    Thread

    looking for a tool that filters usernames / IDs by gender - anyone knows?

    hey, i'm looking for a tool that knows to distinguish between male and female instagram users. Tooligram claims to be able to do that, but i cant...
    Thread by: tinyboxer, Jan 16, 2016, 0 replies, in forum: Instagram
  4. alexel
    Thread

    What HTML Tags Are Significant to Search Engines

    I know that obviously crawlers and spiders index sites based on their meta, H1, H2, alt, title, and all the basic ones. But do search engines...
    Thread by: alexel, Sep 24, 2015, 1 replies, in forum: Black Hat SEO
  5. rodzzlessa93
    Thread

    Developing social media webbots

    Hello I'm interested in developing advanced web-bots for social media websites like twitter, Instagram, YouTube, and Pintrest. To give you guys a...
    Thread by: rodzzlessa93, Jan 3, 2014, 6 replies, in forum: Other Languages
  6. howard310
    Thread

    What is the best scraper our right now?

    Looking for fresh company data across the US.
    Thread by: howard310, Sep 27, 2010, 0 replies, in forum: Black Hat SEO Tools
Showing results 1 to 6 of 6