nonchetos
Newbie
- Aug 1, 2017
- 15
- 1
Hi, recently I've been across a few web 2.0 plataforms with trouble for indexing, like kinja.com who's now using this <meta name="robots" content="noindex"> to avoid new websites to be indexed or webs.com which blocks the robots.txt to avoid being crawled by google. My request is: does anybody have an updated list on this noindex web 2.0 plataforms or the plataforms that could actually be indexed? This could help a lot for newbies on the black hat path. 