Discussion in 'Black Hat SEO' started by aussiejack, Nov 26, 2014.
Custom scraper for the footprint of the htaccess would do it.
Other ways are searching for "brand" "keyword" -site:brand.com gets a list of sites containing brand name and keyword.
Same for the url to find url links.
Mind to elaborate? .htaccess is run server side so there's no way to check what's in it.
Only way to test what's being blocked would be to have lots of domains, spider IDs and try to access the site yourself with the different IDs.
OP - since you joined this forum, nearly half of your posts have been to start a thread asking a question.
You might well get better mileage if you spend more time reading various threads, including the golden old ones that are stickied. Spend time to read and understand first, before starting a thread.
OP probably works for a search engine.
Yea, I work for google and tomorrow we get YOU
I actually spend a lot of time in reading and try not to ask redundant questions. Might be that this question was answered somewhere else. I did not find it. Also I presented my own method how I successfully found some PBNs which might have been helpful to some users as well because I dont like the thought of not providing some value back.
Nevertheless, your thread title is "How can I...", as most of your threads titles are (or words to that effect).
No problem with starting threads - that's what a forum is for, after all.
I just make the point that those who tend to get the most 'help' and meaningful responses on here aren't those who just pop up and start a bunch of threads RE everything that comes into their mind - rather, they're the ones who participate more broadly.
There's more to a forum than just asking questions.
I am active in some other forums as well (not seo related though) where I am in the lucky position to be the specialist and be more the answer-guy rather than the question-guy. Therefore I can completely relate to what you write.
What really is not good for a forum is when the same newbie questions are asked over and over again. That kills the spirit and annoys the experienced users. I consider most of my questions to be well-thought. If I look at the meaningful discussions with a lot of answers that arose from my last two questions for example I think that these threads are useful.
And "how can I" is a very good question. Might sound egocentric first. But it shows clearly what the thread is about and it also tells other users "How can you".
Create a scraper that appears as a rogue bot, see if you are blocked from scraping a site, put it in the "blocked by htaccess" list.
Then filter by keywords on the site, and or competitor brand name mentions.
I have no clue about how to build such a thing, but I think that would do it.
Search "domain.com" on Google. If they're smart they're using naked URL links, so that can show you at least some of them.
"links:sitename" ? It is obvious that webmaster cannot block google. So google will show up the best links for the particular webmaster money website. But using links:sitename we can get the links placed on good sites.
There's a great country song out West called "If tomorrow never comes". I really like that song.
Separate names with a comma.