I know about using robots.txt, but for some reason I still believe that some of them is crawling my sites. Also... Is there a comprehensive list...
Hi guys, did you notice, that Majestic recently indexed lots of PBN-Links although the bot should be blocked via htaccess? Does anyone have...
I need a webcrawler that will go from site to site and create a list of sites where the registration Form contains only mail or mail+password and...
The last roughly 3 days the Google Submit URL linked below has not been working for me. I have tried tons of different URLs from different domains...
Where can I find an updated list of search engines and SEO crawlers to block my money sites from? Tks
Is there a tool or script which can pull up the embed code from a post in instagram and save it somewhere?
I am not new to this form but i never sign up to this form ( usually i never sign up to any website/form). So why you do sign up here you ask?...
Hello all I restart my new journey in this thread as I posted it initially in "Making Money" by mistake. Goals My objective is to launch an...
He everyone, i have a site in which you can get nutritional value on foods (compare them with one another). Right now the URL is like that :...
Hello, i've seen some websites with a board of live instagram Hastags. Like a website with a board pushing all photos posted with hashtag...
Hello blackhatworld users i need a little help about getting python code instagram crawler if there is a existing one so i can modify a little for...
Hi, I have a list of urls from an online retail store, they are all API product pages and display available product information (like title,...
Who can help me with a bot that can do the following: 1. crawl Google and find Magento CMS sites in a specific country . 2. if found, than check...
What is your favorite SEO Crawling tool? Do you use a desktop based tool or Cloud base tool?
hi everyone, so my developer managed to make the perfect crawler.. that can convert a profile into an email. Yes. its possible via emailhunter,...
Hello everyone, here I am again desperately seeking for help. I have list of leads here with their company url. I wanted to find the social...
Hello experts! I have 1000 website urls. I wanted to pull out their marketing persons' information so that I could save it in an excel file....
Hello! I am desperate, I wanted to build a list of leads by their social media accounts. I have here the basic information, the names and email...
Hello BHW Members ! We plan to scrape some DATA from Amazon, The crawler should collect product's information based on a specifc keyword search...
Currently developing website audit software, it gets data such as: Google index Bing index Yahoo index Alexa rank Google backlinks Alexa...
Separate names with a comma.
