How to Extract Domain from Google Search with Specific Words

[bikashdaga]

Hi All,

Wanted to understand if there is any Search Operators that can help me extract a list of domains that included specific search term in them.

Example: I wanted to extract all the available domains which have "JAVA" in them.

Like: learnjava.com

I am asking because I wanted to create backlinks from those domains.

I am using ScrapeBox but not sure if is there any way to do that.

Any help would be appreciated.

 

[Thunder Digitals]

I think you will need a python script to scrape those domains from SERPs.

 

[Steptoe]

The only search operator you could use (on Google at least) would be

inurl:java

But this will bring back all urls with 'java' in either the domain, the path, query string, or the filename. So you'd need some sort of processing script to strip the domains to root and check for the presence of the keyword. I don't think there's an off-the-shelf solution for this, you would have to program it yourself or hire someone to program it.

 

[bikashdaga]

I think you will need a python script to scrape those domains from SERPs.

Thanks for your response, do you have any such python script?

 

[Thunder Digitals]

Thanks for your response, do you have any such python script?

No, I don't have it.

 

[bikashdaga]

No, I don't have it.

Ok, thanks!

 

[bikashdaga]

No, I don't have it.

will this script work

import requests
from bs4 import BeautifulSoup

search_terms = 'example search terms'
num_pages = 5

def extract_domains_from_search_results(search_results):
    domains = []
    for search_result in search_results:
        url = search_result.find('a')['href']
        domain = url.split('/')[2]
        domains.append(domain)
    return domains

domains = []
for page in range(num_pages):
    start = page * 10
    url = f'https://www.google.com/search?q={search_terms}&start={start}'
    response = requests.get(url)
    soup = BeautifulSoup(response.text, 'html.parser')
    search_results = soup.find_all('div', class_='BNeawe UPmit AP7Wnd')
    domains += extract_domains_from_search_results(search_results)

print(domains)

 

[bikashdaga]

The only search operator you could use (on Google at least) would be

inurl:java

But this will bring back all urls with 'java' in either the domain, the path, query string, or the filename. So you'd need some sort of processing script to strip the domains to root and check for the presence of the keyword. I don't think there's an off-the-shelf solution for this, you would have to program it yourself or hire someone to program it.

I have tried this but this will not help with what I am looking for.

 

[mainceaft]

You're doing it wrong, you either scrap domains that ranked on the KW related to Java or you scrap (search) on all registered domains that had java on their domain name and that something not related to google.

 

[bikashdaga]

You're doing it wrong, you either scrap domains that ranked on the KW related to Java or you scrap (search) on all registered domains that had java on their domain name and that something not related to google.

I understand what you are saying, I have been following the method of what you just said. My objective is to gather all the requested domains and retrieve their website metrics from Ahrefs to filter out a list of websites with sufficient traffic. This list will be used to reach out for link-building opportunities.

However, some valuable websites with decent traffic may not rank well due to intense competition, causing us to overlook them.

 

[Steptoe]

Another approach would be to download zonefiles (basically lists of all registered domains) and extract the domains with your required keywords. I shared lists from December 2022 here: https://www.blackhatworld.com/seo/zone-files-november-2022.1453076/

 

[randomstrings]

There was seoquake plugin for Firefox and Chrome it did very much same you are asking. Have you tried

 

[bikashdaga]

There was seoquake plugin for Firefox and Chrome it did very much same you are asking. Have you tried

No, I haven't. I will give it a try. Thanks!

 

[SirLouen]

If you cannot build a scrapper and do it with code, you can go the easy way (more limited, but it will serve you to do this fast and with a good amount of results)

You can use as @Steptoe the inurl command to filter down results

First set your search to maximum in the google configuration settings (100 results per page)
Configure this bookmarklet: https://www.chrisains.com/seo-tools/extract-urls-from-web-serps/

Then you can simply copy this list of URL
You can browse further pages, maybe first 1000 results and keep copying those URL with the bookmarklet in place

Then with all those 1000 results, you can use a Google Spreadsheet copy all them there and with REGEXEXTRACT you can do a simple formula like

=REGEXEXTRACT(A1;"java[^/]*")

 

[bikashdaga]

If you cannot build a scrapper and do it with code, you can go the easy way (more limited, but it will serve you to do this fast and with a good amount of results)

You can use as @Steptoe the inurl command to filter down results

First set your search to maximum in the google configuration settings (100 results per page)
Configure this bookmarklet: https://www.chrisains.com/seo-tools/extract-urls-from-web-serps/

Then you can simply copy this list of URL
You can browse further pages, maybe first 1000 results and keep copying those URL with the bookmarklet in place

Then with all those 1000 results, you can use a Google Spreadsheet copy all them there and with https://support.google.com/docs/answer/3098244?hl=en you can do a simple formula like

=REGEXEXTRACT(A1;"java[^/]*")

Thanks for the suggestion!