Download Website lists by technology

kurosaki4d · May 24, 2018

Hey guys,

I'm searching for a website or a tool that can provide me with a big list of websites sorted by technologies, like : shopify websites, woocommerce websites, etc...

Also with a possibility of identifying with a filter the language of the website if it's american or french, that would be even better !

I tried yesterday https://techlookup.hunter.io/ with Prestashop websites, and i got the list in CSV which was perfectly what i needed for free. However, for some reason this website seems not to work anymore for me with other technologies, it just keep running for several minutes to generatae the CSV of websites, but no hope i get nothing !

I was wondering if some of you have an idea where i could find a similar service to the mentionned website elsewhere (either a website or a tool) or why hunter.io no longer works, if someone can test from his end to export the list for me that would be great also !

Thank you in advance !

ContentWriter · May 25, 2018

Well, I've only come across people who are selling lists. This seems like a good idea, though. I'll be following this to see what we can get from this.

BassTrackerBoats · May 25, 2018

kurosaki4d said:
Hey guys,

I'm searching for a website or a tool that can provide me with a big list of websites sorted by technologies, like : shopify websites, woocommerce websites, etc...

Also with a possibility of identifying with a filter the language of the website if it's american or french, that would be even better !

I tried yesterday https://techlookup.hunter.io/ with Prestashop websites, and i got the list in CSV which was perfectly what i needed for free. However, for some reason this website seems not to work anymore for me with other technologies, it just keep running for several minutes to generatae the CSV of websites, but no hope i get nothing !

I was wondering if some of you have an idea where i could find a similar service to the mentionned website elsewhere (either a website or a tool) or why hunter.io no longer works, if someone can test from his end to export the list for me that would be great also !

Thank you in advance !

Scrapebox can do that for you if you just use the proper KWs to search.

kurosaki4d · May 25, 2018

BassTrackerBoats said:
Scrapebox can do that for you if you just use the proper KWs to search.

Thanks guys,

Nice i googled it, it seems very powerful and handy for a lot of things,

- How can it find the technology, like if it's a Shopify/Soocommerce/Prestashop website, it's not actualy an info that is displayed anywhere in the site that you can find ?

- can it target a specific country or language of website by anychance ?

BassTrackerBoats · May 25, 2018

kurosaki4d said:
Thanks guys,

Nice i googled it, it seems very powerful and handy for a lot of things,

- How can it find the technology, like if it's a Shopify/Soocommerce/Prestashop website, it's not actualy an info that is displayed anywhere in the site that you can find ?

- can it target a specific country or language of website by anychance ?

One of the BHW members has a series of YT videos on how to use it:

https://www.youtube.com/user/looplinescrapebox

kurosaki4d · May 25, 2018

Thank you so much, i will more likely buy it. This seems to fit my needs, that's exactly what i was looking for !

Thank you very much for your help !

Alchemist-Programmer · May 28, 2018

I think you need to find technology signatures in html in order to categorize, its like finding needle in stack of hay !
Scrapebox is good for such kind of work.

kurosaki4d · May 28, 2018

I see, thank you for the headsup, by signature do you mean certain keywords presence in the website, like for example you find in the footer "made by Prestashop" ? Is that what you are refering to ?

chrishar · May 28, 2018

https://builtwith.com/
This is what you should use.

Alchemist-Programmer · May 28, 2018

kurosaki4d said:
I see, thank you for the headsup, by signature do you mean certain keywords presence in the website, like for example you find in the footer "made by Prestashop" ? Is that what you are refering to ?

Yes exactly. as well as HTTP Response codes tells about server technology.

kurosaki4d · May 28, 2018

chrishar said:
https://builtwith.com/
This is what you should use.

Thank you. It seems like a great website for that, it answers to my need, but more importantly does it allow to segment certain details for the listing, like geography, languages of the website these sort of details ?

kurosaki4d · May 28, 2018

Alchemist-Programmer said:
Yes exactly. as well as HTTP Response codes tells about server technology.

How can i interact with the HTTP Response codes ? Would you be kind enough to elaborate on that ?

Alchemist-Programmer · May 28, 2018

it’s possible in Linux by using curl(client for URL) command. To find which web server is running on a machine use blow command.

Please see the underlined text.

#curl -I http://www.linuxnix.com
# curl -I itig

HTTP/1.1 200 OK
Via: 1.1 us-PROXY1, 1.1 us-PROXY2
Connection: close
Proxy-Connection: close
Expires: Mon, 1 Jan 2001 00:00:00 GMT
Date: Mon, 14 Jun 2010 03:26:55 GMT
Content-Type: text/html; charset=utf-8
Server: Apache/2.2.3 (Red Hat)
X-Powered-By: PHP/5.1.6
Set-Cookie: b2933756b1f0288a74dd630e51954bff=5sl7jaqpgliambi2fq7q3ailv6; path=/
P3P: CP=”NOI ADM DEV PSAi COM NAV OUR OTRo STP IND DEM”
Last-Modified: Mon, 14 Jun 2010 03:26:55 GMT
Cache-Control: no-store, no-cache, must-revalidate, post-check=0, pre-check=0
Pragma: no-cache

chrishar · May 28, 2018

kurosaki4d said:
How can i interact with the HTTP Response codes ? Would you be kind enough to elaborate on that ?

Sure you can, but you have to buy their premium plan i think.

kurosaki4d · May 28, 2018

chrishar said:
Sure you can, but you have to buy their premium plan i think.

Yeah, Buildwith has a very expensive monthly plan, even the cheapest one of all !

chrishar · May 28, 2018

Check this then
https://alternativeto.net/software/built-with/

kurosaki4d · May 28, 2018

chrishar said:
Check this then
https://alternativeto.net/software/built-with/

Wow thanks to you i found out a lot of intresting tools, thank you for the useful link

devkareem · May 28, 2018

- Sometimes its found in the generator meta tag in HTML Code, view website source code and search for "generator"

- Not sure if shodan suites you, but i used to scrape drupal websites from there

kurosaki4d · May 28, 2018

ThePentester said:
- Sometimes its found in the generator meta tag in HTML Code, view website source code and search for "generator"

- Not sure if shodan suites you, but i used to scrape drupal websites from there

Thank you that's a realy good tip, i'll surely need it to search for it in Google !

devkareem · May 28, 2018

kurosaki4d said:
Thank you that's a realy good tip, i'll surely need it to search for it in Google !

No problem, if you faced any issues, PM me i will send you my Skype, feel free to ask me about anything.

Download Website lists by technology

Power Member

Banned - Selling via PM

Support Admin

Power Member

Support Admin

Power Member

Newbie

Power Member

Junior Member

Newbie

Power Member

Power Member

Newbie

Junior Member

Power Member

Junior Member

Power Member

Power Member

Power Member

Power Member

Main Menu

Marketplace

Making Money

BlackHat World