⭐ Search Engine To Scrape Unlimited Links ⭐

symonkings

Regular Member

Mar 7, 2024

494

152

• Feb 11, 2025

• #1

Tell me any new or current search engine where I can see/scrape unlimited website links.

I tried yandex but it's hit the limit when scroll, and mojeek too when it reaches 1000 results.

 

samkalt

Regular Member

Jan 3, 2024

279

61

• Feb 11, 2025

• #2

Following

 

hideath

Elite Member

Jun 16, 2019

6,620

7,350

• Feb 11, 2025

• #3

Bing shows up to 2K results, one of the biggest web scrapers as well,
idk if that's enough for you, but you'll get rate limited on most/all of the reliable SEs, use proxies

 

symonkings

Regular Member

Mar 7, 2024

494

152

• Feb 12, 2025
• Thread Starter Thread Starter

• #4

I still not got solution, there definitely must be search engine doesn't matter if it in any other language.

 

Steptoe

Elite Member

Jr. Executive VIP

Jr. VIP

Aug 9, 2017

5,592

6,875

• Feb 13, 2025

• #5

There really is no such thing any more. Every search engine with anything close to a decent-sized database will have throttling, bot detection and other methods in play to limit scraping. Your best bet is to scrape slow, use a load of stop words, negative keywords and so on to force different results sets to be shown.