Ultimate scraper be like ?

[Alex D.]

definitely not sharing the code.
Can you explain more, which places should scrape and who will be interested in the solution ?

I thought you want to contribute to the community by creating free solutions.

 

[GrandScraperG]

I thought you want to contribute to the community by creating free solutions.

I will offer it as free service to bhw community

 

[Fast Cheap Services]

definitely not sharing the code.
Can you explain more, which places should scrape and who will be interested in the solution ?

cse.google.com + serpdigger.com
the ultimate way as i already said roi.im is the place you are looking for their tools are using custome search engine by google this way you can target a website/niche/country or whatever you get a full control of what you are looking for and where to look for it

 

[portina]

GrandScraperG, I've posted in a couple of your threads. What you are doing is interesting to me.

 

[KardoseR]

Do you have experience on automating tasks that take lots of time and bandwidth like video uploading? residential proxies suck for that because they rotate, any other idea besides building my own residential proxy network?

 

[TasDePixels]

Do you have experience on automating tasks that take lots of time and bandwidth like video uploading? residential proxies suck for that because they rotate, any other idea besides building my own residential proxy network?

try to buy proxies with better response time and/or (if you're building ur own thing) implement a system that handle errors and timeouts.

 

[GrandScraperG]

Do you have experience on automating tasks that take lots of time and bandwidth like video uploading? residential proxies suck for that because they rotate, any other idea besides building my own residential proxy network?

I have been using luminati for video upload and it worked fine. you should make sure your scripts for video upload are good.
And also to have a good deal about prices

also netnut is good service.

 

[GrandScraperG]

GrandScraperG, I've posted in a couple of your threads. What you are doing is interesting to me.

Thanks. what are most interesting you?

 

[blingin123]

I can use a solution if you can put one together

 

[GrandScraperG]

Similiarweb - I can do 100%

 

[perimeterbotdetection]

definitely not sharing the code.
Can you explain more, which places should scrape and who will be interested in the solution ?

A site I’m scraping is using PerimeterX. If your solution is able to avoid detection from then, I think we can work out a deal.

 

[perimeterbotdetection]

definitely not sharing the code.
Can you explain more, which places should scrape and who will be interested in the solution ?

Not much. I can't PM or post on your profile since I'm too new. Email me at [email protected]

 

[naskootbg]

1.Scrape sites that have sql query "SELECT something from..." or "UPDAE/INSERT" in HTTP request.
2. Scrape 5+ years old sites, that are not mobile friendly and not W3C valid.

 

[klonja]

using python and lxml xpath to scrape isnt a big deal for an ida and gdb professional but its all about creativity and good planing well i dont think your rev eng career would help you with this i used to be sort of hacker i have worked with alot of heap and use after free bugs but i think creativity is harder thn fuzzing and exploiting some memory leaks or idunno

 

[JimHalpert]

Can you create an ahref + sem rush + majestic + scrapbox?