Recent content by Chonchonts

Hi, I have no problems to scrap mails on any webpage but anyone have tips to how find emails on the web in an optimized way ? For example scrap on forums, tips with search engine, crawl the entire web, crawl some specific websites ?

Hello, I am a kotlin/java/python developer since now 9 years in datamining and data management fields. I have started to build a web scraping tools since few weeks and I am interested to know what are the essential features for you to have in a scraper tool now ? I implemented google results...

Hi, Some nice ideas (I think) I make to improve spintax: Mistakes : { picture!0,10 } => you have 10% of chance to introduce mispelling, to make more realistic the text. You can have: pictute (key missing, based on select keys neighbors), pivture, piccture (doble press), pictuer (swap...

Hello, I implement a simple Spintax interpreter for my bot, like with nested braces. {hello|hi}, my name is {olivier | {david | nobody} }. Do you have listen about advanced options for Spintax with additional features ? I am looking for implements new things if you have ideas. I think...

Hello, I build my bot with C# and Selenium. Anyone have some fast and efficient ways to check if proxy is anonymous or not ? Cheers

I have this problem with Selenium and search google button. Some buttons on the web are not button html tag but div or input tags with javascript function. So you need to simulate a click with your bot at the position of the button.

I think it's depends of lot of what you want to do. Java is very good with Java 8 version, that implements very efficient syntax to manipulate data with Streams. C# have anyway good implementations to manipulate data too with LINQ. Obviously, if you go in web bots you can go on HTML/CSS/JS...

Difficult to make money just with one server. To do what ?

Thanks for all the tips, I am beginning to see the other jobs in webcrawling to adapt my features :p. My webcrawler can crawl html fast with WebClient in C# (so if your don't care about javascript) or activate javascript processing (with Awesomium). I have also a section to make automated...

Hi ! I reup this thread because I have stop my project from years, but I have restart it ! I have implemented all this one: - Extract a very large kind of things in any webpage: links, images, sounds, emails, proxies, PDF, Words, Paragraph, Discogs links. - Extract tweets. - Use custom Regexes...

Another question: why do you use VPN4ALL ? Did you try others VPN ?

Hi, nice journey, clickbank seems to be a good choice for earn money :D. Do you have a lot of accounts for put your videos online ? And one ip one each ?

With C#, use Webrequest and no Webbrowser which is a slow component. I use a chrome extension for see post request and implements them in C#

Nice ! Pipeline idea, extract text to pdf files and filters are good. Also I will think to FTP crawler and multimedia document metadata extractor.

Yes, I can write a little tutorial for make a basic WebCrawler and suggest useful tools and tips ^^ (and others tutorials why not, I will think about that). But I will not post my complete webcrawler sorry :).