Machine learning - databases

Blekhett · Sep 3, 2022

Hi,

I have a custome made scraper which gets the data of companies. The problem is, there are duplicates, missing emails or irrelevant niches.

Does please anybody knows if this part of job which would be otherwise outsourced to the human force can be done my a machine? And if so which platform you recommend?

yellowcat · Sep 3, 2022

you don't need machine learning, just a parser for removing irrelevant data.

PHPInjected · Sep 4, 2022

Are you exporting all of the data to a CSV once you're finished?

Have you thought about Regex? What language is your scraper coded in? Python?

Blekhett · Sep 4, 2022

I export to xlsx. I have a scraper in Apify.

Duplicates - by that I mean there is a business called "Hotel Grand" and "Restaurant Grand" but otherwise contact info are same. There are many scenerios like that.

Missing emails - I scrape the data from the competitor of Google in my country. Guessing even their scraper cannot fetch sometimes emails from the websites.

Irrelevant niches - I need niches with only physical location. It is hard to distinguish without checking the website for couple of seconds sometimes.

Blekhett · Sep 7, 2022

rahuldhawan said:
can i get copy?

Sure, send me 500$ and copy is yours

ThePBrown · Sep 9, 2022

I would suggest to build parser with filters to remove the information you dont need. Those things you need to tune untill its perfect i guess

The Oracle · Sep 9, 2022

This is not what ML is.

ML is more for predictions and 'optimization problems.'

It's silly magical/fantasy thinking to assume 'let the AI do this', LOL- it just goes to show how stupid most people are. Like that old Indian guy who wanted to seed actual clouds in the sky for 'cloud storage.'

You need a simple parser or script that removes duplicates and incomplete records.

And for the love of God educate yourself on some fundamentals of computer science and tech- assuming this isn't a troll post or thread.

Gogol · Sep 9, 2022

Blekhett said:
I have a custome made scraper which gets the data of companies. The problem is, there are duplicates, missing emails or irrelevant niches.

So in other words, the bot doesn’t do what it is supposed to do. Think about not saving duplicates, having email validation etc.. for example…

Quite a common task while making a bot.

Machine learning - databases

Blekhett

Newbie

yellowcat

Regular Member

PHPInjected

Elite Member

Blekhett

Newbie

Blekhett

Newbie

ThePBrown

Junior Member

The Oracle

Regular Member

Gogol

Administrator

Main Menu

Marketplace

Making Money

BlackHat World