Scraping help

D

dannybuoy

Newbie

Nov 16, 2020

38

4

• Nov 18, 2020

• #1

Hi folks,

I hope you're doing great!

I'm trying to scrape the top result in Google for any given keyword, then get the keyword and URL into an RSS feed that automatically updates based on the additional queries and results.

I've got a super complicated workaround using scrapebox and some free tools - can anyone suggest an awesome tool that might be able to help?

 

M

mw3096026

BANNED

Nov 17, 2020

10

2

• Nov 19, 2020

• #2

What do you need to scrape?

 

H

hrlcllr

BANNED

Nov 17, 2020

9

1

• Nov 19, 2020

• #3

Best suggest. You can search and watch on YT. God bless you!

 

tolg

Junior Member

Oct 14, 2018

187

52

• Nov 19, 2020

• #4

Be aware of that google is a pain and monitors many parameters such as browser fingerprints (e.g. canvas). You will need a platform for solving captcha. Are you sure you want to do this in Google rather than Bing? In google is way more difficult.

 

bendutchman

Regular Member

Jr. VIP

Jun 1, 2012

446

215

• Nov 19, 2020

• #5

You would have better results using Google trends, but you really need to be more clear in what your goal is.

 

D

dannybuoy

Newbie

Nov 16, 2020

38

4

• Nov 19, 2020
• Thread Starter Thread Starter

• #6

Hi folks, thanks for your reply. My goal is to create an RSS feed similar to this prototype:

http://feeds.feedburner.com/Dantest
The URLs are the first result in google for that query. Each of the queries is a question - which I want to continue only getting questions.

The tricky part is substituting the title of the page in the feed, for the original searched for query - that's the part that I'm trying to crack.

Does that help? I hugely appreciate your comments.

 

B

bolgerstaceyr20

Newbie

Nov 17, 2020

9

1

• Nov 19, 2020

• #7

Better result in using google Trends. Or you can find it into YT.

 

M

michaelpsaez86

Newbie

Nov 17, 2020

8

0

• Nov 19, 2020

• #8

try to get search about Google trend

 

D

dannybuoy

Newbie

Nov 16, 2020

38

4

• Nov 19, 2020
• Thread Starter Thread Starter

• #9

Thanks so much for your replies. It's less about Google trends and more around getting data into an RSS feed.

 

bendutchman

Regular Member

Jr. VIP

Jun 1, 2012

446

215

• Nov 20, 2020

• #10

dannybuoy said:

Thanks so much for your replies. It's less about Google trends and more around getting data into an RSS feed.

Click to expand...

Just get yourself the Scrapebox automaton plugin. That should do most of the heavy lifting for you.