Google Trends scraper - Interested?

superman100 · #1 (**permalink**) 12-10-2009, 03:29 PM

Hey guys,

I'm currently building a very basic Google Trends Scraper, which will fetch the current top 40 most searched terms every hour and store them into a database.

( www.google.com/trends )

This data can then be used to create autoblogs, twitter updates, etc. to quickly gain traffic from the trending topics.

Is there anyone here that would like to access this kind of data? What kind of features would you like to see?

Let me know before I create a user friendly version (I'll be free, of course!)

Post here if you're interested and i'll update things as we go...

Cheers!

**Ontopic** · #2 (**permalink**) 12-10-2009, 03:34 PM

Yeah, could be interesting.

thestrangefruit · #3 (**permalink**) 12-10-2009, 03:35 PM

Def. would be interested.

superman100 · #4 (**permalink**) 12-10-2009, 03:38 PM

Alright! How would you use it? You don't need to explain in depth but I need good feedback to create the best service possible!

otobse · #5 (**permalink**) 12-10-2009, 03:39 PM

i am also interested in it

**rickstar** · #6 (**permalink**) 12-10-2009, 03:43 PM

Could be interesting but what exactly are you going to be scraping? The articles from the top 40 items or?

superman100 · #7 (**permalink**) 12-10-2009, 03:48 PM

Quote:

Originally Posted by redrabbit

Could be interesting but what exactly are you going to be scraping? The articles from the top 40 items or?

Lets say I'll scrape the 40 most searched (per hour) terms and save them into a database each hour. Then, every day group them together to retrieve the most popular search term that day.

I suppose I could retrieve the numbers of results in Google based on the data to determine if there would be a lot of competition in the search engine results for that specific keyword.

Or, you could just fetch the data from me by rss (or however you guys would prefer.) and create autoblog content.

Don't know, just thinking loud I suppose.

**carsonrathi** · #8 (**permalink**) 12-10-2009, 03:58 PM

This kind of product is already on the way :-)

superman100 · #9 (**permalink**) 12-10-2009, 04:04 PM

Quote:

Originally Posted by carsonrathi

This kind of product is already on the way :-)

Is it something you're working on? If so, hit me up by PM and tell me more

superman100 · #10 (**permalink**) 12-10-2009, 04:17 PM

Quote:

Originally Posted by NYC

Yes guys, I have been working on already for a few months now the perfect solution for this service.

I will start selling the service within a week. The site will be RankMyTrends.com

This service will do many things, scrape articles based on the trend, autoblog it into blogger.com/any blog you direct it too. It makes the title of the blog the trend name, the tags with most relevant keywords related to the trend.

Also, in your "Admin" account, you will be allowed to add your Tracking URL from all of your offers in your network so if it is related to that google trend term, it will use that offer automatically in that article. It will also hide it with a hyperlink (also able to forward a domain to the offer).

These are just a few features of my service, I have been building it and its going to be amazing. I have tested it out and works very well (gets indexed while running and updates each hour as directed in admin control center).

If you are interested in this service, please press "Thank you" button so I can see how many people I can expect will get a service like this."

It will not be free since this has taken me months to create, and makes a lot of money when testing out with Clickbank offers.

Thank you, look forward to seeing some of your responses.

NYC

Cool. But, I do however plan to offer the service for free. Freemium is the future of web services and online marketing :-)

Hit me up by PM if you want to cooperate.

And all others reading: If you would like a FREE version, post a reply in this thread and let me know what kind of features you would love to see!

CHEERS

**sandycat** · #11 (**permalink**) 12-10-2009, 04:30 PM

Hi Superman,

I'd like to play with this too! Thanks for sharing

superman100 · #12 (**permalink**) 12-10-2009, 04:38 PM

Thanks so far guys - As we speak i've started to collect the data, so I will be able to launch some sort of preview/beta in a few days (it's a bit useless without the data, so i'll have to wait for a few days.)

As Google Trends updates every 60 minutes and there are 24 hours in a day i'll store about 960 keywords a day - then group them together so you'll get a rundown on a daily basis about the most popular / searched terms for that day.

Would you prefer a invite only model or public beta / preview?

axedbydax · #13 (**permalink**) 12-10-2009, 04:41 PM

this is a very promising service. I hope to get a copy once its done.

superman100 · #14 (**permalink**) 12-10-2009, 04:43 PM

axedbydax: You will!

Keep following this thread and let me know what kind of features you would like to see and how you will use it and i'll let you guys know once it's ready.

as I just wrote: already collecting data...

mejohnny · #15 (**permalink**) 12-10-2009, 04:44 PM

I'm interested. I would suggest invite.

sapik · #16 (**permalink**) 12-10-2009, 04:44 PM

can help testing maybe also programming?

Blueprint · #17 (**permalink**) 12-10-2009, 04:47 PM

THis would be very interesting, and again i'd agree with people on saying it should be by invite....

Baby Savant · #18 (**permalink**) 12-10-2009, 04:58 PM

Hey superman, would love to get a review copy whenever it's done.

Features I'd love to see:

- Weekly aggregate of top 100 keywords - 500 keywords
- Monthly aggregate of top 100 keywords - 500 keywords
- Personalized graphing capabilities (ie, if you wanted to see a picture representation of the progress of a certain keyword through a certain period of time, etc).

Similar to Wordtracker's service, this could be used to pick out topic trends - what works all around, what works on a seasonal basis, etc.

Creating ebooks/courses/packages on popular topics could potentially bring in alot of money.

In the short-term, this could definitely be used to push traffic to blogs, twitter, etc (again on a very short-term basis).

Sounds like a cool tool, please definitely keep me updated as to how it turns out.

faceicles · #19 (**permalink**) 12-10-2009, 05:13 PM

Sounds very cool. I would be interested in this!

goawayplease · #20 (**permalink**) 12-10-2009, 05:15 PM

How about a hook into Clickbank and CPA searches to show how to monetize the trends as well? Maybe make the graphing, and the monetization premium features.

You could also create a content scraping module for content spinners, and make that a premium feature too.

mahrouqi · #21 (**permalink**) 12-10-2009, 05:15 PM

Count me also! I would love to have your service for free. I suggest that you make the submitting of the keywords to the members on an hourly basis instead of daily as google trends do not last for so long and that's why it would be more useful for members to get these keywords on an hourly basis than daily basis.

RandomName · #22 (**permalink**) 12-10-2009, 05:16 PM

Have a chat with Nossie (member here @ bhw) as he did something similar with his product NicheShark

RN

**rickstar** · #23 (**permalink**) 12-10-2009, 05:20 PM

Quote:

Originally Posted by superman100

Cool. But, I do however plan to offer the service for free. Freemium is the future of web services and online marketing :-)

Hit me up by PM if you want to cooperate.

And all others reading: If you would like a FREE version, post a reply in this thread and let me know what kind of features you would love to see!

CHEERS

Now that you've told me what it is you're scraping, I'd definitely like access to a free version.

It will not only allow me to add new popular keyword variations to findabusiness.org but will allow for all future blog articles to be instantly up there.

So, count me in!

Cheers

ecki · #24 (**permalink**) 12-10-2009, 06:04 PM

And you all honestly think that google will honor you with loads of traffic for
scraping their content?!?!
Complete waste of time and energy!

ubermann · #25 (**permalink**) 12-10-2009, 07:41 PM

I would like to get in on a free version. Sounds like it would be a handy tool for us who use BHCB.

jdmeg6 · #26 (**permalink**) 12-10-2009, 08:13 PM

this would be a helpful useful tool

AlexNovakMillionaire · #27 (**permalink**) 12-10-2009, 08:33 PM

I would be very interested in this tool
If I could try it out, please shoot me a PM

unnamed · #28 (**permalink**) 12-10-2009, 09:05 PM

Quote:

Originally Posted by NYC

Yes guys, I have been working on already for a few months now the perfect solution for this service.

I will start selling the service within a week. The site will be RankMyTrends.com

This service will do many things, scrape articles based on the trend, autoblog it into blogger.com/any blog you direct it too. It makes the title of the blog the trend name, the tags with most relevant keywords related to the trend.

Also, in your "Admin" account, you will be allowed to add your Tracking URL from all of your offers in your network so if it is related to that google trend term, it will use that offer automatically in that article. It will also hide it with a hyperlink (also able to forward a domain to the offer).

These are just a few features of my service, I have been building it and its going to be amazing. I have tested it out and works very well (gets indexed while running and updates each hour as directed in admin control center).

If you are interested in this service, please press "Thank you" button so I can see how many people I can expect will get a service like this."

It will not be free since this has taken me months to create, and makes a lot of money when testing out with Clickbank offers.

Thank you, look forward to seeing some of your responses.

NYC

like your idea,but want to ask if this will be script based selling? or just service like providing generating sites/blogs ?

im looking for something like this long time , so put me in your list bud.)

thanks

Beager · #29 (**permalink**) 12-11-2009, 04:38 AM

Sounds like a nice idea, maybe add ebay labs, or twitter trend searches

axedbydax · #30 (**permalink**) 12-11-2009, 04:47 AM

Quote:

Originally Posted by superman100

axedbydax: You will!

Keep following this thread and let me know what kind of features you would like to see and how you will use it and i'll let you guys know once it's ready.

as I just wrote: already collecting data...

Thanks Superman. It will be great if you have the following features

take the top most site on a specific trend topic and rewrite to make a post.

get another 4 to be listed as related items/sites

If you can do that for at least top 5 trending topic then it will surely be a hit.

Then maybe if possible grab a video via tube sites to be included in the post.
Search for related items via amazon to be embedded on any text within the post for extra income (optional)

Thanks

DrJekyll · #31 (**permalink**) 12-11-2009, 04:54 AM

there is already a wso selling something like this.

will pm you.

showbizvet · #32 (**permalink**) 12-11-2009, 05:02 AM

Super thread... both the paid thing from carsonrathi and what Superman offers. The net moves fast and the fact that multiple people are doing/offering/developing shouldn't be considered a "ah, it's too late for me", rather the fact there is a need. I'm sure there might be a WSO, I do know that Niche Horde is out there, doing SOME, but certainly not all of what carsonrathi mentions.

super-oct · #33 (**permalink**) 12-11-2009, 05:25 AM

PM Im very interested on this ASAP lol

superman100 · #34 (**permalink**) 12-11-2009, 11:43 AM

Hi guys,

Thanks alot for all the feedback, interested and suggestions.

Here's what I've figured out so far:

1. You will be able to access the data on a per-hour basis as google trends updates every 60 minutes, so 5 minutes after their update the data will be accessable.

2. The trends and keywords will be visualized with graphs and charts to make it easy to see trending topics on a daily, weekly and monthly basis. I don't know exactly what it will look like but it will be there aswell.

3. The service will be invite-only in the beginning, and i'll probably let other members invite other members to keep things "within the family", so to speak.

4. I do plan to fetch trends from Twitter as well but it's not my top priority at the moment. Could also fetch from Bing and other search engines but like i just said: Google is the first priority.

5. The service will be free of charge. You'll probably have to opt-in with your email and create a username and password but besides from that, no strings...

Keithcrash · #35 (**permalink**) 12-11-2009, 12:19 PM

I think it sounds great, and would be interested myself.

Giuni · #36 (**permalink**) 12-11-2009, 12:39 PM

please let the creature out !!

i will like to work with your tool
cheers

superman100 · #37 (**permalink**) 12-11-2009, 01:19 PM

Thank you keithcrash and Giuni : You're added to the list of people who will be invited once it's ready. I do however need to work a little bit more on the tool before it can be released.

AND, i really NEED people to tell me how they would like to use the tool so I can create the right features.

ghettogong · #38 (**permalink**) 12-11-2009, 01:47 PM

@superman......YES, we need that softie!!!

It is a very valuable keyword resource.

Could i have a copy for testing. THX
Posted via Mobile Device

makeusrich2009 · #39 (**permalink**) 12-11-2009, 02:19 PM

Def. would be interested. plzz send me the software or script for google trends ??

keenly waiting for your reply....

superman100 · #40 (**permalink**) 12-11-2009, 02:21 PM

I'm very glad that there's a high interest for this. ghettogong and makeusrich2009 : You're added to the list! Please be patient as i'm currently working on a user friendly version.

Post your interest and request for features and i'll keep you updated..

ghettogong · #41 (**permalink**) 12-11-2009, 02:32 PM

Quote:

Originally Posted by superman100

I'm very glad that there's a high interest for this. ghettogong and makeusrich2009 : You're added to the list! Please be patient as i'm currently working on a user friendly version.

Post your interest and request for features and i'll keep you updated..

NICE!

Features what i can think about now is:
1. get the search numbers of the exact Gtrend "keyword"
2. see the hottiness of the keyword in a graph
3. date/time of the keyword
4. saving Gtrends sheet for that specific day

You run with, once i play with, ideas will come up automatically

LOVE

**rickstar** · #42 (**permalink**) 12-11-2009, 03:08 PM

Quite interested in this. Have posted the way in which I would use it further up the page.

Great idea.

Thanks,
Eric

**jaan24** · #43 (**permalink**) 12-11-2009, 03:15 PM

Pls count me in Superman. I would love to try this application as I've never tried anything similar ever. Pls add me in your list

**carsonrathi** · #44 (**permalink**) 12-11-2009, 03:26 PM

Wait guys, i have something on the way :-)

It will solve the problem of your waste domains, i have applied for clickbank, mean while you can look my sales letter at: http://gtrendsassassin.com

Also, the above sales page is not final, its been modified as per CB policy for time being.

bpockets · #45 (**permalink**) 12-11-2009, 03:40 PM

Please add me to your list as well, Superman. I'm very interested in trying this out.