Y T Nuke  
Page 1 of 3 1 2 3 LastLast
Results 1 to 45 of 128
Hey guys, I'm currently building a very basic Google Trends Scraper, which will fetch the ...
  1. #1
    superman100 is offline Junior Member
    Join Date
    Feb 2009
    Posts
    169
    Reputation
    21
    Thanks
    0
    Thanked 386 Times in 37 Posts

    Default Google Trends scraper - Interested?

    Hey guys,

    I'm currently building a very basic Google Trends Scraper, which will fetch the current top 40 most searched terms every hour and store them into a database.

    ( www.google.com/trends )

    This data can then be used to create autoblogs, twitter updates, etc. to quickly gain traffic from the trending topics.

    Is there anyone here that would like to access this kind of data? What kind of features would you like to see?

    Let me know before I create a user friendly version (I'll be free, of course!)

    Post here if you're interested and i'll update things as we go...

    Cheers!
    Last edited by superman100; 12-10-2009 at 02:41 PM.

  2. The Following 28 Users Say Thank You to superman100 For This Useful Post:

    abbygirl (12-20-2009), CClark56 (12-13-2009), comparco (12-12-2009), condenanza (12-11-2009), deepan (04-08-2010), dvdpies (04-08-2010), dw_n_cinti (12-27-2009), eqcacere (12-11-2009), fansfans (03-09-2010), FR33K (05-07-2010), hadly (03-26-2010), haegger (04-03-2010), j0b0123 (12-11-2009), JohnyH (12-12-2009), Keithcrash (12-11-2009), laserblast (02-01-2010), lieren (12-14-2009), mahrouqi (12-11-2009), Mr. Grey (12-16-2009), mr.brightside (02-03-2010), rickstar (12-10-2009), sandycat (12-10-2009), searchguru (12-12-2009), socialmik (12-11-2009), subster (03-26-2010), ToMatriX (12-27-2009), wowzzren (01-21-2010), yeahsure (12-12-2009)

  3. #2
    Ontopic is offline Registered Member
    Join Date
    Aug 2009
    Posts
    72
    Reputation
    10
    Thanks
    3
    Thanked 2 Times in 2 Posts

    Default Re: Google Trends scraper - Interested?

    Yeah, could be interesting.

  4. #3
    thestrangefruit is offline Regular Member
    Join Date
    Jan 2009
    Posts
    205
    Reputation
    10
    Thanks
    64
    Thanked 27 Times in 24 Posts

    Default Re: Google Trends scraper - Interested?

    Def. would be interested.

  5. #4
    superman100 is offline Junior Member
    Join Date
    Feb 2009
    Posts
    169
    Reputation
    21
    Thanks
    0
    Thanked 386 Times in 37 Posts

    Default Re: Google Trends scraper - Interested?

    Alright! How would you use it? You don't need to explain in depth but I need good feedback to create the best service possible!

  6. #5
    otobse is offline Regular Member
    Join Date
    Aug 2009
    Posts
    368
    Reputation
    10
    Thanks
    2
    Thanked 20 Times in 19 Posts

    Default Re: Google Trends scraper - Interested?

    i am also interested in it

  7. #6
    rickstar's Avatar
    rickstar is offline ..is now a consultant
    Join Date
    Dec 2009
    Posts
    786
    Reputation
    103
    Thanks
    496
    Thanked 594 Times in 218 Posts

    Default Re: Google Trends scraper - Interested?

    Could be interesting but what exactly are you going to be scraping? The articles from the top 40 items or?

  8. #7
    superman100 is offline Junior Member
    Join Date
    Feb 2009
    Posts
    169
    Reputation
    21
    Thanks
    0
    Thanked 386 Times in 37 Posts

    Default Re: Google Trends scraper - Interested?

    Quote Originally Posted by redrabbit View Post
    Could be interesting but what exactly are you going to be scraping? The articles from the top 40 items or?
    Lets say I'll scrape the 40 most searched (per hour) terms and save them into a database each hour. Then, every day group them together to retrieve the most popular search term that day.

    I suppose I could retrieve the numbers of results in Google based on the data to determine if there would be a lot of competition in the search engine results for that specific keyword.

    Or, you could just fetch the data from me by rss (or however you guys would prefer.) and create autoblog content.

    Don't know, just thinking loud I suppose.

  9. #8
    carsonrathi's Avatar
    carsonrathi is offline Jr. VIP
    Join Date
    Mar 2008
    Posts
    1,102
    Reputation
    277
    Thanks
    303
    Thanked 726 Times in 216 Posts

    Default Re: Google Trends scraper - Interested?

    This kind of product is already on the way :-)
    Coming Soon <<<

  10. #9
    superman100 is offline Junior Member
    Join Date
    Feb 2009
    Posts
    169
    Reputation
    21
    Thanks
    0
    Thanked 386 Times in 37 Posts

    Default Re: Google Trends scraper - Interested?

    Quote Originally Posted by carsonrathi View Post
    This kind of product is already on the way :-)
    Is it something you're working on? If so, hit me up by PM and tell me more

  11. #10
    superman100 is offline Junior Member
    Join Date
    Feb 2009
    Posts
    169
    Reputation
    21
    Thanks
    0
    Thanked 386 Times in 37 Posts

    Default Re: Google Trends scraper - Interested?

    Quote Originally Posted by NYC View Post
    Yes guys, I have been working on already for a few months now the perfect solution for this service.

    I will start selling the service within a week. The site will be RankMyTrends.com

    This service will do many things, scrape articles based on the trend, autoblog it into blogger.com/any blog you direct it too. It makes the title of the blog the trend name, the tags with most relevant keywords related to the trend.

    Also, in your "Admin" account, you will be allowed to add your Tracking URL from all of your offers in your network so if it is related to that google trend term, it will use that offer automatically in that article. It will also hide it with a hyperlink (also able to forward a domain to the offer).

    These are just a few features of my service, I have been building it and its going to be amazing. I have tested it out and works very well (gets indexed while running and updates each hour as directed in admin control center).

    If you are interested in this service, please press "Thank you" button so I can see how many people I can expect will get a service like this."

    It will not be free since this has taken me months to create, and makes a lot of money when testing out with Clickbank offers.

    Thank you, look forward to seeing some of your responses.

    NYC
    Cool. But, I do however plan to offer the service for free. Freemium is the future of web services and online marketing :-)

    Hit me up by PM if you want to cooperate.

    And all others reading: If you would like a FREE version, post a reply in this thread and let me know what kind of features you would love to see!

    CHEERS

  12. #11
    sandycat's Avatar
    sandycat is offline Jr. VIP
    Join Date
    May 2009
    Posts
    181
    Reputation
    29
    Thanks
    9
    Thanked 60 Times in 32 Posts

    Default Re: Google Trends scraper - Interested?

    Hi Superman,

    I'd like to play with this too! Thanks for sharing

  13. #12
    superman100 is offline Junior Member
    Join Date
    Feb 2009
    Posts
    169
    Reputation
    21
    Thanks
    0
    Thanked 386 Times in 37 Posts

    Default Re: Google Trends scraper - Interested?

    Thanks so far guys - As we speak i've started to collect the data, so I will be able to launch some sort of preview/beta in a few days (it's a bit useless without the data, so i'll have to wait for a few days.)

    As Google Trends updates every 60 minutes and there are 24 hours in a day i'll store about 960 keywords a day - then group them together so you'll get a rundown on a daily basis about the most popular / searched terms for that day.

    Would you prefer a invite only model or public beta / preview?

  14. #13
    axedbydax's Avatar
    axedbydax is online now Regular Member
    Join Date
    Sep 2008
    Posts
    346
    Reputation
    19
    Thanks
    219
    Thanked 48 Times in 43 Posts

    Default Re: Google Trends scraper - Interested?

    this is a very promising service. I hope to get a copy once its done.

  15. #14
    superman100 is offline Junior Member
    Join Date
    Feb 2009
    Posts
    169
    Reputation
    21
    Thanks
    0
    Thanked 386 Times in 37 Posts

    Default Re: Google Trends scraper - Interested?

    axedbydax: You will! Keep following this thread and let me know what kind of features you would like to see and how you will use it and i'll let you guys know once it's ready.

    as I just wrote: already collecting data...

  16. #15
    mejohnny is offline Newbies
    Join Date
    Oct 2009
    Posts
    5
    Reputation
    10
    Thanks
    15
    Thanked 0 Times in 0 Posts

    Default Re: Google Trends scraper - Interested?

    I'm interested. I would suggest invite.

  17. #16
    sapik is offline Newbies
    Join Date
    Nov 2009
    Posts
    16
    Reputation
    10
    Thanks
    2
    Thanked 0 Times in 0 Posts

    Default Re: Google Trends scraper - Interested?

    can help testing maybe also programming?

  18. #17
    Blueprint is offline Junior Member
    Join Date
    Nov 2009
    Posts
    188
    Reputation
    16
    Thanks
    42
    Thanked 50 Times in 26 Posts

    Default Re: Google Trends scraper - Interested?

    THis would be very interesting, and again i'd agree with people on saying it should be by invite....

  19. #18
    Baby Savant's Avatar
    Baby Savant is offline Registered Member
    Join Date
    Jul 2009
    Location
    Mars
    Posts
    74
    Reputation
    23
    Thanks
    76
    Thanked 94 Times in 29 Posts

    Default Re: Google Trends scraper - Interested?

    Hey superman, would love to get a review copy whenever it's done.

    Features I'd love to see:

    - Weekly aggregate of top 100 keywords - 500 keywords
    - Monthly aggregate of top 100 keywords - 500 keywords
    - Personalized graphing capabilities (ie, if you wanted to see a picture representation of the progress of a certain keyword through a certain period of time, etc).

    Similar to Wordtracker's service, this could be used to pick out topic trends - what works all around, what works on a seasonal basis, etc.

    Creating ebooks/courses/packages on popular topics could potentially bring in alot of money.

    In the short-term, this could definitely be used to push traffic to blogs, twitter, etc (again on a very short-term basis).

    Sounds like a cool tool, please definitely keep me updated as to how it turns out.

  20. #19
    faceicles is offline Newbies
    Join Date
    Dec 2009
    Location
    Florida
    Posts
    9
    Reputation
    10
    Thanks
    4
    Thanked 1 Time in 1 Post

    Default Re: Google Trends scraper - Interested?

    Sounds very cool. I would be interested in this!

  21. #20
    goawayplease is offline Regular Member
    Join Date
    Apr 2008
    Posts
    301
    Reputation
    14
    Thanks
    57
    Thanked 65 Times in 38 Posts

    Default Re: Google Trends scraper - Interested?

    How about a hook into Clickbank and CPA searches to show how to monetize the trends as well? Maybe make the graphing, and the monetization premium features.

    You could also create a content scraping module for content spinners, and make that a premium feature too.

  22. #21
    mahrouqi is offline Newbies
    Join Date
    Dec 2008
    Posts
    6
    Reputation
    10
    Thanks
    2
    Thanked 0 Times in 0 Posts

    Default Re: Google Trends scraper - Interested?

    Count me also! I would love to have your service for free. I suggest that you make the submitting of the keywords to the members on an hourly basis instead of daily as google trends do not last for so long and that's why it would be more useful for members to get these keywords on an hourly basis than daily basis.

  23. #22
    RandomName is offline Junior Member
    Join Date
    Apr 2009
    Posts
    105
    Reputation
    14
    Thanks
    94
    Thanked 72 Times in 42 Posts

    Default Re: Google Trends scraper - Interested?

    Have a chat with Nossie (member here @ bhw) as he did something similar with his product NicheShark

    RN

  24. #23
    rickstar's Avatar
    rickstar is offline ..is now a consultant
    Join Date
    Dec 2009
    Posts
    786
    Reputation
    103
    Thanks
    496
    Thanked 594 Times in 218 Posts

    Default Re: Google Trends scraper - Interested?

    Quote Originally Posted by superman100 View Post
    Cool. But, I do however plan to offer the service for free. Freemium is the future of web services and online marketing :-)

    Hit me up by PM if you want to cooperate.

    And all others reading: If you would like a FREE version, post a reply in this thread and let me know what kind of features you would love to see!

    CHEERS
    Now that you've told me what it is you're scraping, I'd definitely like access to a free version.

    It will not only allow me to add new popular keyword variations to findabusiness.org but will allow for all future blog articles to be instantly up there.

    So, count me in!

    Cheers

  25. #24
    ecki is offline Registered Member
    Join Date
    Aug 2007
    Posts
    57
    Reputation
    8
    Thanks
    14
    Thanked 14 Times in 10 Posts

    Default Re: Google Trends scraper - Interested?

    And you all honestly think that google will honor you with loads of traffic for
    scraping their content?!?!
    Complete waste of time and energy!

  26. #25
    ubermann is offline Newbies
    Join Date
    Jul 2008
    Posts
    48
    Reputation
    12
    Thanks
    257
    Thanked 44 Times in 5 Posts

    Default Re: Google Trends scraper - Interested?

    I would like to get in on a free version. Sounds like it would be a handy tool for us who use BHCB.

  27. #26
    jdmeg6's Avatar
    jdmeg6 is offline Junior Member
    Join Date
    Mar 2008
    Location
    Mass
    Posts
    107
    Reputation
    10
    Thanks
    90
    Thanked 14 Times in 7 Posts

    Default Re: Google Trends scraper - Interested?

    this would be a helpful useful tool

  28. #27
    Join Date
    Nov 2009
    Location
    Lafayette, IN, USA
    Age
    20
    Posts
    7
    Reputation
    10
    Thanks
    1
    Thanked 1 Time in 1 Post

    Default Re: Google Trends scraper - Interested?

    I would be very interested in this tool
    If I could try it out, please shoot me a PM

  29. #28
    unnamed's Avatar
    unnamed is offline Junior Member
    Join Date
    May 2008
    Location
    EU shithole
    Posts
    128
    Reputation
    43
    Thanks
    41
    Thanked 37 Times in 25 Posts

    Default Re: Google Trends scraper - Interested?

    Quote Originally Posted by NYC View Post
    Yes guys, I have been working on already for a few months now the perfect solution for this service.

    I will start selling the service within a week. The site will be RankMyTrends.com

    This service will do many things, scrape articles based on the trend, autoblog it into blogger.com/any blog you direct it too. It makes the title of the blog the trend name, the tags with most relevant keywords related to the trend.

    Also, in your "Admin" account, you will be allowed to add your Tracking URL from all of your offers in your network so if it is related to that google trend term, it will use that offer automatically in that article. It will also hide it with a hyperlink (also able to forward a domain to the offer).

    These are just a few features of my service, I have been building it and its going to be amazing. I have tested it out and works very well (gets indexed while running and updates each hour as directed in admin control center).

    If you are interested in this service, please press "Thank you" button so I can see how many people I can expect will get a service like this."

    It will not be free since this has taken me months to create, and makes a lot of money when testing out with Clickbank offers.

    Thank you, look forward to seeing some of your responses.

    NYC
    like your idea,but want to ask if this will be script based selling? or just service like providing generating sites/blogs ?

    im looking for something like this long time , so put me in your list bud.)

    thanks

  30. #29
    Beager is offline Newbies
    Join Date
    Apr 2009
    Posts
    33
    Reputation
    10
    Thanks
    47
    Thanked 53 Times in 16 Posts

    Default Re: Google Trends scraper - Interested?

    Sounds like a nice idea, maybe add ebay labs, or twitter trend searches

  31. #30
    axedbydax's Avatar
    axedbydax is online now Regular Member
    Join Date
    Sep 2008
    Posts
    346
    Reputation
    19
    Thanks
    219
    Thanked 48 Times in 43 Posts

    Default Re: Google Trends scraper - Interested?

    Quote Originally Posted by superman100 View Post
    axedbydax: You will! Keep following this thread and let me know what kind of features you would like to see and how you will use it and i'll let you guys know once it's ready.

    as I just wrote: already collecting data...
    Thanks Superman. It will be great if you have the following features

    take the top most site on a specific trend topic and rewrite to make a post.

    get another 4 to be listed as related items/sites


    If you can do that for at least top 5 trending topic then it will surely be a hit.

    Then maybe if possible grab a video via tube sites to be included in the post.
    Search for related items via amazon to be embedded on any text within the post for extra income (optional)


    Thanks

  32. #31
    DrJekyll's Avatar
    DrJekyll is offline Senior Member
    Join Date
    Aug 2007
    Location
    1-9 for your Keyword goodluck
    Posts
    833
    Reputation
    53
    Thanks
    187
    Thanked 676 Times in 253 Posts

    Default Re: Google Trends scraper - Interested?

    there is already a wso selling something like this.

    will pm you.

  33. #32
    showbizvet's Avatar
    showbizvet is offline Power Member
    Join Date
    Oct 2008
    Location
    Tennessee and around
    Posts
    753
    Reputation
    27
    Thanks
    124
    Thanked 248 Times in 157 Posts

    Default Re: Google Trends scraper - Interested?

    Super thread... both the paid thing from carsonrathi and what Superman offers. The net moves fast and the fact that multiple people are doing/offering/developing shouldn't be considered a "ah, it's too late for me", rather the fact there is a need. I'm sure there might be a WSO, I do know that Niche Horde is out there, doing SOME, but certainly not all of what carsonrathi mentions.
    Custom Video Creation and Traffic Geyser Distribution - 2 Day Delivery

  34. #33
    super-oct is offline Junior Member
    Join Date
    Jul 2009
    Location
    Orange County,714 Area
    Posts
    157
    Reputation
    8
    Thanks
    4
    Thanked 10 Times in 9 Posts

    Default Re: Google Trends scraper - Interested?

    PM Im very interested on this ASAP lol

  35. #34
    superman100 is offline Junior Member
    Join Date
    Feb 2009
    Posts
    169
    Reputation
    21
    Thanks
    0
    Thanked 386 Times in 37 Posts

    Default Re: Google Trends scraper - Interested?

    Hi guys,

    Thanks alot for all the feedback, interested and suggestions.

    Here's what I've figured out so far:

    1. You will be able to access the data on a per-hour basis as google trends updates every 60 minutes, so 5 minutes after their update the data will be accessable.

    2. The trends and keywords will be visualized with graphs and charts to make it easy to see trending topics on a daily, weekly and monthly basis. I don't know exactly what it will look like but it will be there aswell.

    3. The service will be invite-only in the beginning, and i'll probably let other members invite other members to keep things "within the family", so to speak.

    4. I do plan to fetch trends from Twitter as well but it's not my top priority at the moment. Could also fetch from Bing and other search engines but like i just said: Google is the first priority.

    5. The service will be free of charge. You'll probably have to opt-in with your email and create a username and password but besides from that, no strings...

  36. #35
    Keithcrash is offline Newbies
    Join Date
    Dec 2009
    Posts
    0
    Reputation
    10
    Thanks
    5
    Thanked 0 Times in 0 Posts

    Default Re: Google Trends scraper - Interested?

    I think it sounds great, and would be interested myself.

  37. #36
    Giuni's Avatar
    Giuni is offline Regular Member
    Join Date
    May 2009
    Posts
    368
    Reputation
    40
    Thanks
    63
    Thanked 181 Times in 99 Posts

    Default Re: Google Trends scraper - Interested?

    please let the creature out !!
    i will like to work with your tool
    cheers

  38. #37
    superman100 is offline Junior Member
    Join Date
    Feb 2009
    Posts
    169
    Reputation
    21
    Thanks
    0
    Thanked 386 Times in 37 Posts

    Default Re: Google Trends scraper - Interested?

    Thank you keithcrash and Giuni : You're added to the list of people who will be invited once it's ready. I do however need to work a little bit more on the tool before it can be released.

    AND, i really NEED people to tell me how they would like to use the tool so I can create the right features.

  39. #38
    ghettogong is offline Regular Member
    Join Date
    Oct 2009
    Posts
    302
    Reputation
    5
    Thanks
    55
    Thanked 26 Times in 21 Posts

    Default Re: Google Trends scraper - Interested?

    @superman......YES, we need that softie!!!

    It is a very valuable keyword resource.

    Could i have a copy for testing. THX
    Posted via Mobile Device

  40. #39
    makeusrich2009 is offline Newbies
    Join Date
    Apr 2009
    Posts
    4
    Reputation
    10
    Thanks
    0
    Thanked 0 Times in 0 Posts

    Default Re: Google Trends scraper - Interested?

    Def. would be interested. plzz send me the software or script for google trends ??

    keenly waiting for your reply....

  41. #40
    superman100 is offline Junior Member
    Join Date
    Feb 2009
    Posts
    169
    Reputation
    21
    Thanks
    0
    Thanked 386 Times in 37 Posts

    Default Re: Google Trends scraper - Interested?

    I'm very glad that there's a high interest for this. ghettogong and makeusrich2009 : You're added to the list! Please be patient as i'm currently working on a user friendly version.

    Post your interest and request for features and i'll keep you updated..

  42. #41
    ghettogong is offline Regular Member
    Join Date
    Oct 2009
    Posts
    302
    Reputation
    5
    Thanks
    55
    Thanked 26 Times in 21 Posts

    Default Re: Google Trends scraper - Interested?

    Quote Originally Posted by superman100 View Post
    I'm very glad that there's a high interest for this. ghettogong and makeusrich2009 : You're added to the list! Please be patient as i'm currently working on a user friendly version.

    Post your interest and request for features and i'll keep you updated..
    NICE!

    Features what i can think about now is:
    1. get the search numbers of the exact Gtrend "keyword"
    2. see the hottiness of the keyword in a graph
    3. date/time of the keyword
    4. saving Gtrends sheet for that specific day

    You run with, once i play with, ideas will come up automatically

    LOVE

  43. #42
    rickstar's Avatar
    rickstar is offline ..is now a consultant
    Join Date
    Dec 2009
    Posts
    786
    Reputation
    103
    Thanks
    496
    Thanked 594 Times in 218 Posts

    Default Re: Google Trends scraper - Interested?

    Quite interested in this. Have posted the way in which I would use it further up the page.

    Great idea.

    Thanks,
    Eric

  44. #43
    jaan24's Avatar
    jaan24 is offline Jr. VIP
    Join Date
    Nov 2008
    Posts
    586
    Reputation
    81
    Thanks
    165
    Thanked 481 Times in 97 Posts

    Default Re: Google Trends scraper - Interested?

    Pls count me in Superman. I would love to try this application as I've never tried anything similar ever. Pls add me in your list

  45. #44
    carsonrathi's Avatar
    carsonrathi is offline Jr. VIP
    Join Date
    Mar 2008
    Posts
    1,102
    Reputation
    277
    Thanks
    303
    Thanked 726 Times in 216 Posts

    Default Re: Google Trends scraper - Interested?

    Wait guys, i have something on the way :-)

    It will solve the problem of your waste domains, i have applied for clickbank, mean while you can look my sales letter at: http://gtrendsassassin.com

    Also, the above sales page is not final, its been modified as per CB policy for time being.
    Coming Soon <<<

  46. #45
    bpockets's Avatar
    bpockets is offline Regular Member
    Join Date
    May 2009
    Posts
    215
    Reputation
    10
    Thanks
    17
    Thanked 14 Times in 11 Posts

    Default Re: Google Trends scraper - Interested?

    Please add me to your list as well, Superman. I'm very interested in trying this out.

Page 1 of 3 1 2 3 LastLast
Natural Slow Link Building


SEO Blasts - High quality link building service

Tags for this Thread

Bookmarks

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •  
  SEnukeX SEO Software
Proudly Powered by Hostwinds.com Web Hosting Click Here For Exclusive BHW Discounts!

Cheap Web Hosting


1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75