1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

Who needs Content? EVERYONE. I'll scrape any site for FREE

Discussion in 'Cloaking and Content Generators' started by admhat, Jul 1, 2010.

  1. admhat

    admhat Newbie

    Joined:
    Jun 13, 2010
    Messages:
    48
    Likes Received:
    12

    Who needs Content?...
    EVERYONE


    This is legit. I will scrape *almost* any site for content for the low price of FREE.

    Yes free. I am in the process of creating a program that makes running your own naughty google spider on crack as easy as 1-2-3! I need to run some test to make sure this app rocks before releasing it. You also probably need content. Why not satisfy each others needs? (not like that... man not like that...)

    Why? you ask...
    • Content research
    • Rich content autoblogs
    • Autosites, no longer is automated content limited to blogs
    • Clone any website's content
    • Advance/deep URL harvesting
    • Much more... use your imagination

    I'm also try to gauge the need for an app or services like this. I recently did a test and scraped a bunch of lyrics websites to create my own lyrics site. I managed to scrape over a million songs and lyrics and exported them to SQL and as Emril would say... BAM content. Just send message me with details on what you want to scrape and i'll get to it. Remember this is free so I do have the right to reject any request.


    -adm

    Note: What you do with the data is completely up to you. By contacting me you are releasing any legal liabilities from me. Blah blah blah... etc.
     
    • Thanks Thanks x 3
    Last edited: Jul 1, 2010
  2. iason

    iason Newbie

    Joined:
    Jan 4, 2010
    Messages:
    1
    Likes Received:
    0
    Hi,

    especially for people who are not native english speakers the part of content creation is one of the biggest obstacles to overcome. In my opinion, there exists great demand from this target group alone. I'm certainly very interested in this service.

    At the moment I need content for two niches: watch brands (especially mens watches) and public records (especially california public records, ca death records, ca marriage records ...).

    Can you help me scraping some content or do you need more detailed information?

    Thanks
    iason
     
  3. admhat

    admhat Newbie

    Joined:
    Jun 13, 2010
    Messages:
    48
    Likes Received:
    12
    I sure can. Just message me some information such as the URLs to the source of content and what you want from each page. Though, for now i'd like to keep it to one site/domain per person, so choose which niche you wana focus on wisely.
     
  4. Brutus69

    Brutus69 Newbie

    Joined:
    Jun 24, 2010
    Messages:
    38
    Likes Received:
    26
    Occupation:
    DB Develoer
    Location:
    US
    First of all... Sweet! As a developer myself I love success stories like yours. So many think grabbing content is all about getting it from on place and pushing it to another. If you put it into a DB first... THAT'S when the magic happens... you can dynamically do anything with it at that point and it becomes uniquely yours. :)

    There's Definetly a need for your service! Although I'm sure there are some who wished you'd keep it to yourself.

    Can your tool follow links from one URL and grab the content from associated links on the page? Or does it only grab data from the single URL you give it?

    I'd like info about a TV Show.

    If your tool follows links to grab more content, use this URL:

    imdb.com/title/tt1327801/fullcredits

    If your tool only grabs data from the specific URL (ie doesn't follow the links), use this URL:

    en.wikipedia.org/wiki/Characters_of_Glee


    My 13 year old daughter is doing a project and I'd love to help her out. :)

    Congrats again on your success!

    Mike
    Posted via Mobile Device
     
  5. admhat

    admhat Newbie

    Joined:
    Jun 13, 2010
    Messages:
    48
    Likes Received:
    12

    Yes my tool can follow urls on a page. It takes in a set of rules to tell the scraper it can or can't follow urls on a page and if it can which ones are allowed and denied.

    For example, I can point the scraper to youtube.com and give it the rule "allow /watch?v=.+&?" (i.imgur.com/RqxrD.png - copy paste) and it will crawl all of youtube and scrape what ever you want that is on the video page.

    What exactly do you want from imdb?
     
  6. daydie

    daydie Junior Member

    Joined:
    Jan 13, 2010
    Messages:
    123
    Likes Received:
    5
    he scraping urls for me as we speak :) very kind, also awaiting for the links. i will post again on update what i recive and details. :)

    cheers again buddy
     
  7. ultimategamechain

    ultimategamechain Power Member

    Joined:
    Apr 22, 2010
    Messages:
    527
    Likes Received:
    76
    Location:
    Here!
    Home Page:
    this is good and free. I will try it.
     
  8. dwpg002

    dwpg002 Senior Member

    Joined:
    Dec 29, 2008
    Messages:
    916
    Likes Received:
    47
    How you will post your content to our blogs
     
  9. Brutus69

    Brutus69 Newbie

    Joined:
    Jun 24, 2010
    Messages:
    38
    Likes Received:
    26
    Occupation:
    DB Develoer
    Location:
    US
    My daughter is putting together a Glee "magazine" with episode summaries and character summaries. IMDB looked like a URL that would like to show, episode and character summaries. But even though your tool follows links still might be hard to program for all summary data. And I don't want this free offer to be difficult for you.

    Can you just extract the character summaries from the wikipedia link above? If that's simple let's go with that. Whatever is easiest for you.

    Does your tool automatically put the data into a DB or comma-delimited flat file format? If so that would be best for me.

    Honestly I'm just looking for a sample of extracted data so I can compare website with extracted data and how easy it is for me to get into a DB. If that data sample can be Glee episode or character summaries all the better.

    I've built some RSS to DB code... but it's limited to feeds of data. I'm also working with some APIs but you've got to be careful not to get too agressive. If you can just scrape data from almost any site then I may need to hire you rather than trying to write this code myself cause I could see it getting very complex to code and support.

    Thanks!

    Mike
    Posted via Mobile Device
     
  10. admhat

    admhat Newbie

    Joined:
    Jun 13, 2010
    Messages:
    48
    Likes Received:
    12
    The tool has support for multiple export formats. SQL and CSV is done. I'm planning on support JSON, XML, Excel, and also a direct to blog export tool that will allow you to publish directly to popular blog engines. I could also implement a direct to db exporter too. You would put in host, username, and password. This would heavily reduce strain on the machine running the app.

    I can export to CSV. It selects content on the page using a mix of CSS selectors and regular expressions. Though i'm partially confused. Looking at the wikipedia page, all of the character summaries are on one page. In your case wouldn't it be easier to just copy and paste them?
     
  11. Brutus69

    Brutus69 Newbie

    Joined:
    Jun 24, 2010
    Messages:
    38
    Likes Received:
    26
    Occupation:
    DB Develoer
    Location:
    US
     
  12. Brutus69

    Brutus69 Newbie

    Joined:
    Jun 24, 2010
    Messages:
    38
    Likes Received:
    26
    Occupation:
    DB Develoer
    Location:
    US
    Posting via phone is a witch (at least for me).

    "I can program" hangin around at the bottom of my post... Duh... I can program but I can't type on this here smart phone thingy... Duh

    what a goober

    Mike
    Posted via Mobile Device
     
  13. headliner

    headliner Registered Member

    Joined:
    Oct 2, 2009
    Messages:
    51
    Likes Received:
    5
    I remember coding up a scraper in PHP that worked fairly well. Let me see if I can dig it up.
     
  14. admhat

    admhat Newbie

    Joined:
    Jun 13, 2010
    Messages:
    48
    Likes Received:
    12
    Ah I see. I would post the lyrics as an example but that is a whopping 10GB file and tbh, I don't really even like the thought of trying to open it haha.

    What I did do for you is grab a bunch of youtube videos and meta related to Glee.

    Code:
    yousendit.com/download/K0JUZGVUaytrUmswTVE9PQ
    and the saved script that ends up being made by my program looks like...

    Code:
    pastebin.com/RRHSxcQw
     
    • Thanks Thanks x 1
  15. daydie

    daydie Junior Member

    Joined:
    Jan 13, 2010
    Messages:
    123
    Likes Received:
    5
    cant wait to get all the links im asking for, if he can get all of them.....

    HES A GOD!
     
  16. Brutus69

    Brutus69 Newbie

    Joined:
    Jun 24, 2010
    Messages:
    38
    Likes Received:
    26
    Occupation:
    DB Develoer
    Location:
    US
    Perfect!

    I'll have to download/review this when I'm back home at my laptop. I'll check it in a few hours and post back.

    Thanks!

    Mike
    P.S. If there's anything I can do for you let me know.
    Posted via Mobile Device
     
  17. Brutus69

    Brutus69 Newbie

    Joined:
    Jun 24, 2010
    Messages:
    38
    Likes Received:
    26
    Occupation:
    DB Develoer
    Location:
    US

    Not sure HOW your system does what it does but... WOW!... I would definitely pay for a service like this.

    FYI for those who don't want to take the time to download the yousendit download file... it contains 900+ records from YouTube with content like URL, Description, Tags, etc.

    With a service that outputs data like this as quickly as you did I'm now sure I will not be spending the time to code something like this myself.

    Thanks for the sample please let me know how much you would charge for a service like this. If you PM me I won't be able to PM you back until I reach 15 posts.

    Great Job and thanks again!

    Mike
     
    • Thanks Thanks x 1
  18. GreyWolf

    GreyWolf Executive VIP Jr. VIP

    Joined:
    Aug 17, 2009
    Messages:
    1,930
    Likes Received:
    5,387
    Gender:
    Male
    Occupation:
    Artist / Craftsman
    Location:
    sitting at my PC
    Can you scrape complete articles and provide them in a usable format?
    for example scrape a newsite for articles relating to a particular niche and then provide me the date, title, author, sources, and content of article?

    This is probably a no-brainer type question, but I just want to be sure about what I can get. LOL
     
  19. admhat

    admhat Newbie

    Joined:
    Jun 13, 2010
    Messages:
    48
    Likes Received:
    12

    Yes! ...but you must provide a source site.
     
  20. kboxer7

    kboxer7 Jr. VIP Jr. VIP Premium Member

    Joined:
    Jan 25, 2010
    Messages:
    872
    Likes Received:
    700
    Freaking Awesome!!!! I would def be interested in this service. Trying to find a good site to have you check out. I'll be back soon!