1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

Anyone successfully scraped EzineArticles?

Discussion in 'Blogging' started by trapmuzik, Jun 2, 2009.

  1. trapmuzik

    trapmuzik Junior Member

    Joined:
    Mar 20, 2009
    Messages:
    192
    Likes Received:
    22
    I was going to do a scraper in php when I thought it might be easier to just use yahoo pipes. Well I tried using a pipe and EzineArticles basically blocked the scrape :( lol.. Here is what I got back after I tried to scrape:

    You are browsing E*ineArticles*com faster than a normal human would.

    You may be seeing this page for one of the following reasons:

    • You have performed too many searches in a short period of time.
    • You have requested too many pages in a short period of time.
    • You have exceeded the daily allowable limit of page views.
    • You have used a script or program to scrape content or keywords.
    I didnt even know there was a daily limit on how many articles you could view. anyone else scrape from here?
     
  2. zozor

    zozor Junior Member

    Joined:
    Dec 24, 2008
    Messages:
    113
    Likes Received:
    70
    Yeah. Never got that. Pm me your pipe so so I can fix it
     
  3. ddf1980

    ddf1980 Junior Member

    Joined:
    Aug 19, 2008
    Messages:
    110
    Likes Received:
    51
    lmfao..pause
     
  4. linuxgeek

    linuxgeek Newbie

    Joined:
    Oct 3, 2008
    Messages:
    18
    Likes Received:
    3
    Occupation:
    Systems Administrator
    Location:
    Pennsylvania
    I've gotten errors like that while using wget. I usually just add a delay and change the user agent to one that looks like firefox or ie. I also add a proper referrer on some sites to keep from getting hotlink errors.
     
  5. hertz314

    hertz314 Newbie

    Joined:
    Dec 10, 2008
    Messages:
    33
    Likes Received:
    14
    I'm thinking about selling scraped ezine articles in packs. How much should I charge for 1000 articles?
     
  6. trapmuzik

    trapmuzik Junior Member

    Joined:
    Mar 20, 2009
    Messages:
    192
    Likes Received:
    22
    well i wasnt even finished with it yet. I was just checking the output so I could do the cut from part and ezine blocked me.. I will pm you what i have so far though

    lol..

    yea i was thinking i would just use php and add a delay to the scrape.. but yahoo just made it so much easier to setup.
     
  7. c0ntenth|ef

    c0ntenth|ef Power Member

    Joined:
    May 20, 2009
    Messages:
    788
    Likes Received:
    118
    Location:
    california
    maybe change the timer setting on your scraper if you can, i use everprofit toolbar for that.
     
  8. blackhat+er

    blackhat+er Regular Member

    Joined:
    Feb 19, 2009
    Messages:
    217
    Likes Received:
    150
    The last time I was using my scrapper I kept getting my ip banned.
    I was only scrapping 10 articles per hour but for some reason kept bannin me so ive put that on ice and used other methods to get my content so please let me know what you use as a time delimiter.

    Nice thread highjack you should kick yourself in the nuts
     
  9. kaene

    kaene Junior Member

    Joined:
    Nov 12, 2008
    Messages:
    138
    Likes Received:
    19
    It's very difficult to scrap EzArticles, the only way is using a bunch of proxies and limiting the queries of each one. Another useful thing is to test first whether the articles are in google cache or similar sites, if isn't, then to go EzArticles, this way it will take longer to get banned.
     
  10. affdba1

    affdba1 BANNED BANNED

    Joined:
    Jul 22, 2008
    Messages:
    8
    Likes Received:
    1
    For a while I used their RSS feeds with Yahoo Pipes to get content to my autoblog, but something changed recently and the full feed module doesn't work anymore.
    If you could solve the full feed module problem, the scrape would work again.
     
  11. matrik

    matrik Newbie

    Joined:
    Nov 1, 2009
    Messages:
    13
    Likes Received:
    1
    Yahoo pipes no longer works with Ezin*ear*ticles. They changed the robots.txt to block Yahoo Pipes user agent. Anyone interested can check the robots.txt. However, there are other ways to swiftly scrape'em. :D
     
  12. mtayyabrana

    mtayyabrana Registered Member

    Joined:
    Sep 13, 2009
    Messages:
    91
    Likes Received:
    3
    Occupation:
    Computer related but, But My business in Online bu
    Location:
    Pakistan
    Hey You talked about this scrapping, but I create one article and it is Approved and online Now :D
     
  13. bizhobby

    bizhobby Newbie

    Joined:
    May 22, 2010
    Messages:
    28
    Likes Received:
    9
    Not sure if this has been discussed already but just a crazy idea here...

    What if... :) you... put as the agent 'GoogleBot' we all know sites OBEY Google and don't question it. Would that work?
     
  14. toptips44

    toptips44 Regular Member

    Joined:
    Jul 10, 2009
    Messages:
    249
    Likes Received:
    45
    I just scrape from articlebase. Why not just use them instead ? They are much easier to scrape from as they don't care how many articles you grab.
     
  15. unbeatable

    unbeatable Newbie

    Joined:
    Aug 17, 2010
    Messages:
    13
    Likes Received:
    0
    I still cant do it, Ezine are quite smart, if anyone can do it, let us know!
     
  16. bwh48

    bwh48 BANNED BANNED

    Joined:
    Jun 30, 2007
    Messages:
    56
    Likes Received:
    54
    The easy way to scrape EZA is by using the same technique that WP-Robot uses.
    They scrape their content from Bing & Yahoo.

    Simple search Bing for the keyword you want, using the site:ezinearticles.com qualifier and then scrape the content from the cached page that is associated with each listing.

    It easy, quick and you never, ever have to hit EZA to get their content..

    Hope this helps someone..

    B.
     
    • Thanks Thanks x 6
  17. bizhobby

    bizhobby Newbie

    Joined:
    May 22, 2010
    Messages:
    28
    Likes Received:
    9
    Can anyone offer the scraped content as an archive?
     
  18. srb888

    srb888 Elite Member

    Joined:
    Jul 30, 2008
    Messages:
    3,260
    Likes Received:
    5,067
    Gender:
    Male
    Occupation:
    WebzSurfer
    Location:
    Sun, Mon, Tue, WTF, Sat!!! :)
    I used a method to scrape as many eza articles and as fast as my PC/net resources permitted without getting the message, and I used a Google search trick in combination (which unfortunately Google has now blocked), but I still wouldn't give the combo away as I fear that eza will quickly place specific blocks just like Google did.
     
  19. zelma143

    zelma143 Power Member

    Joined:
    Jun 25, 2010
    Messages:
    571
    Likes Received:
    37
    Occupation:
    PHP programmer,Bot maker,iMacro script maker
    did you try curl with php?
     
  20. surferket

    surferket Junior Member

    Joined:
    Dec 5, 2008
    Messages:
    179
    Likes Received:
    116
    I use Carty's Autoblogging Software to scrape a full feed thru ezine's own RSS feed.