Scraping Y Answers... does it work in big?

cooooookies

Senior Member

Oct 6, 2008

1,133

283

• Apr 6, 2009

• #1

Hi,

I want to use the Y answers api to get Q and A in mass. Does anyone here have experience how many posts I can steal until my Y api id gets banned? Or can you buy Y api ids?

Thanks,
cooooookies

 

detox1978

Junior Member

Jan 16, 2009

110

34

• Apr 6, 2009

• #2

It's not stealling, they created the API for this very purpose.

When I setup scrapper (3 months ago), I read the limit was 5,000 per day. That should be more than enough.

If you really want more, setup additional API ID's - they are free.

 

cooooookies

Senior Member

Oct 6, 2008

1,133

283

• Apr 6, 2009
• Thread Starter Thread Starter

• #3

Thanks detox. I want to build up a content database, so I need many more, but 5k sounds quite good for starting.

 

detox1978

Junior Member

Jan 16, 2009

110

34

• Apr 6, 2009

• #4

That's 5k per day, how many posts did you want?

Can I ask what you have in mind for them ?

 

cooooookies

Senior Member

Oct 6, 2008

1,133

283

• Apr 6, 2009
• Thread Starter Thread Starter

• #5

n00by said:

world domination

Click to expand...

Exactly. Not more and not less.

Actually, my PLR articles - have many of them - are mostly crap and I am looking for better content to fill my autoblogs. Are there legal issues when using those Q/A? My host is based in luxembourg.

 

detox1978

Junior Member

Jan 16, 2009

110

34

• Apr 6, 2009

• #6

You need to read the Yahoo API T&C's, it's really short (one page i think from memory).

All they ask is for a text link "Powered by Yahoo! Answers" to be displayed on your website.

 

jimbobo2779

Elite Member

Jr. VIP

Sep 17, 2008

6,037

3,974

• Apr 6, 2009

• #7

Its not 5k a day its 5k search requests daily and that can add up to, with max of 100 results per query, 500k questions and answers daily. Im sure you could probably deal with this daily limit lol.

Don't forget though that not all searches will return 100 results but 5k requests is enough for any webhost to start complaining about B/W lol. Some of those results can be pretty dang large guys so take that into consideration.

This is from someone who uses Yahoo APIs daily, the Q&A API can return some large XML.

Jim

 

detox1978

Junior Member

Jan 16, 2009

110

34

• Apr 6, 2009

• #8

Yeah, 5,000 requests per day.


If you want to make more than 5,000 requests it's quite easy;

"Yahoo! Web Search web services are limited to 5,000 queries per IP per day per API"

http://developer.yahoo.com/answers/

 

cooooookies

Senior Member

Oct 6, 2008

1,133

283

• Apr 6, 2009
• Thread Starter Thread Starter

• #9

Thanks guys, this is great. I will flood the web