How to get full content with RSS feeds

B

back2black

Junior Member
Joined
Dec 20, 2008
Messages
118
Reaction score
29
Hi Guys,

Anyone know how to get the full content when you make wordpress posts from RSS feeds?

Im using autoblogged at the moment - Ive tried changing the tags (i.e. %post% %description% %content% etc) but the actual feed doesnt have the full content... is there a workaround for this? (fyi my rss is from bbc.c0.uk)

Cheers all!
 
Advertise on BHW
If you're pulling data from another site's RSS feed, you're limited by what their feed provides. If it doesn't give the full article, then you can't force it to do so.
 
Actually, I saw a script that puled whole content once. It was online then to test, but if I remember good, he pulled it back and retired.
 
If it was pulling full content, then it was probably using the RSS feed to get the URL of the article so that it could scrape the content from there. If the feed's creator chooses not to display the full content (by structuring the RSS feed, which is an XML file, in such a way that only part of the article is given), then you won't be able to retrieve this information from the XML file itself.
 
you would need to have/write a scraper, but its not that easy.

You get the URL of the feed, then follow the URL and get to the actual page. Now you need to parse the page (and every page could have a different way how its built) to extract the actual content.

the difficulty is that one scraper might work with one feed/source...but not with the other because you always need to know the patterns, where and what to extract to get the actual content.
 
almir said:
Actually, I saw a script that puled whole content once. It was online then to test, but if I remember good, he pulled it back and retired.
Click to expand...

I have a copy of that script. But it too powerful to be released anywhere.

Plus that would hurt my earnings.
 
If you want to have full content of a rss feed this is easily doable with yahoo pipes.
If you want to have full content of blogsearch,technorati etc this is impossible
 
if u are using only bbc.c0.uk rss feeds, then u could write a scraper script. but this may not work on any other feed sources...
 
If it's not in the RSS XML feed in the first place, there's nothing (including Yahoo Pipes) that's going to put it there. However, the individual URL links in the XML feed take you back to the original article which is where software can help.
 
Freller sorry but you are wrong
Heres how I do with yahoo pipes :

Fetch feed url :
For each item in feed
Fetch item.link
Cut content from "" to ""
Assign to item.description
Feed output

and voila you got the full feed. Take 5 minutes work with AC,ezinearticles and any blogs
 
ContentSolution.com scrapes from EzineArticles.com and a few other article sites and exports those articles in .TXT .HTML or RSS.
 
zozor said:
Freller sorry but you are wrong
Heres how I do with yahoo pipes :

Fetch feed url :
For each item in feed
Fetch item.link
Cut content from "" to ""
Assign to item.description
Feed output

and voila you got the full feed. Take 5 minutes work with AC,ezinearticles and any blogs
Click to expand...

What freller and others, including myself, have said is correct. Yahoo! Pipes can get the full feed, but it doesn't have some magical way of extracting data out of thin air. It finds an entry's URL in the site's feed, and then it goes to that URL and grabs the full content. Here's a brief explanation of how Yahoo! Pipes pulls the full feed:

Code: 
Fetch Full Feed
Replaces abbreviated description with full content fetched from webpage using link entity. Uses regex to strip out just the meaningful content.

Source: http://pipes.yahoo.com/feedjournal/fetchfullfeed

In other words, Pipes is a scraper, as are Dapper and other applications that claim to get the full content from a feed that offers just the item descriptions or truncated articles.
 
Guys, thank you so much for your help... honestly this community is great....Im going to check out pipes now, sounds like a good content solution for autoblogging
 
well i know about Y pipes and dapper, but arent all those banned w/ Google? Eg scraping Google news and similar...
 
blackhaze said:
well i know about Y pipes and dapper, but arent all those banned w/ Google? Eg scraping Google news and similar...
Click to expand...

Yeah, G was/is blocking Pipes from scraping G news:

Code: 
http://www.google.com/#hl=en&q=google+blocking+pipes&btnG=Google+Search&aq=f&oq=google+blocking+pipes&fp=3WTwdsC3GPc
 
well i tend more and more in the direction that IF you scrape you should scrape content which is NOT indexed.

So scraping google is about the worst thing you can do.

There ARE ways to get content which is not indexed if you have sources with very OLD content....i am also looking into the option to scrape Google Usenet/Groups archive. I dont think that messages from 1985 are still in their active index?

The other idea is actually using something like yahoo groups or forums and scrape content as soon as it appears in the forum before SE even had a chance to index it.

Yahoo mailing lists/groups is also very good since the content never makes it into google index.
 
fresh content is where it's at. with more and more places getting into ghostwriting services, the price per article is dropping lower and lower.
 
doug said:
fresh content is where it's at. with more and more places getting into ghostwriting services, the price per article is dropping lower and lower.
Click to expand...

Ah! Now - this is interesting - Ive just setup a blog the complete opposite way Im used to - Im doing it a whitehat way... Ive been paying for original content and Im going to try and really make a go of it for 2 reasons:

1) It will be easier to rank in google as its not copied content
2) With half decent content I hope to build up a community who actually want to come back to my site to hear what I have to say.
2) Its easier on my conscience! :D

(well, this is very wh for a bh forum I guess!)

Im paying $3 per article - do you think this is reasonable - how much are you paying?
 
Advertise on BHW
You must log in or register to reply here.
Sign up now!

Main Menu

Home Main Forum List Black Hat SEO White Hat SEO BHW Newbie Guide Blogging Black Hat Tools Social Media Downloads

Marketplace

Account Selling Content / Copywriting EBooks / Methods Hosting Misc Proxies For Sale SEO - Link building SEO - Packages Social Media Social Media - Panels Web Design More

Making Money

Affiliate Programs Hire a Freelancer IM Journeys Making Money Pay Per Click Site Flipping Want to Buy

BlackHat World

Dispute Resolution Forum Suggestions & Feedback Introductions Lounge
Back
Top