Web Scrapper, how to get any content from the web and publish it on wordpress/joomla?

Discussion in 'Blogging' started by mraz, Jul 6, 2012.

  1. mraz

    mraz Registered Member

    Aug 10, 2008
    Likes Received:
    I needed something better than Wprobot since Wprobot is set on predefined plugins and these sources are sometimes not enough to grab contents from. What we need is something like a webcrawler that grab contents (text, video, pictures) from anything on the web based on specified keywords and then publish them into wordpress/joomla. The downside with Wprobot is currently its only able to grab content from rss and other existing plugins which most of the time only contains intro text or summary text, not the full text, let alone grabbing images, it simply can't do that. Wprobot is however good for grabbing products from amazon and etc.

    Any ideas of such script and whether its actually possible to scrape any contents based on our keywords from anything on the web and then extract and publish these contents on our web?

    What script does those content aggregators like flipboard, pulse, google currents and etc are using anyways?