1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

Pulling list of URL's from search

Discussion in 'Blogging' started by dynamicvb, Jan 2, 2008.

  1. dynamicvb

    dynamicvb Regular Member

    Sep 2, 2007
    Likes Received:
    I'm knew around here and really to the whole idea of cheating a little, but its something I now need to do.

    I'm looking for a method or software that will allow me to search for and pull out a list of Blog post URLs and save to a text file. I only need it to pull out 20 or 30 thousand of these. Can someone give me a clue and point me in the right direction. I've been looking at all my current software and the web but have not had any revelations yet.

  2. isotropicspin

    isotropicspin Newbie

    Oct 10, 2007
    Likes Received:
    if you do some research into Spidering (there are books out there) I am sure you could create a spider for .xml files or RSS that just pulls the post URLS. In fact this would be close to the "Hello World" application of spidering. You could even use Perl to do this.