1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

Collect Post URLs?

Discussion in 'Black Hat SEO Tools' started by Italian Hawk, Aug 1, 2010.

  1. Italian Hawk

    Italian Hawk Power Member

    Joined:
    Apr 21, 2009
    Messages:
    589
    Likes Received:
    126
    Is there any software out there (free/trial versions preferred, but feel free to list ones that cost money), that will collect all the post urls of a blog and leave out categories, tag pages, and other pages that aren't actual posts?

    Edit: I've been using scrapebox with the site:url but it includes posts, tags, categories, author pages, pages, and a ton of stuff I don't want and I have to go through deleting them manually. If it were for a small one or two blogs this would be fine, but I've got a ton that I need to do.
     
  2. crazyflx

    crazyflx Elite Member

    Joined:
    Nov 9, 2009
    Messages:
    1,674
    Likes Received:
    4,825
    Location:
    http://CRAZYFLX.COM
    Home Page:
    You actually already have the best tool for the job, you just need a few more operators in your search.

    If it is a blog with the date in the post pages (www.blog.com/2010/06/blog-title) then this will work for collection ONLY blog post URLs:

    site:blog.com AND /*/*/

    (the AND must be capitalized) You see, google will treat the asterisks as wildcards and the use of an all uppercase AND means that google can only return results to you that meet both of your requirements.

    If the blogs in question don't use dates in their permalink structure, just find the common denominator of the URLs that you don't want and put them after the operator -inurl:thing you don't want returned

    Example: site:blog.com -inurl:tag -inurl:page -inurl:wp-content -inurl:rss -inurl:feed -inurl:category

    See what I'm saying?

    Google "boolean operators" (without quotes) to learn some more interesting things you can make Google do for you when searching.
     
    • Thanks Thanks x 2
    Last edited: Aug 1, 2010
  3. UK2011

    UK2011 Newbie

    Joined:
    Jul 31, 2010
    Messages:
    15
    Likes Received:
    1
    thanks for the post
     
  4. Italian Hawk

    Italian Hawk Power Member

    Joined:
    Apr 21, 2009
    Messages:
    589
    Likes Received:
    126
    Thanks, that will save me a ton of time! I'll try it out.

    And I thought I was good with Google and search engines :rolleyes: