tool to extract urls from websites needed

Claudiu

Regular Member
Joined
Feb 28, 2009
Messages
374
Reaction score
129
hi,

looking for a tool that can generate a list of urls from a website.
pretty much like a sitemap tool.

the issue with sitemap tools is that it checks if the connection with the pages is good. i don't like this as it takes to much time, which i don't have cause of my big list of websites to crawl.

so, a tool that can check for urls only (and not the connectivity) is what i'm interested in.

i'd appreciate your input on this if you know of a tool that can help me.
 
Download "link gopher" plugin for firefox. Simply click "extract all links" and it gives you a list of the urls :)

EDIT: oh, you want a spider .... this wont do that, just the current page..
 
HTML:
http://www.xml-sitemaps.com/

Up to 500 pages for free. You could also use scrape box with site:domain.com
 
The best tool I know for this is "Web Data Extractor".
 
i dont know any soft, but if you have still problem - pm me - i can make for you script/bot

or type in google 'sitemap creator'
 
Back
Top