freeufcdotinfo
Power Member
- Jun 12, 2008
- 685
- 157
Is this possible other than doing it manual - need every URL from domain to a txt file.
Need ALL WP URL to txt file
- Thread starter freeufcdotinfo
- Start date
PlanetSEO
BANNED
- Dec 20, 2010
- 279
- 403
Im not sure its what you looking for but you can try this one :
Also just google it there is tons of results
And im not that familiar with Scrapebox although i have it just not using it to much but i know you can do it with that.
Code:
http://wordpress.org/extend/plugins/wp-web-scrapper/Also just google it there is tons of results
And im not that familiar with Scrapebox although i have it just not using it to much but i know you can do it with that.
freeufcdotinfo
Power Member
- Jun 12, 2008
- 685
- 157
Thanks but I dont think this will do what I want.
I have several wp blogs - each with 1000's of posts
each post has url such as
domain.com/blahblah.htm
need the urls exported to txt file
I have several wp blogs - each with 1000's of posts
each post has url such as
domain.com/blahblah.htm
need the urls exported to txt file
PlanetSEO
BANNED
- Dec 20, 2010
- 279
- 403
So in this case i think Scrapebox is your best answer, they have a great url scraper/harvester.
Although i cant really tell you have to do that since im not using this tool
Good luck
crazyflx
Elite Member
- Nov 9, 2009
- 1,623
- 4,870
ScrapeBox has a sitemap scraper.
If your WP blog doesn't already have a sitemap, simply install this plugin (it's free):
http://wordpress.org/extend/plugins/google-sitemap-generator/
It takes about two seconds, and will build a sitemap for your site which lists every URL your WP site has.
Then use the SB "sitemap scraper" addon to scrape the sitemap, which it will then export every URL it scraped into a .txt file.
All in all, this will take less than 3 minutes, from start to finish.
If your WP blog doesn't already have a sitemap, simply install this plugin (it's free):
http://wordpress.org/extend/plugins/google-sitemap-generator/
It takes about two seconds, and will build a sitemap for your site which lists every URL your WP site has.
Then use the SB "sitemap scraper" addon to scrape the sitemap, which it will then export every URL it scraped into a .txt file.
All in all, this will take less than 3 minutes, from start to finish.
Autumn
Elite Member
- Nov 18, 2010
- 2,197
- 3,073
Here is a simple PHP script for people who don't have / don't want to use Scrapebox.
1) Install the xml sitemaps plugin as above ( http://wordpress.org/extend/plugins/google-sitemap-generator/ )
2) Upload this script to one of your domains.
3) Load the script in your browser and enter the url to your sitemap. The script will fetch the sitemap and display a plain text list of all the urls.
1) Install the xml sitemaps plugin as above ( http://wordpress.org/extend/plugins/google-sitemap-generator/ )
2) Upload this script to one of your domains.
3) Load the script in your browser and enter the url to your sitemap. The script will fetch the sitemap and display a plain text list of all the urls.
PHP:
<?php
// Sitemap url extractor
// Upload this to your hosting to extract plain text urls from your sitemap
// You must install the xml sitemaps plugin to generate the sitemap for your blog
// Then enter the sitemap url in the form and press submit button
// pwned by Autumn @ BHW
// Fetch sitemap and extract urls
if($_POST['save'] == 'yes') {
// Fetch the sitemap
$sitemap_url = $_POST['sitemap_url'];
$html = file_get_contents($sitemap_url) or die("ERROR: Could not fetch sitemap");
//echo "$html\n";
// Strip out urls
preg_match_all('/<loc>([^<]*)<\/loc>/imsU', $html, $regs);
// Print urls if found
if($regs) {
//echo "<pre>";
//print_r($regs);
//echo "</pre>";
foreach($regs[0] as $url) {
echo "$url<br>\n";
}
} else {
// Otherwise print error message and original xml
echo "ERROR: No urls found<br>\n";
echo "Here is your original xml:<br>\n";
// Convert angle brackets for each display
$html = preg_replace('/</', '<', $html);
$html = preg_replace('/>/', '>', $html);
// Echo original xml
echo "<pre>\n";
echo "$html\n";
echo "</pre>\n";
}
exit;
}
// Print form to enter the sitemap url
echo '<html>
<head>
<title>Sitemap to url extractor</title>
</head>
<body>
Enter your sitemap url eg. http://www.yourdomain.com/sitemap.xml :<br>
<form action="'. $_SERVER['PHP_SELF'] .'" method="post">
<input type="hidden" name="save" value="yes">
<input type="text" name="sitemap_url" size="80">
<input type="submit" name="submit" value="Extract urls from sitemap">
</form>
<br>
</body>
</html>';
?>crazyflx
Elite Member
- Nov 9, 2009
- 1,623
- 4,870
Rename to .php and upload the attached file to your wp-content folder so that you can access it like yousite.com/wp-content/postlinks.php
It will then give you the page urls, page titles, page tags of ALL the pages of your blog.
You can then save it as txt on your computer and import it into excel for further filteration if you like.
I hope this helps.
It will then give you the page urls, page titles, page tags of ALL the pages of your blog.
You can then save it as txt on your computer and import it into excel for further filteration if you like.
I hope this helps.
Usman Nasir
Registered Member
- Sep 3, 2010
- 68
- 6
Im not sure its what you looking for but you can try this one :
Also just google it there is tons of resultsCode:http://wordpress.org/extend/plugins/wp-web-scrapper/
And im not that familiar with Scrapebox although i have it just not using it to much but i know you can do it with that.
this thing helped me as well