I need to extract URLs from a piece of code, but how?

X

xthoms

Regular Member
Joined
Sep 14, 2010
Messages
280
Reaction score
100
Let's say I have some code like this

Code: 
<a target="_blank" href="http://link.com">
abc</a> </p>
But I of course have a lot more. How can I automatically extract the URLs?
 
Advertise on BHW
I hope this will help you

http ://social.msdn.microsoft.com/Forums/en/sharepointdevelopment/thread/7381457b-2538-46c5-9ff4-a82bf8a75dd9

http ://bytes.com/topic/php/answers/824906-extract-url-rss-feed-source-code-regex
 
Read the following text on regex, it will cover 99% of the regex you will use

Code: 
Assuming you're using PHP:
I suck at regex, but a good trick is to just learn this, it's almost all you ever need anyway:
Code:

preg_match("/abc(.*?)/","abcdefg",$match); $output = $match[1];
echo $output;
//Will return defg

(.*?) will, for the most part, work for everything.
I just put the variable assignment on the same line.

If you want to be more specific with caps etc add Uism:
Code:

preg_match("/abc(.*?)/Uism","abcdefg",$match); $output = $match[1];
echo $output;
//Will return defg

Also, let's say you only want it to return "de":
Code:

preg_match("/abc(.*?)fg/Uism","abcdefg",$match); $output = $match[1];
echo $output;
//Will return de

If you want to work with preg_match_all instead, I suggest you debug with print_r:
Code:

preg_match_all("/abc(.*?)fg/Uism","abcdefgabcdefg",$match);
print_r($match);
 
save the code as a text file and open the text file in a spreadsheet. Use find and replace to replace *href="
leave the replace box blank
Then do it again with ">*

That should leave you with a clean list of just the URLs.
 
Thanks for all the answers but unfortunately none of them can do what i really want.

I have a big code and I want to return all values that are put in between

href=" and ">

I don't know if the regex is doing that, but couldn't figure out how to adapt it to my situation
 
Look for 'URL Extractor Ver 1.0' from focal media.
Url Extractor has been created to extract URL addresses from web pages. It can fetch all valid URL's any HTML file and generate an output file with formatted Url's without duplicates. You are also able to filter Url's to prevent addresses (you do not wish to be included in your list) to be added.
Click to expand...
If it doesn't work, find something else, the internet is full with such free tools. Just make sure they are actual apps, not some virus.

Alternatively, get someone on the freelancer sites to make you a custom one for $2-3 or something, it's a couple of code lines.
 
There is a very good product called 'Web Data Extractor'. It can extract not only URLs, but many other things like metadata, phone numbers etc. You can customize it to suit your needs.

You can download the trial free at

Code: 
http://www.webextractor.com/download.htm
 
Here is a quick PHP script that seems to do what you are after (I think) - download it from http://shrinkr.info/1tv1mq (my webspace).

In the grab.php file, just change put the URL you want to grab links from.

edit: Forgot to say, just run it from the command line - works on my Ubuntu laptop, haven't tested on windows.
 
Last edited:
thread bookmarked. thanks all for providing some utilities.

Was looking for something like this too..
 
I haven't tested this but it should work so long as each link is in the exact format you mentioned with one link per line:
PHP: 
<?php
$urllist=file("yourfile.txt");
foreach($urllist as $urlline){
 $array=explode('"',$urlline);
 echo $array[3]."\n";
}
?>
then just view source and copy/paste
 
Download EmailSpider Gold 10, upload the file to ur server and extract the emails from there...:D
 
I got it all figured out. Thanks for everyone's contribution.
It's not 1 per line, it's the code from a website, but I got it now :)
 
dvdvids said:
Download EmailSpider Gold 10, upload the file to ur server and extract the emails from there...:D
Click to expand...

Will try it and reply. If i have some doubt will post here for sure to get help from you guys..
 
Advertise on BHW
You must log in or register to reply here.
Sign up now!

Main Menu

Home Main Forum List Black Hat SEO White Hat SEO BHW Newbie Guide Blogging Black Hat Tools Social Media Downloads

Marketplace

Account Selling Content / Copywriting EBooks / Methods Hosting Misc Proxies For Sale SEO - Link building SEO - Packages Social Media Social Media - Panels Web Design More

Making Money

Affiliate Programs Hire a Freelancer IM Journeys Making Money Pay Per Click Site Flipping Want to Buy

BlackHat World

Dispute Resolution Forum Suggestions & Feedback Introductions Lounge
Back
Top