1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

how to grab all destination urls from a big list of bit.ly?

Discussion in 'Black Hat SEO Tools' started by GoogleAlchemist, Feb 10, 2012.

Tags:
  1. GoogleAlchemist

    GoogleAlchemist Regular Member

    Joined:
    Nov 25, 2009
    Messages:
    249
    Likes Received:
    28
    Occupation:
    Bad Ass SEO Consultant
    Location:
    Wherever I want
    Home Page:
    I've got a whole bunch of bitly shortened links. I want to collect all of the nonshortened actual destination urls in a separate file.

    Is there an easy auto way to do this in order to avoid loading them all up in a browser manually and copy pasting them one by one?

    Thanks
     
  2. Gary Becks

    Gary Becks Power Member

    Joined:
    Apr 11, 2010
    Messages:
    675
    Likes Received:
    282
    Location:
    Atl
    Home Page:
    Nothing short of coding a simple app to do it for you, at least not that I know of.
     
  3. Grizzy

    Grizzy Senior Member

    Joined:
    Nov 11, 2008
    Messages:
    919
    Likes Received:
    999
    Expanding bit.ly urls is pretty easy to do with a little bit of php:
    PHP:
    <?php

    $lines 
    file('bitly.txt');

    set_time_limit (1000);

    function 
    expandURL($url)
    {
        
    $retVal 'Error';
        
    $ch curl_init($url);
        
    curl_setopt($chCURLOPT_RETURNTRANSFER1);
        
    curl_setopt($chCURLOPT_FOLLOWLOCATION1);
        if(
    curl_exec($ch) != false)
        {
            
    $response curl_exec($ch);
            if(
    $response != false)
            {
                
    $responseInfo curl_getinfo($ch);
                if(
    $responseInfo['http_code'] == 200)
                {
                    
    $retVal =  $responseInfo['url'];
                }
                else if(
    $responseInfo['http_code'] == 404)
                {
                    
    $retVal =  'URL Not found';
                }
                else
                {
                    
    $retVal =  'HTTP error: '.$responseInfo['http_code'];
                }
            }
            else
            {
                
    $retVal =  curl_error($ch);
            }
        }
        else
        {
            
    $retVal =  'cURL error ocurred : '.curl_error($ch);
        }
        
    curl_close($ch);
        return 
    $retVal;
    }

    foreach (
    $lines as $line) {
        echo 
    expandURL(trim($line)) . "<br />";
    }

    ?>
    bitly.txt is just a file containing one short link per line.
     
    • Thanks Thanks x 3
  4. GoogleAlchemist

    GoogleAlchemist Regular Member

    Joined:
    Nov 25, 2009
    Messages:
    249
    Likes Received:
    28
    Occupation:
    Bad Ass SEO Consultant
    Location:
    Wherever I want
    Home Page:
    brilliant thanks. someone gave me a script for something else that is php based i think, the instructions are in a pm at another forum which is down at the moment so im not sure how to implement this?

    i save it to a text file and need to save it as a vbs extension or something to run it?

    or do i need to upload this as a file to my host ?
     
  5. Grizzy

    Grizzy Senior Member

    Joined:
    Nov 11, 2008
    Messages:
    919
    Likes Received:
    999
    Yea save it to a file with a .php extension and upload it to your host. Upload a bitly.txt and put it the same directory as the php file. Then just browse to that php file, and the bit.ly links in the text file will be expanded one by one and outputted in your browser.
     
    • Thanks Thanks x 2
  6. kokoloko75

    kokoloko75 Elite Member

    Joined:
    Jan 1, 2011
    Messages:
    1,628
    Likes Received:
    1,935
    Occupation:
    Design director
    Location:
    Paris (France)
    Or you can use it in Localhost, with XAMPP for example.
    Don't forget to press Thanks on Grizzy script ;)

    Beny
     
    • Thanks Thanks x 1
  7. sockpuppet

    sockpuppet Junior Member

    Joined:
    Nov 7, 2011
    Messages:
    155
    Likes Received:
    145
    if you are running linux you can use this little bash script
    Code:
    #!/bin/sh
    while read url; do
    	curl -I $url 2>/dev/null | awk 'BEGIN { FS = ": " } { if  ( $1 == "Location" ) { print $2 } }'
    done
    
     
    • Thanks Thanks x 1
  8. om39a

    om39a Regular Member

    Joined:
    May 29, 2011
    Messages:
    212
    Likes Received:
    67
    Location:
    BHW
    Buddy!! You got the entire script!! Wat else you need!! just put it in a host or localhost XAMPP!! Simple!!
    Easy thing will be putting it in your web host. In local XAMPP there will be a problem with curl with default settings. You need to change php.ini files... It will be better to run it in a host!!
     
  9. GoogleAlchemist

    GoogleAlchemist Regular Member

    Joined:
    Nov 25, 2009
    Messages:
    249
    Likes Received:
    28
    Occupation:
    Bad Ass SEO Consultant
    Location:
    Wherever I want
    Home Page:
    thanks all, when i first read the above bit i thought it was going to act like the urlopener site and just open them all up in tabs

    but just ran it and YEAH! PERFECT! JUST WHAT I WAS LOOKING FOR!

    I tell ya straight up, you guys are like digital wizards to me, incredible. 10 seconds vs god knows how many hours and hours of tedious work it would have taken me to accomplish the same thing

    truly appreciate it
     
  10. GoogleAlchemist

    GoogleAlchemist Regular Member

    Joined:
    Nov 25, 2009
    Messages:
    249
    Likes Received:
    28
    Occupation:
    Bad Ass SEO Consultant
    Location:
    Wherever I want
    Home Page:
    I'm doing something wrong here guys, missing something and not sure what

    I have about 600 bitly links I'm trying to get all the destination urls too. i let this run on the whole list, it seemed to work great, got a few errors from urls that were no longer valid no biggy ,but in the end I only got about 25% conversions, no clue what happened to the rest of the urls?

    tried to run only 200 at a time instead of the full 600 and only got 20

    i even tried adding a 0 to the timeout but still same deal

    any ideas? is this something my host would have problems with resource wise or what am i missing?

    thanks