1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

Something to extract URL's from a .txt file

Discussion in 'Black Hat SEO Tools' started by Kronz, Nov 20, 2014.

  1. Kronz

    Kronz Newbie

    Joined:
    Oct 1, 2014
    Messages:
    7
    Likes Received:
    0
    Basically I have a large .txt file that contains various URL's mixed with some text.

    I want to extract those URL's.

    I googled hard, but I couldn't find a free program that does that simple task.

    Please help.
     
  2. mktanny

    mktanny Regular Member

    Joined:
    Oct 22, 2009
    Messages:
    225
    Likes Received:
    62
    Occupation:
    Blog editor and IM
    • Thanks Thanks x 1
  3. scriptdaddy

    scriptdaddy Newbie

    Joined:
    Nov 17, 2014
    Messages:
    1
    Likes Received:
    0
    You can use a regex for that easily, probably in python.
     
  4. todordonev

    todordonev Regular Member

    Joined:
    Nov 23, 2012
    Messages:
    379
    Likes Received:
    228
    Gender:
    Male
    Location:
    Bulgaria
    Home Page:
    Notepad++ and use regex in the find window (ctrl+f)
    You can also google textmechanic and do the following:
    Make new line before http
    Replace empty space with new line (cause the urls are separated with empty space otherwise they will merge and wont work)
    Remove lines that dont contain http
    Remove duplicate lines.
    Save to txt.
     
    • Thanks Thanks x 1
  5. Davidnrog

    Davidnrog Regular Member

    Joined:
    Apr 21, 2014
    Messages:
    219
    Likes Received:
    30
    Occupation:
    Owner, Designer, Entrepreneur, Marketeer..
    Location:
    Besides you
    I'll help you, Just pin me your .txt file..
     
  6. ItsBlinkHere

    ItsBlinkHere Regular Member

    Joined:
    Apr 27, 2014
    Messages:
    409
    Likes Received:
    150
    Location:
    At Large
    Post a sample...it shouldn't be to hard to code up.
     
  7. tornadododo

    tornadododo Newbie

    Joined:
    Nov 20, 2014
    Messages:
    22
    Likes Received:
    4
    get application from the apple store
     
  8. seohosting

    seohosting Newbie

    Joined:
    Nov 19, 2014
    Messages:
    23
    Likes Received:
    8
    Location:
    Gilbert, Arizona
    Home Page:
    If you are using google chrome you can download the "Jon's all in one domain tool" extension. Has a great simple domain cleaner built in.

    You can also find a domain cleaner over at function.me
     
  9. hpasha

    hpasha Jr. VIP Jr. VIP

    Joined:
    May 15, 2011
    Messages:
    1,189
    Likes Received:
    179
    Location:
    Kepler 186F
    put that txt file into scrapebox. it will extract the urls and then check their headers
     
  10. Asif WILSON Khan

    Asif WILSON Khan Executive VIP Premium Member

    Joined:
    Nov 10, 2012
    Messages:
    10,139
    Likes Received:
    28,607
    Gender:
    Male
    Occupation:
    Fun Lovin' Criminal
    Location:
    London
    Home Page:
    • Thanks Thanks x 2
  11. Aty

    Aty Jr. VIP Jr. VIP

    Joined:
    Jan 27, 2011
    Messages:
    5,416
    Likes Received:
    3,701
    Home Page:
  12. Kronz

    Kronz Newbie

    Joined:
    Oct 1, 2014
    Messages:
    7
    Likes Received:
    0
    Thank you very much. It saved me.

    Only problem is it is a bit slow. I have been trying to process a 200MB file today and it hasn't even extracted 1/8 of the links in 5 hours.

    Thanks, but this one crashes.
     
  13. Atomic76

    Atomic76 Registered Member

    Joined:
    May 24, 2014
    Messages:
    67
    Likes Received:
    33
    Is the text file in some sort of consistent format? Such as a comma separated or tab separated? If so, you can just paste it into Excel and do a Data > Text To Columns, and select what the delimiter is.
     
  14. prab1996

    prab1996 Elite Member

    Joined:
    Jan 8, 2013
    Messages:
    3,496
    Likes Received:
    2,028
    Occupation:
    your gf's <3 ♥♥♥♥
    Location:
    Prab1996.com
    Home Page:
    pm i might do it for free if it will not consume much time. i do this type of work every day.
    -=-
     
  15. COLDEXE

    COLDEXE Junior Member

    Joined:
    Aug 29, 2013
    Messages:
    104
    Likes Received:
    24
    Location:
    UK
    I'll do this for free if you like, will only take a few minutes as I've written similar software. I've sent you a pm anyway mate.
     
  16. botrockets

    botrockets Regular Member

    Joined:
    Mar 16, 2013
    Messages:
    272
    Likes Received:
    463
    Occupation:
    Software Developer
    Location:
    Saint T.N.