how to remove urls dont have .aspx type??scrapebpx

Discussion in 'Black Hat SEO Tools' started by bleach, Jun 10, 2010.

  1. bleach

    bleach Senior Member

    Joined:
    Oct 12, 2008
    Messages:
    962
    Likes Received:
    83
    Location:
    New York
    i want to ask.
    For example i scraped 10k of urls blog engine and i want to remove urls that dont have .aspx type
    What can i do?
     
  2. shylesson

    shylesson Power Member

    Joined:
    Jan 10, 2008
    Messages:
    665
    Likes Received:
    2,090
    Location:
    ‹^›‹(•¿•)›‹^›
    Code:
    http://textmechanic.com/Remove-Lines-of-Text-Containing.html
    in the 'search for text' line, input the .aspx or w/e you want to contain in the urls, paste the list in the box, hit 'remove lines NOT containing' and boom--good to go.
     
  3. BlackOut

    BlackOut Registered Member

    Joined:
    Jan 25, 2008
    Messages:
    55
    Likes Received:
    31
    Gender:
    Male
    You can also use grep. Save the results to file and then grep .aspx file.txt > newfile.txt in Linux or use grep for windows.
     
  4. BlackOut

    BlackOut Registered Member

    Joined:
    Jan 25, 2008
    Messages:
    55
    Likes Received:
    31
    Gender:
    Male
    You can also use grep. Save the results to file and then grep .aspx file.txt > newfile.txt in Linux or use grep for Windows.
     
  5. BlackOut

    BlackOut Registered Member

    Joined:
    Jan 25, 2008
    Messages:
    55
    Likes Received:
    31
    Gender:
    Male
    You can also use grep. Save the results to file and then grep .aspx file.txt > newfile.txt in Linux or use grep for windows.
     
  6. houseaz

    houseaz Regular Member

    Joined:
    Mar 25, 2010
    Messages:
    296
    Likes Received:
    73
    Occupation:
    Making money
    Location:
    Wherever I can Make Money
    Does he have to use Grep 3 times though? Or is once enough?
     
  7. BlackOut

    BlackOut Registered Member

    Joined:
    Jan 25, 2008
    Messages:
    55
    Likes Received:
    31
    Gender:
    Male
    sorry...something went wrong while posting yesterday. maybe someone can delete these other posts.