1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

how to remove urls dont have .aspx type??scrapebpx

Discussion in 'Black Hat SEO Tools' started by bleach, Jun 10, 2010.

  1. bleach

    bleach Senior Member

    Joined:
    Oct 12, 2008
    Messages:
    934
    Likes Received:
    82
    Location:
    New York
    i want to ask.
    For example i scraped 10k of urls blog engine and i want to remove urls that dont have .aspx type
    What can i do?
     
  2. shylesson

    shylesson Power Member

    Joined:
    Jan 10, 2008
    Messages:
    665
    Likes Received:
    2,090
    Location:
    ‹^›‹(•¿•)›‹^›
    Code:
    http://textmechanic.com/Remove-Lines-of-Text-Containing.html
    in the 'search for text' line, input the .aspx or w/e you want to contain in the urls, paste the list in the box, hit 'remove lines NOT containing' and boom--good to go.
     
  3. BlackOut

    BlackOut Registered Member

    Joined:
    Jan 25, 2008
    Messages:
    55
    Likes Received:
    30
    Gender:
    Male
    You can also use grep. Save the results to file and then grep .aspx file.txt > newfile.txt in Linux or use grep for windows.
     
  4. BlackOut

    BlackOut Registered Member

    Joined:
    Jan 25, 2008
    Messages:
    55
    Likes Received:
    30
    Gender:
    Male
    You can also use grep. Save the results to file and then grep .aspx file.txt > newfile.txt in Linux or use grep for Windows.
     
  5. BlackOut

    BlackOut Registered Member

    Joined:
    Jan 25, 2008
    Messages:
    55
    Likes Received:
    30
    Gender:
    Male
    You can also use grep. Save the results to file and then grep .aspx file.txt > newfile.txt in Linux or use grep for windows.
     
  6. houseaz

    houseaz Regular Member

    Joined:
    Mar 25, 2010
    Messages:
    296
    Likes Received:
    73
    Occupation:
    Making money
    Location:
    Wherever I can Make Money
    Does he have to use Grep 3 times though? Or is once enough?
     
  7. BlackOut

    BlackOut Registered Member

    Joined:
    Jan 25, 2008
    Messages:
    55
    Likes Received:
    30
    Gender:
    Male
    sorry...something went wrong while posting yesterday. maybe someone can delete these other posts.