1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

Scraping search engine results - need extra tool for ending up with domain name only

Discussion in 'Black Hat SEO Tools' started by phirex, Dec 23, 2012.

  1. phirex

    phirex Power Member

    Joined:
    Nov 17, 2009
    Messages:
    514
    Likes Received:
    258
    Hey
    I am scraping search results using an excellent free tool from bottopia
    the thing is i only need the main domain and not the whole URL
    for example i am getting urls like this:
    domain1 . com /blabla/bla.htm
    domain1 . com/blat/bla.html
    domain2 . com/whatever

    i only need
    domain1 . com
    domain2 . com

    and tool that you know that will delete the extra slashes and leave me with the root domain

    after that, i will easily find a tool to delete the duplicates...
    thanks in advance
     
  2. phirex

    phirex Power Member

    Joined:
    Nov 17, 2009
    Messages:
    514
    Likes Received:
    258
    Just in case anyone is looking for the same, and doesnt find
    i found this website
    does exactly what i want
    dncleaner . com
     
  3. phirex

    phirex Power Member

    Joined:
    Nov 17, 2009
    Messages:
    514
    Likes Received:
    258
    Sorry i'm popping this up again
    apparently all of these online cleaners that i found thinks that only .com, .net, .org and .biz are domain names
    stuff like .ru , .ua just gets deleted
    which is annoying
    please give me an alternative if you know any
     
  4. ferma231

    ferma231 Elite Member

    Joined:
    Jul 14, 2011
    Messages:
    1,698
    Likes Received:
    3,508
    Location:
    Internet
    Scrapebox is all you need
     
  5. phirex

    phirex Power Member

    Joined:
    Nov 17, 2009
    Messages:
    514
    Likes Received:
    258
    Scrapebox is much more than i need, and thats why it costs money
    i prefer a simple free tool
    thanks
     
  6. fatboy

    fatboy Elite Member

    Joined:
    Aug 13, 2008
    Messages:
    1,618
    Likes Received:
    3,227
    Occupation:
    Retired
    Location:
    Old Peoples Home
  7. phirex

    phirex Power Member

    Joined:
    Nov 17, 2009
    Messages:
    514
    Likes Received:
    258
    I'm no good at writing php, and not going to get into it for that little thing i need
     
  8. fatboy

    fatboy Elite Member

    Joined:
    Aug 13, 2008
    Messages:
    1,618
    Likes Received:
    3,227
    Occupation:
    Retired
    Location:
    Old Peoples Home
    Did you look at the page? Its got full examples - so what you are saying you can't be bothered to copy and paste some code, make (at a guess) 3 changes to make it loop through a list of URLs but you are happy for other people to feed you free tools to do it?

    Sounds about right for this place lately.
     
    • Thanks Thanks x 1
  9. Z0mbie

    Z0mbie Regular Member

    Joined:
    Jun 24, 2012
    Messages:
    339
    Likes Received:
    151
    Use Excel and the text-to-columns tool in it.
     
  10. phirex

    phirex Power Member

    Joined:
    Nov 17, 2009
    Messages:
    514
    Likes Received:
    258
    Sorry fatboy
    but these 3 changes that seem so easily done to you, might take a few hours to me, and yes, if someone else knows how to do it and wants to contribute a little then they could just do it, for me and for others who might need that.
    I'll be looking over that excel tool, i'm using linux so i have LibreOffice Calc but i bet it has same option...
     
  11. s3v007

    s3v007 Junior Member

    Joined:
    Jun 25, 2012
    Messages:
    104
    Likes Received:
    29
    Location:
    The Internet
    Just keep extracting them like that and then paste it in a Domain Parser. I PM'ed you a link for it.

    I would suggest getting Scrapebox or GSA email spider to make everything easier for you though.

    - s3v
     
    Last edited: Dec 23, 2012