1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

BIG Problem with ScrapeBox and remove duplicate domains

Discussion in 'Black Hat SEO Tools' started by blackhaze, Mar 16, 2011.

  1. blackhaze

    blackhaze Power Member

    Joined:
    Jan 11, 2008
    Messages:
    661
    Likes Received:
    167
    Occupation:
    self made millionaire
    Location:
    in the matrix
    Home Page:
    So i have a bunch of lists, and they contain many URLs with blog pages, eg.

    blog .com/post1.html
    blog .com/post2.html
    blog .com/post3.html (<- ALL the same domain here)

    Since i don't want to spam my links many times to the SAME domain i click "remove duplicate domains".

    But Scrapebox trims the result to root domain, leaving then only
    blog .com

    PROBLEM: blog.com is the main URL and not a sub-URL where i can post comments!

    So every time when i "remove duplicate domain" it removes/filters the actual post/comment URLs and leaves only the main URL at the end and i cannot post to that!
     
  2. Maruk

    Maruk Power Member

    Joined:
    Jun 15, 2009
    Messages:
    562
    Likes Received:
    899
    Home Page:
    It seems that of clicking the dropdown button, you actually click the "Trim to root" button that is positioned exactly underneath the "Remove duplicate domains" button.

    Check that out.
     
  3. jheyslow

    jheyslow Senior Member

    Joined:
    May 11, 2010
    Messages:
    895
    Likes Received:
    718
    Occupation:
    trying new things

    I second to that. OP clearly click the "trim to root" not the "Remove duplicate domains". Please check it again op.
     
  4. Maruk

    Maruk Power Member

    Joined:
    Jun 15, 2009
    Messages:
    562
    Likes Received:
    899
    Home Page:
    Well, actually if the "Remove" menu expands, the button OP is looking for lays ontop of "Trim to Root".
    So it is either a mistake on his part or a bug in SB that makes him click the wrong button.
     
    Last edited: Mar 16, 2011
  5. Kickflip

    Kickflip BANNED BANNED

    Joined:
    Jan 29, 2010
    Messages:
    2,038
    Likes Received:
    2,465
    OR scrapebox is using the shortest entry available when it removes duplicate domains, so if the root domain is listed in the list of blogs, it will choose to keep the root domain instead of one of the postable pages.

    The only solution I can think of is to use the option which says like "Remove URLs not Containing" and remove any domains which don't have .html on the end if all the pages have .html. You can do the same with .php and combine the 2 filtered lists together after if needed. I guess theoretically you could do the same with ".com/" ".net/" ".org/" ".info/" because scrapebox doesn't display root domains with a trailing / after the tld.
     
    • Thanks Thanks x 2
  6. blackhaze

    blackhaze Power Member

    Joined:
    Jan 11, 2008
    Messages:
    661
    Likes Received:
    167
    Occupation:
    self made millionaire
    Location:
    in the matrix
    Home Page:
    guys, i am NOT retarded. I didnt click "trim to root".

    This (unwanted) trim happens not with all domains, i dont know what the criteria is.
    Kickflip, might check into this...sounds reasonable.
     
  7. andreyg13

    andreyg13 Jr. VIP Jr. VIP

    Joined:
    Nov 13, 2009
    Messages:
    915
    Likes Received:
    1,776
    Occupation:
    SEO
    Location:
    http://seoshark.org
    Home Page:
    Instead why don't you check for pr first and then go for the option: split duplicate domains. Then you should have several files each with unique domains.

    If you ask why check for pr, this is because it will split the high pr ones first and so on.
     
  8. scott8610

    scott8610 Junior Member Premium Member

    Joined:
    Apr 11, 2010
    Messages:
    100
    Likes Received:
    21
    Location:
    Charleston, SC
    Home Page:
    Yeah what Kickflip said. Since the urls are organized by name the domain homepage is usually put before the other urls and scrapebox will only keep the first url when removing dupe domains. Not sure exactly how to get around this but that's the problem.
     
  9. lacy1978

    lacy1978 Junior Member

    Joined:
    Jan 5, 2011
    Messages:
    154
    Likes Received:
    39
    You have the latest version of SB? Did this just start happening?
     
  10. HoNeYBiRD

    HoNeYBiRD Jr. VIP Jr. VIP

    Joined:
    May 1, 2009
    Messages:
    7,148
    Likes Received:
    8,140
    Gender:
    Male
    Occupation:
    Geographer, Tourism Manager
    Location:
    Ghosted
    you can do this w/o SB easily, just check out this thread by crazyflx:
    Code:
    http://www.blackhatworld.com/blackhat-seo/black-hat-seo-tools/251546-have-over-1-million-urls-need-remove-duplicate-urls-domains-here-you-go.html