1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

Scrapebox - Do you remove duplicate domain or urls?

Discussion in 'Black Hat SEO Tools' started by macdonjo3, Jul 31, 2010.

?

What do you remove?

  1. I remove duplicate domains.

    0 vote(s)
    0.0%
  2. I remove duplicate URLs.

    0 vote(s)
    0.0%
  1. macdonjo3

    macdonjo3 Jr. VIP Jr. VIP Premium Member

    Joined:
    Nov 8, 2009
    Messages:
    5,566
    Likes Received:
    4,317
    Location:
    Toronto
    Home Page:
    Just curious what you guys remove.
     
  2. jartym

    jartym Junior Member

    Joined:
    May 22, 2008
    Messages:
    120
    Likes Received:
    8
    I am removing dupe domains.
     
  3. LyNHS

    LyNHS Regular Member

    Joined:
    Jul 20, 2010
    Messages:
    282
    Likes Received:
    98
    Occupation:
    Google AdSense
    Home Page:
    Hey!

    I trim to root and remove duplicated URLs ...

    Well saying that, it really depends on what I'm harvesting lol. If I'm harvesting URLs from a custom Footprint, then I'm more than likely to remove URLs, if it's for WordPress blogs, then Domains ...

    :)
     
  4. G0D0VERY0U

    G0D0VERY0U Regular Member

    Joined:
    Apr 6, 2010
    Messages:
    439
    Likes Received:
    282
    BE - all urls
    WP - all domains
     
  5. sunshyne

    sunshyne Regular Member

    Joined:
    Jun 17, 2010
    Messages:
    359
    Likes Received:
    54
    Location:
    localhost
    remove both dup domains and urls. Like posted above get the meat
     
  6. corky20

    corky20 Regular Member

    Joined:
    Oct 1, 2009
    Messages:
    291
    Likes Received:
    68
    Location:
    Northern Ireland
    Home Page:
    Remove duplicate URLS, check pagerank, keep good PR links, then remove duplicate domains!
     
    • Thanks Thanks x 1
  7. mandom

    mandom Regular Member

    Joined:
    May 4, 2010
    Messages:
    373
    Likes Received:
    120
    Depends on why I'm harvesting. I harvest for many different purposes. Generally I just get rid of duplicate URLs.
     
  8. Italian Hawk

    Italian Hawk Power Member

    Joined:
    Apr 21, 2009
    Messages:
    589
    Likes Received:
    126
    As with everyone else, it depends. For me, I usually remove duplicate domains unless the scrape was a lower number of blogs in which case I'll usually leave them and hope they both get accepted which isn't too likely but whatever.
     
  9. xxmalouinxx

    xxmalouinxx Junior Member

    Joined:
    May 9, 2010
    Messages:
    153
    Likes Received:
    37
    Location:
    Canada
    Personally, I always remove the duplicate domain. Except on some rare occasion where I want more than 1 domain website URL with high PR.
     
  10. macdonjo3

    macdonjo3 Jr. VIP Jr. VIP Premium Member

    Joined:
    Nov 8, 2009
    Messages:
    5,566
    Likes Received:
    4,317
    Location:
    Toronto
    Home Page:
    Removing the duplicate domains does remove the duplicate urls.
     
  11. shylesson

    shylesson Power Member

    Joined:
    Jan 10, 2008
    Messages:
    665
    Likes Received:
    2,090
    Location:
    ‹^›‹(•¿•)›‹^›
    think he meant he uses both options in different situations like the poster above him. i like this too because the remove domains one will remove all urls from the domain and leave only have to play around with one bl on the site and not risk over-spamming a single site in WP cases making it harder to get chit to stick, but i like the remove urls one when I want the different urls, but not the dupes (obviously) without caring about them being on the same domain as in how the poster posted. =P
     
  12. macdonjo3

    macdonjo3 Jr. VIP Jr. VIP Premium Member

    Joined:
    Nov 8, 2009
    Messages:
    5,566
    Likes Received:
    4,317
    Location:
    Toronto
    Home Page:
    Question: What does trim to root do?
     
  13. Scripteen

    Scripteen Elite Member

    Joined:
    Sep 19, 2009
    Messages:
    1,811
    Likes Received:
    1,918
    Home Page:
    That's it.
     
  14. raj5151

    raj5151 Regular Member

    Joined:
    Jan 25, 2009
    Messages:
    242
    Likes Received:
    141
    Location:
    BHW
    Home Page:
    I remove duplicate urls
     
  15. dungxvn

    dungxvn Junior Member

    Joined:
    Nov 11, 2009
    Messages:
    182
    Likes Received:
    47
    Home Page:
    You will get abc.com from abc.com/abcd.{html|php|htm|asp|etc}
     
    • Thanks Thanks x 1
  16. macdonjo3

    macdonjo3 Jr. VIP Jr. VIP Premium Member

    Joined:
    Nov 8, 2009
    Messages:
    5,566
    Likes Received:
    4,317
    Location:
    Toronto
    Home Page:
    Thanks, and do you find this useful or what?
     
  17. dungxvn

    dungxvn Junior Member

    Joined:
    Nov 11, 2009
    Messages:
    182
    Likes Received:
    47
    Home Page:
    It's really useful to spam. For example, you find abc.com/abc.html is not moderated ( you found the link right after posting). You will want to get more post from this blog as you know they are not moderated too. You will find this is very useful when you have huge urls list or providing services
     
  18. gahoachma

    gahoachma Junior Member

    Joined:
    Dec 16, 2009
    Messages:
    126
    Likes Received:
    21
    Home Page:
    Usually I just remove dupe urls. Shotgun technique I guess. Bigger then target the more I have to hit. Side of a barn.. lol