turbopugsleylx
Elite Member
- Jun 6, 2008
- 3,424
- 988
Here's the issue I am running into guys....
So lets say a have a scraped url list of 100k and I sort the list by PR....problem is some domains may have a URL that is a PR5 and one URL that is a PR1....and if I do eliminate duplicate domains...it could eliminate both the PR URL's and keep the PR N/A link....
How in the heck can I eliminate all the other URL's except the highest PR URL from that particular domain?
Since I do my posting in batches based by PR how in the heck can I eliminate this issue where I end up posting to the same domain 100 times (with diff URLS mind you...but maybe the same base domain...ie im using my domain and sub domains)....
Any help or advice on this issue would be appreciated!
Turbo
So lets say a have a scraped url list of 100k and I sort the list by PR....problem is some domains may have a URL that is a PR5 and one URL that is a PR1....and if I do eliminate duplicate domains...it could eliminate both the PR URL's and keep the PR N/A link....
How in the heck can I eliminate all the other URL's except the highest PR URL from that particular domain?
Since I do my posting in batches based by PR how in the heck can I eliminate this issue where I end up posting to the same domain 100 times (with diff URLS mind you...but maybe the same base domain...ie im using my domain and sub domains)....
Any help or advice on this issue would be appreciated!
Turbo