[need help] remove subdomains of url list

kkvsam

Senior Member

Oct 11, 2009

938

570

• Jan 26, 2012

• #1

I've scraped huge list of URLs through SB. But there are lots of sub domains like abc.domain.com. I need to remove all sub domains and want to get main domains. Any one know how to do this?

Code:

abc.domain.com to domain.com

Thank you.

 

Last edited: Jan 26, 2012

Techkat

Failed to resolve dispute

Aug 14, 2011

170

98

• Jan 26, 2012

• #2

You can use Sick Scraper (free tool) for this task:

Code:

[B][URL]http://sickmarketing.com/beta/scraper.exe[/URL][/B]

 

kkvsam

Senior Member

Oct 11, 2009

938

570

• Jan 26, 2012
• Thread Starter Thread Starter

• #3

I've sick scraper. Then what?

Techkat said:

You can use Sick Scraper (free tool) for this task:

Code:

[B][URL]http://sickmarketing.com/beta/scraper.exe[/URL][/B]

Click to expand...

 

H

hpv222

Power Member

Feb 8, 2010

751

279

• Jan 26, 2012

• #4

regular expressions should help

what's easier for me though would be to import the list in access, using the dot (.) as a delimiter - then you will end up with three columns:
sub domain com
sub domain1 com

etc. - just copy the 2nd and the third and you'll end up with

domain com
domain1 com

then just replace the space with dot (.) It takes less than 10 seconds

 

kkvsam

Senior Member

Oct 11, 2009

938

570

• Jan 26, 2012
• Thread Starter Thread Starter

• #5

but if the site is abc.domain.com.uk, then what? I need to have domain.com.uk for those domains.

 

Techkat

Failed to resolve dispute

Aug 14, 2011

170

98

• Jan 26, 2012

• #6

kkvsam said:

I've sick scraper. Then what?

Click to expand...

Then this:

Sick Scraper -> Import Urls -> Trim to root domain

 

kkvsam

Senior Member

Oct 11, 2009

938

570

• Jan 26, 2012
• Thread Starter Thread Starter

• #7

it is not removing . Check your self.

Techkat said:

Then this:

Sick Scraper -> Import Urls -> Trim to root domain

Click to expand...

 

milans

Regular Member

Mar 26, 2011

382

161

• Jan 26, 2012

• #8

http://www.scrapebox.com/free-dupe-remove

Maybe you can remove it with scrapebox dupe remove?

 

kkvsam

Senior Member

Oct 11, 2009

938

570

• Jan 26, 2012
• Thread Starter Thread Starter

• #9

No. Dude. I can remove dupe with SB. What I need is remove subdomains from the URL list and get the root domain like below.

Code:

abc.domain.com  --->  domain.com
abc.domain.co.uk ---> domain.co.uk

 

Techkat

Failed to resolve dispute

Aug 14, 2011

170

98

• Jan 26, 2012

• #10

kkvsam said:

it is not removing . Check your self.

Click to expand...

You are right.
It only works for
domain.com/subdomain

 

spoon22

Regular Member

May 7, 2011

323

155

• Jan 26, 2012

• #11

kkvsam said:

but if the site is abc.domain.com.uk, then what? I need to have domain.com.uk for those domains.

Click to expand...

Lol, he gave you the solution and you don't see it?

Just copy 2,3,4 and replace space with dot in notepad (STRG+H)