URL List Editor Needed

T

TragikGod

Registered Member
Joined
Mar 13, 2014
Messages
89
Reaction score
27
If someone can help me with this I would be VERY appreciative.

I'm hoping someone can help me out with a program that is powerful enough to...
1. Load a URL list. (List 1)
2. Load a 1mill+ URL list. (List 2)
3. Remove (List 2) from (List 1)
4. Save (List 3) which is (List 1 - List 2) = List 3

OR

Something like Scrapebox's exclude list but on a bigger scale.

I looked all over the internet only to find editiors that just find the difference and highlight it and UNIX stuff.

Thank you in advance.
 
Advertise on BHW
Ubot can do this, might have to make the larger list smaller depending on the memory your computer has tho.
 
What do you want it written in - Ubot in its default form will most definitely choke and die if you throw that many links at it at one go, perhaps with the decent plugins it may work.

If you want to run scripts on Linux, then Perl will do it in a heartbeat....

Give more details and you will probably get better answers :)
 
I have one simple script which all my friends and me use very often.

Free. Open Source
bit.ly/1voiZU5

you need to install python 2.7 on your machine to run it.

Script works with 3 files
  • i.txt - input data
  • d.txt - delete from input data
  • o.txt - output data

Also it can delete not unique lines.
 
WMAid said:
I have one simple script which all my friends and me use very often.

Free. Open Source
bit.ly/1voiZU5

you need to install python 2.7 on your machine to run it.

Script works with 3 files
  • i.txt - input data
  • d.txt - delete from input data
  • o.txt - output data

Also it can delete not unique lines.
Click to expand...


It is against the rules to link directly to a download and hide the link with a URL shortener and you also need to provide a VirusTotal report

https://www.virustotal.com/en-gb/fi...1ce9e82fedb5a06ddae2ce76/analysis/1411812007/
 
W130SN said:
It is against the rules to link directly to a download and hide the link with a URL shortener and you also need to provide a VirusTotal report
Click to expand...

Thanks for the virus total report.
I'm newbie here and still can't pos tnormal links. URL shortener helped me to help topic starter.
I don't mind if moderators will remove or change my link.
 
Treeofl1 said:
Use excel. Just paste all the URLs in one column, and then click "Remove Duplicates'. Then simply copy the remaining URLs into notepad and save it.

If excel can't handle that many URLs, try Notepad++ with the TextFX plugin: http://www.cathrinewilhelmsen.net/2012/05/16/notepad-remove-duplicates-remove-blank-lines-and-sort-data-in-one-operation/
Click to expand...
Excel is limited to 1m rows unfortunately :(
TextFX is awesome though, recommended!
 
Advertise on BHW
You must log in or register to reply here.
Sign up now!

Main Menu

Home Main Forum List Black Hat SEO White Hat SEO BHW Newbie Guide Blogging Black Hat Tools Social Media Downloads

Marketplace

Account Selling Content / Copywriting EBooks / Methods Hosting Misc Proxies For Sale SEO - Link building SEO - Packages Social Media Social Media - Panels Web Design More

Making Money

Affiliate Programs Hire a Freelancer IM Journeys Making Money Pay Per Click Site Flipping Want to Buy

BlackHat World

Dispute Resolution Forum Suggestions & Feedback Introductions Lounge
Back
Top